Search Resources

4 Results

View
Selected filters:
  • Michael C. Frank
Building a collaborative Psychological Science: Lessons learned from ManyBabies 1
Only Sharing Permitted
CC BY-NC-ND
Rating

The field of infancy research faces a difficult challenge: some questions require samples that are simply too large for any one lab to recruit and test. ManyBabies aims to address this problem by forming large-scale collaborations on key theoretical questions in developmental science, while promoting the uptake of Open Science practices. Here, we look back on the first project completed under the ManyBabies umbrella – ManyBabies 1 – which tested the development of infant-directed speech preference. Our goal is to share the lessons learned over the course of the project and to articulate our vision for the role of large-scale collaborations in the field. First, we consider the decisions made in scaling up experimental research for a collaboration involving 100+ researchers and 70+ labs. Next, we discuss successes and challenges over the course of the project, including: protocol design and implementation, data analysis, organizational structures and collaborative workflows, securing funding, and encouraging broad participation in the project. Finally, we discuss the benefits we see both in ongoing ManyBabies projects and in future large-scale collaborations in general, with a particular eye towards developing best practices and increasing growth and diversity in infancy research and psychological science in general. Throughout the paper, we include first-hand narrative experiences, in order to illustrate the perspectives of researchers playing different roles within the project. While this project focused on the unique challenges of infant research, many of the insights we gained can be applied to large-scale collaborations across the broader field of psychology.

Subject:
Social Science
Material Type:
Reading
Author:
Casey Lew-Williams
Catherine Davies
Christina Bergmann
Connor P. G. Waddell
Jessica E. Kosie
J. Kiley Hamlin
Jonathan F. Kominsky
Krista Byers-Heinlein
Leher Singh
Liquan Liu
Martin Zettersten
Meghan Mastroberardino
Melanie Soderstrom
Melissa Kline
Michael C. Frank
Date Added:
11/13/2020
Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition
Unrestricted Use
CC BY
Rating

Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.

Subject:
Information Science
Material Type:
Reading
Provider:
Royal Society Open Science
Author:
Alicia Hofelich Mohr
Bria Long
Elizabeth Clayton
Erica J. Yoon
George C. Banks
Gustav Nilsonne
Kyle MacDonald
Mallory C. Kidwell
Maya B. Mathur
Michael C. Frank
Michael Henry Tessler
Richie L. Lenne
Sara Altman
Tom E. Hardwicke
Date Added:
08/07/2020
A Practical Guide for Transparency in Psychological Science
Unrestricted Use
CC BY
Rating

The credibility of scientific claims depends upon the transparency of the research products upon which they are based (e.g., study protocols, data, materials, and analysis scripts). As psychology navigates a period of unprecedented introspection, user-friendly tools and services that support open science have flourished. However, the plethora of decisions and choices involved can be bewildering. Here we provide a practical guide to help researchers navigate the process of preparing and sharing the products of their research (e.g., choosing a repository, preparing their research products for sharing, structuring folders, etc.). Being an open scientist means adopting a few straightforward research management practices, which lead to less error prone, reproducible research workflows. Further, this adoption can be piecemeal – each incremental step towards complete transparency adds positive value. Transparent research practices not only improve the efficiency of individual researchers, they enhance the credibility of the knowledge generated by the scientific community.

Subject:
Psychology
Material Type:
Reading
Author:
Alicia Hofelich Mohr
Frederik Aust
Gustav Nilsonne
Hans Ijzerman
Henrik Danielsson
Johannes Breuer
Michael C. Frank
Olivier Klein
Tom E. Hardwicke
Wolf Vanpaemel
Date Added:
06/29/2018
A Practical Guide for Transparency in Psychological Science
Unrestricted Use
CC BY
Rating

The credibility of scientific claims depends upon the transparency of the research products upon which they are based (e.g., study protocols, data, materials, and analysis scripts). As psychology navigates a period of unprecedented introspection, user-friendly tools and services that support open science have flourished. However, the plethora of decisions and choices involved can be bewildering. Here we provide a practical guide to help researchers navigate the process of preparing and sharing the products of their research (e.g., choosing a repository, preparing their research products for sharing, structuring folders, etc.). Being an open scientist means adopting a few straightforward research management practices, which lead to less error prone, reproducible research workflows. Further, this adoption can be piecemeal – each incremental step towards complete transparency adds positive value. Transparent research practices not only improve the efficiency of individual researchers, they enhance the credibility of the knowledge generated by the scientific community.

Subject:
Psychology
Material Type:
Reading
Author:
Alicia Hofelich Mohr
Frederik Aust
Gustav Nilsonne
Hans Ijzerman
Henrik Danielsson
Johannes Breuer
Michael C. Frank
Olivier Klein
Tom E. Hardwicke
Wolf Vanpaemel
Date Added:
06/04/2020