Written by: Michelle Rivers, Rachael Soicher, and Dr. Morton Ann Gernsbacher
- What is transparency in research, and why does it matter?
Research transparency, also referred to as “open science,” is the process of making research and its dissemination accessible to all levels of society. Practices that increase research transparency accelerate progress toward solving the persistent problems of society. For example, you may have heard about the “replication crisis” in psychology and other sciences. Engaging in open science practices can help make science more reproducible and thus more trustworthy.
As the field of psychology works toward greater research transparency, how can you get involved? In this post, we talk about some baby steps you can take as a graduate student, and we address some common challenges you may face along the way.
2. How can you get started?
Below we’ve crafted a short list of ways you can make your research more transparent. Think of baby steps. Don’t feel like you have to do all of them at once!
- Pre-register your study design, hypotheses, and planned analyses. Place them in a locked file (hardcopy or electronically) or post them on free websites like AsPredicted or Open Science Framework.
- Make your study materials, data, preprints, and post-prints publicly available online, such as on your lab’s website or on Open Science Framework.
- Avoid “Questionable Research Practices,” such as selectively reporting dependent measures/entire experiments, revising your sample size after viewing your data, or so-called “HARKing” (hypothesizing after the results are known).
- Take advantage of research collaborations. Do you have or need access to a particular demographic sample? Check out StudySwap or Psychological Science Accelerator, online platforms for research replication, collaboration, and resource exchanges.
- Use free and open-source software so it’s easier for others to reproduce your exact methods and analyses. For example, OpenSesame and PsychoPy are open-source applications for running psychology experiments, LimeSurvey is open-source survey software, G*Power is free software for running power analyses, and R is free software for statistical computing and graphics
- Practice disseminating your research to diverse audiences. For example, one of us, Michelle, manages a blog (cogbites.org) run by early-career academics that focuses on translating scientific research about the study of mental processes to a general audience. To learn more about science communication and outreach, consider applying to attend ComSciCon, a workshop for graduate students, by graduate students.
- Stay informed! If you have an advanced stats or methods course in your program, encourage the instructor to include recent readings on research transparency. One of us, Morton, teaches an open-access active-learning Research Methods course, and you can find other teaching resources here. Consider starting a reading group with other students in your program!
3. What impact can baby steps have?
As famed physicist and Nobel Prize winner, Richard Feynman once said, “You must not fool yourself” — meaning you must not misremember what you did and did not plan, hypothesize, and predict – and “you are the easiest person to fool,” because you have the most investment in how your study comes out!
4. Who can you connect with to feel community?
Us! And the Society for Improving Psychological Science, the Berkeley Initiative for Transparency in the Social Sciences, Twitter, and other graduate students, perhaps in your circle of friends and colleagues who are also looking for camaraderie.
5. What if you encounter resistance from your advisor?
Perhaps your advisor has heard only hearsay, so taking them through the baby steps might help. Consider, for example, having a planning meeting that lays out some of these baby steps. Offer to do any of the “extra work,” which often isn’t that much extra.
6. What if working more transparently feels odd compared with how you’ve been trained?
Again, think of baby steps and ways that transparency might have already been built into your (or your lab’s) culture (e.g., announcing proposed designs, data analysis plans, or predictions at a lab meeting; sharing data or materials with younger students; posting journal article PDFs on your lab’s website).
7. Doesn’t working more transparently take too long?
It doesn’t have to! Pre-registering is simply documenting your work (and good documentation is a love letter you send to your future self). Registered Report journal articles can flow through the publication pipeline more quickly than endless cycles of revise and resubmit. One of us, Raechel, already has two publications pre-accepted! Preparing materials and data for posting (on OSF or the like) is an investment in your own future research.
8. Where is the reward structure for working more transparently?
We’re already seeing some job ads prioritize research transparency along with a handful of awards. Plus, making your work more transparent can increase the impact, value, and discoverability of your research. But the greatest reward is personal – knowing that you’re contributing to improving psychological science and public trust in science more broadly.
9. What if your ideas change or you learn new skills?
You can propose additional hypotheses, further analyses, or future research. You can also engage in exploratory research. Just be sure to distinguish between exploratory and confirmatory analyses (and remember that null hypothesis testing relies on a confirmatory approach).
10. What if someone finds you’ve made a mistake?
We all do! It’s ok! And preparing materials, data, and analyses for transparency is the best way to find those mistakes yourself (or consider swap-checking with a friend or colleague).
——
We hope you’ll become more enthusiastic about working transparently, but if you find your enthusiasm going over the edge… Reach out to us!
The article gives reason to think. Something to think about.