Author Archives: Marianne Chirica

The Argument Against P-Values

There is concern that a substantial proportion of published research presents largely false findings (Ioannidis, 2005). This problem, in part, stems from social science’s reliance on null hypothesis statistical testing (NHST) given the incentive to achieve statistical significance (e.g., publications, grant funding). Research in the social sciences has historically adopted a Frequentist perspective, primarily reporting results using a dichotomous reject or non-reject decision strategy based on whether some test statistic surpasses a critical value and results in a statistically significant p-value (usually p > 0.05). Although useful in several ways, p-values are largely arbitrary metrics of statistical significance (Greenland et al., 2016), and they are often used incorrectly (Gelman, 2016). The use of p-values encourages a binary mindset when analyzing effects as either null or real, however, this binary outlook provides no information on the magnitude or precision of the effect. P-values can vary dramatically based on the population effect size and the sample size (Cumming, 2008). This reliance on an unstable statistical foundation has been discussed in the literature (Wasserstein, 2016), and while some journals have taken matters into their own hands (for example, Basic and Applied Social Psychology banned p-values and NHST), the field of psychology has largely failed to address the concerns raised by the use of NHST.                                     

Research is moving towards adopting new statistics as best practice, relying instead on estimations based on effect sizes, confidence intervals, and meta-analysis (Cumming, 2014). We, as graduate students in training, are in a position to push towards thinking in terms of estimations and away from dichotomously constrained interpretations. In contrast to the binary nature of p-values, a confidence interval is a set of plausible values for the point estimate. Although perhaps wide, the confidence interval accurately conveys the magnitude of uncertainty of the point estimate (Cumming, 2014), as well as the level of confidence in our results. For example, a 95% confidence interval that includes values for a population mean, μ, indicates 95% confidence that the lower and upper limits are likely lower and upper bounds for μ. The APA Publication Manual (APA, 2020) specifically outlines recommendations to report results based on effect size estimates and confidence intervals, rather than p-values. P-values are not well suited to drive our field forward in terms of precision and magnitude of estimates. Researchers should therefore focus on advancing the field by gaining an understanding of what the data can tell us about the magnitude of effects and the practical significance of those results. It is important for graduate students to adopt practices to produce reproducible and reliable research. One way to do so is to move beyond p-values.

How to move beyond p-values:

  • Prioritize estimation instead of null hypothesis testing or p-values
    • Formulate research questions in terms of estimation. Ex: How large is the effect of X on Y; to what extent does X impact Y?
  • Report confidence intervals and corresponding effect sizes
  • Include confidence intervals in figures (preferred over standard error bars)
  • Make interpretations and conclusions based on the magnitude of the effects rather than a dichotomous decision based on “statistical significance”

References                                                     

American Psychological Association. (2020). Publication manual of the American Psychological Association 2020: the official guide to APA style (7th ed.). American Psychological Association.

Cumming, G. (2008). Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science, 3, 286– 300. doi:10.1111/j.1745-6924.2008.00079.x     

Cumming, G. (2014). The new statistics: why and how. Psychological science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966

Gelman, A. (2016). The problems with p-values are not just with p-values. The American Statistician, 70(10).

Greenland, S., Senn, S.J., Rothman, K.J. et al. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 31, 337–350. https://doi.org/10.1007/s10654-016-0149-3                                          

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. Retrieved from http:// www.plosmedicine.org/article/info:doi/10.1371/journal .pmed.0020124

Wasserstein, R.L., & Lazar, N.A. (2016). The ASA’s Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70:2, 129-133, DOI: 10.1080/00031305.2016.1154108

Written by Marianne Chirica, an APAGS Science Committee member and a third-year graduate student in the Psychological and Brain Sciences Ph.D. program at Indiana University. Feel free to reach out to Marianne with any questions you may have!

Mentorship as a Grad Student: How and Why

Mentoring is a dynamic, collaborative relationship wherein a mentor and mentee work together to facilitate a mentee’s professional development and success. As a first-generation American, my success in academia would not have been possible without the help of several mentors along the way. My current mentoring efforts are inspired by positive experiences with my own mentors, and the hope that I can pay it forward to others. Becoming a mentor can help disseminate knowledge, foster an environment for psychological safety, learning, and development, reveal the “hidden curriculum” of graduate school, and enhance diversity in higher education.

Mentoring Prospective Graduate Students

As a graduate student, there are several opportunities to become a mentor. One opportunity is during the graduate school application process. There are many programs designed to connect eager undergraduates and post-baccalaureates with mentors during the graduate school application process. Some are within certain schools or types of programs (e.g., Social Psychology; School Psychology), for underrepresented individuals (e.g., NextGen Psych Scholars; Project Short), or just general programs (e.g., APSSC, APA Division 19). The structure and level of commitment will vary by the program and by each mentor/mentee relationship. Typically, the mentor will meet a couple of times with the mentee, provide helpful materials for crafting personal statements and CVs, and edit materials. Particularly salient for underrepresented mentees, the mentor can help select graduate programs that are a good match and identify scholarships and funding opportunities. If that sounds too time intensive, there are other ways to help prospective PhD psychology applicants. See Application Statement Feedback Program (ASFP).

Mentoring Undergraduate and Younger Graduate Students

A more traditional mentorship role in graduate school is to mentor undergraduate students in your research lab. Undergraduates often help with literature reviews, running participants, or other miscellaneous research tasks. As a mentor, you could help lead journal clubs or provide direct research opportunities within one of your research projects, carefully observing and guiding them throughout the process. For example, I am currently mentoring an undergraduate student involved in a systematic review I am working on. As a mentor, I provide didactic instruction on types of reviews and guidance on the research process.

As a graduate student, you might also serve another role as a teaching assistant. Teaching assistants have a more formal teaching relationship with their students but can still provide mentorship. This can look like holding office hours and meeting one-on-one with students who show interest in your research or in a PhD program.

You can also mentor students in their early graduate school career. As you progress through the program, you gain valuable knowledge not only in your selected discipline, but also in how to be a graduate student. Younger students may benefit from any wisdom, tips, and skills you can pass on. These topics can vary from time management skills, course selections, clinical practicum advice, advice on advocating for oneself, and handling rejection. Things that may seem like second nature now, may be valuable wisdom to a first-year graduate student.

Irrelevant of the type of mentoring relationship, mentees are eager to learn from you. You can disseminate information about the field (e.g., different types of paths post undergrad), research (e.g., how to go about starting a literature review), classes (e.g., which classes to take to best prepare for graduate school or for a certain degree), or other areas that may help your mentee grow professionally. Although it may seem challenging, mentoring is a rewarding experience, fostering a collaborative relationship that benefits both individuals involved!

Want to become a mentor?

  1. Sign up to mentor students applying to graduate programs (see links above).
  2. Reach out to your advisor and ask if you can mentor undergraduates.
  3. Take initiative and offer to help younger graduate students.

Written by Marianne Chirica, an APAGS Science Committee member and a second-year graduate student in the Psychological and Brain Sciences Ph.D. program at Indiana University. Feel free to reach out to Marianne with any questions you may have!