What is the replication crisis within psychology?
Jason Brien.
The inability to successfully replicate the results of previous scientific studies relating to core psychological concepts such as social priming, ego depletion, episodic memory, facial-feedback and temporal judgement, to name just a few, appears to becoming an increasing problem within the field of psychology. The inability to replicate previous studies and findings puts fundamentally important concepts under a cloud of suspicion and confusion and no longer can researchers confidently assert the validity or reliability of these concepts.
This inability to replicate previous findings has become known as the replication crisis and it is starting to jeopardise the integrity of psychology as a ‘scientific’ discipline. Psychology as a discipline has strongly emphasized the importance of rigid, valid, reliable and ethical testing of hypotheses and theories throughout its ranks. It was this emphasis on the scientific approach which allowed psychology to move out from under the wings of philosophy and become a ‘scientific’ discipline in its own rights. The magnitude to which failure to replicate has disseminated throughout all domains of psychology has stunned both psychology and its researchers into thought and action.
Whilst it is true that the majority of the failures to replicate have occurred primarily within the social psychology domain, the replication crisis has been demonstrated to exists among other psychology disciplines too. For example, a study conducted by conducted by the Open Science Collaboration sought to successfully replicate one hundred statistically significant studies which had been previously published within three top-tier psychology journals. Surprisingly, the study demonstrated that only slightly more than a quarter of replications yielded a significant result compared to the original studies which all reported and published significant results.
What is causing the replication crisis?
1.Publication bias; Publication bias can exist as either a failure by psychology journals to publish replication studies and/or favouritism by psychology journals towards publishing only those studies which have achieved statistically significant results. Yeung (2017) deals with the first issue regarding journals choosing to publish original studies over replication studies. Yeung’s findings showed that most journals did not unambiguously welcome replications. The journals preferred ‘original’ and ‘unique’ studies.
2.Publication bias 2; Studies which achieved statistically significant results are published more often than the studies which found no statistically significant results. It has been argued that this difference stems from the fact that studies with statistically significant results are favoured by editors, are more enjoyed by peer reviewers, and will likely elicit more citations post publication. Researchers often lack motivation to write up and submit studies which achieved null results given the unlikelihood of publication and recognition in the form of citations.
3.Questionable research practices; Given that funding for any given research is usually dependent upon successful studies, significant results and the number of papers which can be successfully published, it is clear to see how researchers may be coerced or feel pressured to consistently provide and publish statistically significant results by any means necessary. Since realistic data or methodology practices have been falsely reported in original studies, the ability to replicate the original findings becomes exponentially more difficult.
4.Unrelenting desire for success; It can be extremely difficult to get a paper published in a well-recognised and respected academic journal. This is because each paper that is submitted must be peer reviewed by a number of people. If the paper is not given the green light, it is not published. This competition for limited places, coupled with a strong desire for success, fame or recognition can force some researchers to ‘alter’ their findings in order to improve their chances of being published.
5.Changes in ethics and morals; Sometimes previous studies cannot be replicated because the ethics and morals of today’s society will not permit such studies. All research proposals must be approved by an ethics committee before a study can even begin. Pavlov’s conditioning experiments on dogs or Albert Banduras little Alberts’s fear conditioning experiment for example would never be granted permission in todays climate. They are just too unethical and inhumane.
I don’t think that people should completely lose faith in historical findings. I think it is still important to understand and appreciate the theory behind such concepts as ego depletion, social priming or episodic memory despite their findings not being replicated today. All information serves a purpose even if that purpose is simply to discuss the validity of the information. That is, if we recognise that ego depletion is not contributing towards certain mental health concerns then we can continue to look for what is contributing.
Resources.
Agnoli, F., Wicherts, J. M., Veldkamp, C. S., Albiero, P., & Cubelli, R. (2017). Questionable research practices among Italian research psychologists. Plos ONE, 12(3), 1-17.
Anderson, S. F., & Maxwell, S. E. (2017) Addressing the “Replication Crisis”: Using Original Studies to Design Replication Studies with Appropriate Statistical Power. Multivariate Behavioral Research, 52:3, 305-324.
Baldwin, S. A. (2017). Improving the rigor of psychophysiology research. International Journal of Psychophysiology, 111; 5-16.
Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016). Editorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly. Journal of Business and Psychology, 31(3); 323-338.
Boeck, P., & Jeon, M. (2018). Perceived crisis and reforms: Issues, explanations, and remedies. Psychological Bulletin, 144(7), 757-777.
Burger, J. M. (2009). Replicating Miligram: Would people still obey today? American Psychologist, 64; 1-11.
Brandt, M. J., Jzerman, I., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorella, R., Grange, J. A., Perugini, M., Spies, J. R., and Veer, A. V. (2014). The Replication Recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50 (1); 217-224
Chopik, W. J., Bremner, R. H., Defever, A. M., & Keller, V. N. (2018). How (and Whether) t to Teach Undergraduates About the Replication Crisis in Psychological Science. Teaching of Psychology, 45(2); 158-163.
Crandall, C. S., & Sherman, J. W. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66; 93-99.
Yeung, AW. K. (2017). Do neuroscience journals accept replications? A survey of literature. Frontiers in Human Neuroscience, 11(468).