Beginnings of a Revolution
Around the year 2010, a series of events accumulated in psychology that, taken individually, could have been dismissed as isolated incidents, but that together painted a negative picture of the science. These cases fall into two categories: the Stapel affair was a clear case of fraud through the “invention” of data or the reporting of studies that were never conducted. The other cases were not clear-cut fraud; instead, they involved methods that were, at the time, accepted within the field, used to produce results.
The Stapel Affair
By chance, in 2011 early-career researchers discovered that the data for a study by their colleague, Diederik Stapel, had never been collected by anyone. It had been invented, or fabricated. As of July 2024, 58 of Stapel’s journal articles have been identified and retracted for containing fabricated or manipulated data.1 Scientific institutions such as peer review, whose purpose is quality assurance, had failed. Since the incident, several further cases have come to light, in part through renewed analysis of the data from the studies in question (O’Grady, 2021), and often through whistleblowers—that is, people with knowledge of the wrongdoing who wish to remain anonymous for their own protection. Surveys of researchers in the Netherlands have found that up to 10% of all researchers have engaged in the fabrication or manipulation of data (Gopalakrishna et al., 2021). It should be noted here that fabricated data tend to make studies especially innovative, surprising, or clear-cut—properties that increase the likelihood of publication in a journal.
Bem: Feeling the Future
A short time later, Daryl Bem—known for foundational psychological and philosophical theories such as self-perception theory (Bem, 1967)—published the finding that people can predict the future (Bem, 2011). More precisely, some people, under certain conditions, can predict the future. Across nine studies, the research group found that people could make predictions about erotic images. The results were published in the highly regarded Journal of Personality and Social Psychology. For many psychologists, it was immediately clear: either fundamental assumptions of their worldview were wrong (“people cannot predict the future”), or something was wrong with the results. Several researchers attempted to explain how these results had come about. Analyses of the published data using alternative statistical methods arrived at the same conclusion (Wagenmakers et al., 2011), yet replications by independent researchers failed (Muhmenthaler et al., 2022; Robinson, 2011; Roe et al., 2012).
Bargh: Influence through Priming
His studies were celebrated in marketing and sold as neuroscientific insights: people who drink a hot beverage (compared to a cold one) rate other people as “warmer” (more generous, more considerate) (Williams & Bargh, 2008). People who solve anagrams related to old age (e.g., “TREEMTIRNE” instead of “RETIREMENT,” or “YARG” instead of “GRAY,” or, to try it yourself, “NOISER”) subsequently walk more slowly (Bargh et al., 1996). Many of these studies were not replicated: researchers noticed, in the anagram study, that Bargh and colleagues had measured walking time with stopwatches while knowing which participants had solved the “old age” words and which had solved the neutral anagrams—even though every first-year psychology student learns that this should not be the case, and that experimenters should be “blind” to the purpose of the study and to which condition each participant was assigned. In their replication (Doyen et al., 2012), Doyen and colleagues first measured time themselves, just as Bargh et al. had in the original study: this problematic measurement approach produced the same result. When they instead had time measured using light barriers, without the experimenters knowing which hypothesis was being tested or which participant was solving which anagrams, the effect could not be replicated.
In behavioral biology, Crabbe et al. (1999) showed much earlier that findings on the influence of genes on behavior depended strongly on which laboratory the mice were tested in. Although calls for improvements to the research process have recurred ever since (Kafkafi et al., 2018), a revolution comparable to that in psychology never materialized.
A somewhat different problem arose in the course of sequencing human DNA: there, companies filed for intellectual property rights while the research process was still underway. The negative social and scientific consequences were discussed early on (Moore, 2000) and remain part of scientific and political debate to this day.
Problems with reproducibility—that is, confirming published results based on the same data—are also known from preclinical animal studies of various drugs (biology), cancer research (medicine), genetics (biology), treatments for ALS (neurology), and widely cited medical interventions (medicine) (Royal Netherlands Academy of Arts and Sciences, 2018, p. 22, Table 1).
References
Retracted articles can be searched by topic, author, journal, etc. using the Retraction Database: http://retractiondatabase.org/.↩︎