Conclusion and Outlook
Researchers work under considerable pressure, and the academic system rewards innovation and clear-cut results more than it rewards care and openness. In the social sciences, more and more researchers are joining forces to address these problems. Emerging from the trust crisis, large parts of psychology already find themselves in a renaissance. Discussions are being held every day, at the personal, institutional, and political level, about how science should be conducted. It has become clear that, unlike some economic markets, science does not have an “invisible hand” that brings everything into balance – instead, it is researchers themselves who actively work toward the self-correction of science. Before an intervention to improve research can take place, however, an inventory is first needed (where are improvements necessary?), followed by observation. This book offers an inventory of problems and proposed solutions. It is now the role of meta-science, that is, science about science, to examine which approaches work, and how well.
Excellent Research Is Open Research
In some areas, Open Science has developed from something positive that was not missed when absent into something taken for granted. In the discourse, this is occasionally referred to as the Open Science Kano model. While this does not hold for all facets of Open Science (for example, not for citizen science), one thing is clear: enabling a (rapid) growth of knowledge requires that knowledge be verifiable. This is only possible with open research reports (Open Access), open data (Open Data), and verifiable calculations (Open Code, reproducibility).
What Has Improved?
I hope the chapters on solutions have made clear that there is no problem for which at least one proposed solution does not already exist, and that these solutions are already being implemented in diverse ways (Korbmacher et al., 2023). For example, guidelines have been developed for replication research, and replications are increasingly seen not as personal attacks but as a useful tool (Nosek et al., 2022). Stricter requirements for reporting statistical tests have had a marked effect on their traceability (Giofrè et al., 2023), preregistration has reduced p-hacking (Brodeur et al., 2024), and in some cases extremely high replication rates have been achieved using modern methods (Protzko et al., 2023; see also the critique by Bak-Coleman & Devezer, 2023). The proportion of researchers in psychology using these approaches rose sharply between 2010 and 2020, from 49% to 87% (Ferguson et al., 2023).
What Remains to Be Done?
In many disciplines, active engagement with Open Science is still pending. While social and personality psychology play a pioneering role among the social sciences, engagement in consumer psychology, or beyond psychology in medicine, education science, or sociology, only began a few years ago. And even within psychology, substantial gaps remain. For example, only 34 of 88 journals published replication studies, which accounted for just 0.2% of all published studies overall (Clarke et al., 2023; Feldman, 2024).
In general: to move these discussions beyond mere intuition, a meta-scientific evaluation of the new methods is indispensable (Dudda et al., 2024; Phaf, 2024; Soderberg et al., 2021). This brings us back to the spectrum of possible reactions to replication failures: should a theory be discarded immediately after a failed replication, or would that be too radical, with a certain persistence on the part of researchers instead helping to establish strong theories (Phaf, 2024)? Interdisciplinary collaboration would also be valuable: for instance, it would save a great deal of effort if terminology were developed across disciplines rather than separately (and with much redundancy) for psychology (Hüffmeier et al., 2016), the humanities (Schöch, 2023), and marketing (Urminsky & Dietvorst, 2024). In some areas it remains entirely unclear what replication success even looks like, making it difficult to judge whether two studies examining the same question arrive at the same result. Even where this is clear, the success rate depends heavily on how replication success is measured (https://forrt.org/FReD/articles/success_criteria.html).
Open Science is occasionally equated with Open Access. While Open Science is much more than that, the debate over Open Access is one of the oldest and, to some extent, independent of issues of research integrity. The social dilemma of how researchers can become independent of commercial publishers is still far from resolved. Whether transformative agreements between universities and publishers (for example, Germany’s nationwide DEAL agreements, negotiated collectively by German research organisations) will succeed remains uncertain.
Even research that addresses the topic of Open Science sometimes fails to meet the very criteria it calls for. For example, an article by Open Science advocates (Protzko et al., 2023) has drawn criticism because it deviated from its preregistration without transparently reporting this.
What Is Happening Right Now?
The ideas and findings discussed here are a current snapshot of a dynamic discourse. More and more universities are founding Open Science centers and engaging with the associated topics. Germany’s national research funder, the DFG, funds a Priority Programme (Schwerpunktprogramm) on meta-science and replicability, coordinated from LMU Munich (Ludwig Maximilian University of Munich). The American Center for Open Science is working on numerous projects, and within the European FORRT initiative, projects and teams are being organized. It is now only a matter of time before all researchers become aware of the problems and proposed solutions and take part in improving scholarship.
A topic still relatively little addressed at present is that of war. Many research institutions are subject to embargoes that require collaborations with researchers from certain countries, such as China or Russia, to be reported or even avoided altogether. Depending on one’s epistemological standpoint, different developments are conceivable:
Positivism: Assuming that there is one single truth (Wittgenstein, 1922/2004) that everyone merely needs to uncover, opposing nations must be seen as competitors with whom knowledge should not be shared. Science would therefore need to close itself off toward them. How this could be achieved with respect to online databases is primarily a technical question.
Relationist: In the sense of Fleck (1935/2015), truth is socially anchored, and different societies can, in theory, operate with different truths. Researchers from opposing nations might, as a result, have no interest at all in each other’s viewpoints, or might dismiss them as politically tainted.
Nationalist: From this perspective, one could argue that a society that funds science should be the primary beneficiary of it.
Altruistic/idealistic: Drawing on the Mertonian norms (Merton, 1968), science can be understood as a commons belonging to all people, to which no one has privileged access (the communism of scientific knowledge) and from which all people can benefit. Under this view, science does not stop at political borders. From this perspective, Open Science could be understood as a diplomatic strategy for scientific exchange despite political differences.
Which perspective will ultimately prevail remains uncertain. In the context of the war in Israel and Gaza and an open letter from researchers on the subject, the German Federal Ministry of Education and Research compiled, in June 2024, lists of university faculty with particular political views and examined the possibility of cutting their research funding. This funding affair is perceived across the academic community – irrespective of its political background – as a threat to academic freedom, and is more compatible with nationalist and positivist perspectives.
There is a tension in the communication between researchers and society: among themselves, researchers must communicate that reforms and improvements in science are necessary – which is most easily achieved by pointing to problems. In dialogue with society, however, pointing to these problems can undermine trust in science (Penders, 2024). It is therefore worth emphasizing that science is not finished (indeed, strictly speaking, it is never finished), but this does not mean it is any less trustworthy than rumors, conspiracy theories, individual experiences, or religion. Rather, a self-critical stance is a central component of science and a sign that its self-correction mechanisms are working.
Further Information
Retractionwatch reports on current news about errors in research: https://retractionwatch.com.
On her YouTube channel, Mai Thi Nguyen-Kim, a German science communicator, discusses problems such as publication bias and self-correction mechanisms (in German): https://www.youtube.com/watch?v=DHyRaUeHcGY.
Sabine Hossenfelder addresses, in a YouTube video, the question of how climate scientists (should) communicate their findings: https://www.youtube.com/watch?v=gMOjD_Lt8qY.
The German network of Open Science initiatives connects researchers for talks and workshops on Open Science: OSF wiki of the network.
An introduction to Open Science for students is offered by Pennington (2023), Matters of Significance (2024), and Chambers (2017).
A glossary of Open Science terms is available online via FORRT: https://forrt.org/glossary/english.
Open Science is not yet the norm; Kohrs et al. (2023) summarize how everyone can contribute to it.