The System

Alongside the ideal of what academia should be or how it should work, there is academia as it is actually integrated into our social system. One particularity here is that researchers pursue this activity as a profession — that is, they earn money doing it. Committees decide how this money, which comes largely from taxpayers, is distributed, and these committees themselves consist of researchers (academic self-governance). Because of the heavy workload of simultaneously conducting and administering research, decision-makers simplify their work by using shortcuts to select highly qualified individuals. As a result, the currency of academia has largely become the number of research articles (papers) published in academic journals. Another commonly used indicator is the number of other scholarly articles that cite a person’s research — in other words, citation counts. Role models such as Charles Darwin or William James, and even some current Nobel laureates, would stand no chance of obtaining a permanent position in academia by today’s standards — they simply did not write enough papers. Many of the problems discussed here have been known in psychology, for example, for several decades, making a replication crisis all but inevitable (Baker, 2016; Cronbach et al., 1991; Greenwald, 1976).

Scholarship versus the Academic Business

While some natural and social sciences relied heavily on book publications in their early days, with a substantial body of work built up mostly by individual scholars (e.g., Galileo Galilei for physics or William James for psychology), today it is academic journals that take center stage. These journals are comparable to the magazines found at a newsstand or supermarket, except that they consist of articles (mostly written in English) authored by researchers, and in many cases are only available online or through university libraries. Researchers then download individual articles from these journals over the internet via their university network, and libraries hold contracts with publishers, paying money so that members of the university have access to the catalogs. Every submitted article addresses a research question defined by the researchers themselves (e.g., how many psychological studies replicate on average?). The articles usually report studies along with their results. Before publication, articles are reviewed (review) — not by journal staff, but by colleagues (peers). This peer review is meant to ensure the quality of research. Typically, reviewers check that the conclusions are justified by the data collected, that the research question is clearly answered, that the article is comprehensible, and that the findings are exciting or surprising. Journals differ in which topics they cover (e.g., social psychology, consumer behavior, applied sports science), how rigorous their peer review is, and how many researchers read and cite them. The academic disciplines are thus deeply embedded in a system that allows researchers and publishers to earn a steady income.

Precarious Working Conditions

So much for the general framework: researchers conduct studies and share their results mostly in the form of publications, such as journal articles or books. As for jobs in academia (that is, primarily at universities), they are hierarchically structured. In Germany in particular, permanent positions are few and are almost exclusively professorships, with everyone below that level — doctoral candidates and postdoctoral researchers (postdocs) — on fixed-term positions. (Systems with permanent lectureships or tenure-track assistant professorships, as in the UK and US, distribute security somewhat differently, but the underlying competition for scarce permanent posts is comparable.) Anyone working in academia usually finds themselves in fierce competition for one of the few permanent positions (Rahal et al., 2023). The underlying assumption is that competition among researchers boosts productivity. From the start of a doctorate1 to an appointment to a professorship — one of the rare permanent positions — contracts typically last only one to three years, and positions are usually less than full-time. At the same time, in many disciplines it is uncommon to complete a doctorate within the typical three years while working fewer than 40 hours a week. While the majority of scientific publications are based on studies conducted as part of doctoral projects, doctoral candidates are simultaneously the people in the system considered to have the least value — that is, whose labor is cheapest. Researchers in fixed-term positions are thus exposed to enormous performance pressure. Mental health problems such as burnout or depression are widespread among doctoral candidates (Jaremka et al., 2020; Liu et al., 2019). The path to a professorship is achieved above all by those who publish many articles in prestigious journals. Given the immense workload and the sheer number of articles submitted to journals, there is no longer time to carefully check and recompute results (Nuijten et al., 2017); instead, what matters most is how clearly the results answer the research question — or, more precisely, how clearly they confirm it (Giner-Sorolla, 2012; Mynatt et al., 1977). In other words, the very part of scientific work that gets rewarded is the one that is not actually in the researchers’ hands: the outcome of an investigation. From that point on, published articles and prestige — rather than quality (Brembs, 2018) — become the currency of academia: they determine who receives research funding, and research funding and publications in turn determine who is appointed to professorships. In the hiring committees that make these decisions, members usually do not read the applicants’ articles; they merely count how many are listed and in which journals. Applicants are sometimes asked to report their citation counts. Some of these figures (e.g., the impact factor) are owned by corporations, and access to them must be purchased through the institution.

In addition to these devastating problems, there are systemic issues of sexual harassment (Hoebel et al., 2022) and abuse of power, which are difficult to resolve given the currently strictly hierarchical structure of the system (Forster & Lund (2018); see also https://www.netzwerk-mawi.de/ and various recent reports, [1; 2; 3]). According to reports from the German network against abuse of power in academia, cases are kept quiet and the perpetrators quietly move to another university, so the problem never actually gets resolved. Elsherif et al. (2022) illustrate vividly who has it particularly easy in this system with their Academic Wheel of Privilege (p. 85; see also https://www.psychologicalscience.org/observer/gs-navigating-academia-as-neurodivergent-researchers). For example, female doctoral candidates in the Netherlands received worse grades than their male counterparts specifically when the dissertation committee — the group of professors evaluating the doctorate — consisted entirely of men (Bol, 2023). On top of this, universities fall well short of the statutory quota for employing people with recognised severe disabilities that applies to German employers, and further below the rates achieved in other sectors (German-language source).

Too Much Research

During a doctorate, researchers must typically publish three scientific articles within a total of 3-6 years, plus parental leave (or, in fairer cases, produce three articles worthy of publication). For postdocs, the number required is even higher. Under Germany’s Academic Fixed-Term Contract Act (Wissenschaftszeitvertragsgesetz, WissZeitVG), individuals may be employed at German universities for a maximum of six years before and six years after completing a doctorate. For the Habilitation — a second, post-doctoral qualification thesis that in Germany and several other continental European countries is the traditional prerequisite for a full professorship, with no direct equivalent in the UK or US system — a similar time frame is allotted; the rule of thumb in psychology, for instance, is around six articles. How extensive the articles are plays only a secondary role. For example, conducting a meta-analysis — in which existing findings on a particular topic are systematically collected and statistically summarized or compared — often takes several years. A longitudinal study can, depending on the research question, even take decades. By contrast, a cross-sectional survey using an online questionnaire can be completed within a few weeks. A doctoral candidate who conducts a single meta-analysis could not obtain a doctorate with it. Had she instead conducted three simple online studies and published them separately, it would have been easier.2 These arbitrary requirements have led researchers to place an enormous burden on themselves merely by reviewing their colleagues’ articles — a burden that hampers scientific progress (Hanson et al., 2023).

To illustrate the workload involved, consider a thought experiment: suppose there were 10 researchers who together published 10 articles a year — sometimes alone, sometimes as a group — and each article was reviewed by two people; then each person would need to review two articles. For the system to work, each person would need to review the average number of articles published per person, multiplied by the number of reviewers required. With 10 publications per person and three reviewers, that would be 10x3=30 reviews. But not all articles are published by the journal they are first submitted to, nor are they published immediately. Researchers often submit their articles to the “top-ranked” journals. After several reviewers there have assessed the article, it gets rejected (Jaremka et al., 2020). At best, revisions are requested, which often trigger another round of peer review, and even then the article is not always published afterward. So our calculation does not add up: let us assume, conservatively, that an article is reviewed nine times in total (e.g., three reviewers once, then rejection, then three reviewers again, revision, a second review, acceptance). 10x3 becomes 10x9 — accounting for some vacation time, that amounts to a bit more than two reviews per week, ideally taking up to two working days. Under this arithmetic, there is less time left for teaching, knowledge transfer, supervising students or doctoral candidates, acquiring research funding, university self-governance, and so on. Because of the sheer number of articles that must be published and the strict review system, academia piles up a mountain of work on itself — one that is not realistically manageable, and under which the quality of research ultimately suffers. For example, neither reviewers nor journal editors noticed that more than 30 articles contained the phrase “Regenerate response” right in the middle of the text — a phrase that, in OpenAI’s ChatGPT, lets a user reword AI-generated text at the click of a button (https://retractionwatch.com/2023/10/06/signs-of-undeclared-chatgpt-use-in-papers-mounting/). Some articles even stated, “As an AI language model, I …” (PubPeer search for this phrase“). In one case, the publisher Elsevier altered a published article — not through the recommended route3 of a transparent erratum or corrigendum, i.e., a public notice explaining the change and its reasons, but without any explanation or the authors’ consent (https://predatory-publishing.com/elsevier-changed-a-published-paper-without-any-explanation/).

Publish or Perish

Anyone working in academia should know the most important rule of the game: whoever wants to survive must publish articles. In short: publish or perish. For a doctorate, a Habilitation (see above), acquiring research funding, and being called to a professorship (in Germany, the formal Berufung procedure), publications are the top criterion. The defining feature of a currency is that it allows a value to be assigned to things. So what does value look like in research? To describe and select journal subscriptions (that is, explicitly not to evaluate) across different research fields, bibliometricians devised various metrics, such as the impact factor or the h-index. Both figures are calculated from how often articles are cited, broken down by journal or by person. For instance, the journal impact factor is calculated by dividing the total number of citations in a given year by the number of citable publications in the reference years (e.g., the three preceding years). Journals advertise high impact factors and sometimes do not allow citation of comments (or publish comments under the same reference as the original studies) in order to keep their impact factors as high as possible. They can, of course, also choose whether to report the 3-year or 2-year impact factor. Calculation methods also differ in terms of which underlying data they draw on. The journal impact factor database currently belongs to the company Clarivate Analytics and is not publicly accessible. This means the exact figures cannot easily be recalculated and checked. Given the important yet opaque role of citation metrics, it is hardly surprising that they get manipulated, and that 3 of the top 10 journals in one field turned out not to be scientific journals at all. It is also clear that citation metrics are either unrelated to, or negatively related to, scientific quality (Brembs, 2018; Etzel et al., 2024, Table 3). For example, there was the problem that Microsoft Excel recognized gene names as dates, changed their format, and rendered the actual names unrecognizable. As a result, parts of the corresponding results became useless (Ziemann et al., 2016). This problem occurred especially in prestigious journals. They did nothing about it; instead, Excel itself was adjusted a few years later.

Besides journal rankings, rankings of individuals can also be created. Researchers from Stanford published one such list of the top 2% most-cited researchers worldwide. Aside from inviting misuse, it contains numerous errors (Abduh, 2023), such as researchers who supposedly published articles for hundreds of years — both before and after their death. Companies that own publishing houses and other tools used in research (e.g., reference-management software) also collect data about researchers (e.g., which articles are accessed for how long, which passages of text are highlighted). Libraries are sometimes asked to install publishers’ tracking software. The publishers then sell the collected data back to the researchers. This practice endangers academic freedom, since states can require publishers to disclose the names of researchers working on politically sensitive topics. The initiative Stop Tracking Science campaigns against this practice.

Researchers often choose the journal in which to publish their results based on citation metrics, and in hiring committees, applicants are frequently selected on the same basis. In personal accounts, for instance, professors report that across countless hiring committees over several decades, they never once read a single one of the applicants’ research articles. To get their articles into “high-impact journals,” researchers are even willing to embellish their results — in order to increase their chances of publication (Bohorquez et al., 2024). Calls for the responsible use of these metrics have been made for some time now (Hicks et al., 2015), and numerous universities and researchers have joined forces to make the evaluation of research more meaningful (e.g., through the San Francisco Declaration on Research Assessment). How incentive structures and career status affect research misconduct has been studied with mixed results. The underlying problem: as soon as a system has a clear evaluation criterion, everything gets oriented toward it (gaming the system). According to Fanelli et al. (2022), incentive structures that lead to poor research — in the form of image manipulation in biology — are especially prevalent in China, and considerably less so in the USA, the United Kingdom, and Canada.

Another product of the publish-or-perish structure is that most instruments developed in psychology to measure personality traits are used only a handful of times — and mainly by their own developers, at that (Elson et al., 2023). Elson et al. (2023) make the point: psychological measurement instruments are not toothbrushes! An even more extreme version of this problem can be seen with so-called paper mills (van Noorden, 2023): organizations that automatically generate large quantities of scientific articles without actually conducting the studies described in them. Researchers can then buy co-authorships to increase their number of published articles and gain more citations. Depending on the journal, these articles are not caught during peer review, that is, evaluation by other researchers. There are concerns that the buyers of such articles can themselves become very successful, even becoming journal editors, thereby protecting themselves from being caught. The true extent of the paper-mill problem is unclear and remains largely unexplored (Byrne & Christopher, 2020). One telltale sign for detecting articles produced with the help of artificial intelligence is so-called tortured phrases (Cabanac et al., 2021) — phrases that are grammatically correct but rarely occur in ordinary usage or make little sense.

The Bottleneck Hypothesis and the Drive for Innovation

Renowned scientific journals receive countless submissions every day but publish only a limited number of articles. They must therefore select very strictly what gets reviewed and, ultimately, published. Because a journal’s goal is to be widely read, journal editors choose the articles with the greatest potential to become well known and heavily cited (Giner-Sorolla, 2012). This applies, for example, to contributions with particular practical implications, surprising findings, or especially consistent findings. Studies whose results do not permit clear conclusions — or whose authors draw their conclusions too cautiously — are therefore out of the running. In the neurosciences, for instance, some journals openly state that they do not publish replication studies and that they select for novelty, while most other journals take no position on the matter at all (Yeung, 2017). In psychology, in 2017 only 33 out of 1,151 journals stated that they accept replications (Martin & Clarke, 2017). Innovative findings, or findings presented as innovative (Stavrova et al., 2024), are cited more often, so the journal gains more readers, more submissions, and thus more money through subscriptions and publication fees — yet heavily cited articles tend to replicate worse than less-cited ones (Serra-Garcia & Gneezy, 2021), and prestigious journals are magnets for questionable research practices (Kepes et al., 2022) and for research that is demonstrably no better, or even of lower quality (Brembs, 2018).

How does this come about? People speak of a bottleneck — many submissions but few publications. Combined with the incentive to publish in precisely these journals, this leads researchers to use every means available to gain a chance at publication. At some institutes, publishing in a particular journal automatically earns a doctoral candidate the highest possible grade. Other institutes already specify, in the job posting for a doctoral position, which journal the results of the research project must be published in. The fact that prestigious journals such as Nature or Science mainly publish articles with clear-cut messages — articles that also get read more often (Stavrova et al., 2024) — spurs researchers on to manufacture clear-cut results. If a finding happens not to fit the hypothesis being tested, it is either distorted through data-analysis practices that are currently tolerated in many places (questionable research practices) until it does fit, or it is not published at all and ends up in the file drawer (Gopalakrishna et al., 2021; Schneider et al., 2024). Likewise, this discourages researchers from repeating an experiment that has already been conducted — that is, from running a replication study (Marefat et al., 2024).

The File-Drawer Problem

It has been known for several decades that researchers primarily publish those studies that support their theories (Rosenthal, 1979; Sterling, 1959). In an extreme case, someone might run five studies to test a theory, confirm the theory in only one of them, and publish only that one. Other researchers who then search the (published) literature see only the “successful” study. This creates the impression that the theory is correct, even though the majority of the studies do not actually support that conclusion. Because study results are subject to natural statistical fluctuation, it is likely that, among many studies, at least one will show the desired result purely by chance. The file-drawer problem allowed entire lines of research to develop that have since gone completely extinct once awareness of replication studies took hold (Brockman, 2022; Mac Giolla et al., 2022).

For meta-analyses — studies that summarize existing findings — a range of methods have already been developed to assess the severity of the file-drawer problem. Numerous methods for correcting this bias also already exist (Fisher et al., 2017; Hedges & Vevea, 1996; Schimmack, 2020; Simonsohn et al., 2014; van Aert & van Assen, 2018). However, none of these methods works in every possible scenario (Carter et al., 2019). So researchers cannot avoid actually publishing their failed studies.

Medicine represents a special case: all studies conducted there must be publicly registered. Any subsequent publication must then cite a registration number. Public information on registered studies therefore makes it possible to track which individuals, institutions, or countries actually publish how many of the studies they conduct. Researchers in Berlin have developed a website with an interactive dashboard for exactly this purpose, allowing the collected data to be searched and displayed (Franzen et al., 2023; Riedel et al., 2022). As of the current state (July 2024), https://quest-cttd.bihealth.org/ shows that, of all registered studies, only 46% were published within the following two years and 74% within the following five years. People who volunteer as participants in these studies, or third-party funders, thus gain insight into the size of the drawer in which failed studies and “not-so-exciting results” end up.

Access to Knowledge

Because research is embedded in the commercial publishing system, much of academia finds itself trapped in a social dilemma that results in severely restricted access to knowledge. The dominant model for academic journals — most of which belong to publishers such as Springer, Elsevier, Sage, or Taylor and Francis — is a subscription model. University libraries regularly pay money to publishers so that members of the university (students and staff) have access to the works published there. Anyone without a subscription can buy individual articles. If you try to download an article online without being on a university network, it is not free: the article sits behind a paywall. If an article is to be published so that everyone can access it freely (open access), this costs the authors extra — typically between €2,000 and €9,000 per article, and up to €20,000 for books (article/book/chapter processing charges). This also affects the article’s own authors: upon publication, they sign away all their rights to the publisher, and can only access their own article through a university with a subscription, or by paying a fee (around 30 US dollars), unless they paid the enormous open-access fees in the first place. So in order to read research at all, universities must buy subscriptions or individual articles. As a result, individuals, institutions, countries, or even entire regions with fewer financial resources — such as the Global South — are systematically disadvantaged in their ability to take part in international scholarly discourse.

This social dilemma lies precisely in how difficult it is to change this system. Brembs et al. (2023) describe it as follows: libraries sign subscription contracts with publishers in order to make research available at their universities. If they canceled these contracts, that could slow down research and weaken a university’s standing. Researchers cannot boycott the well-known journals of commercial publishers, because doing so would jeopardize their careers. They depend on publishing in prestigious journals. It is also extremely difficult to suddenly change the behavior of millions of researchers, most of whom have lived with the current system for many years. The journals, as the third party in this dilemma, profit from this dependency and can raise prices at will and worsen publication terms as they please (e.g., requiring authors to sign away all rights to their research). Aside from their brand value — that is, the prestige and trust mistakenly (Brembs, 2018) placed in them — they contribute almost nothing to this system. Universities and countries already have the option of managing research articles online in journals of their own; peer review and quality assurance are carried out by volunteer researchers rather than journal staff; and, as is evident from existing journals4 (Carlsson et al., 2017), researchers can handle the formatting of articles themselves with little effort. Corresponding solutions are discussed in the chapter Open Access Publications.

Wasting Taxpayer Money on Commercial Publishers?

The German-language science show MAITHINK X explains why taxpayer money is wasted under the current system. For instance, the total amount of APCs — that is, publication costs for articles in open-access journals, not including library subscription costs, book publications, and much more — is estimated to have exceeded 2.5 billion US dollars worldwide, and more than tripled between 2019 and 2023 (Haustein et al., 2024). What the program falls short on is discussing possible solutions (see the chapter Solutions > The System).

Scientific Quality Control

The problem of quality control runs through all the problems of the scientific system described so far. During the review process before publication, many errors go undetected, and after publication, reports are so opaque that it is often not even possible to expose fabricated data. Reviews of scientific articles usually remain confidential, and when one journal rejects an article, it simply gets submitted to another. The criticisms already raised are then lost along the way (Aczel et al., 2021).

Scientific articles end with a paragraph on potential conflicts of interest. There, researchers must disclose whether they received benefits from companies for their research, or whether they profit in any way from the research (e.g., by owning stock in the company whose drug they tested favorably). In gambling research, such disclosures are frequently missing (Heirene et al., 2021). Some large companies systematically undermine scientific consensus by spreading false information: a well-known example is the tobacco industry and the well-established negative effect of smoking on health (Reed et al., 2021).

Some journals exploit this situation by publishing research in exchange for high publication fees (e.g., €4,500 per article) despite weak or entirely absent quality control. This practice is called predatory publishing and refers to researchers effectively buying publications, with publishers profiting from it. Because the peer-review process is anonymous and kept confidential, it is sometimes unclear whether a given journal has any quality control in place at all. Paper mills go a step further still. There, people can buy co-authorships on articles. These articles are frequently produced by individuals or generated using algorithms. Cases in which such nonsensical articles are exposed are common, but the articles are rarely retracted or given a public notice explaining the error (Cabanac & Labbé, 2021). Further detection methods are being developed, checking, for example, whether the authors of journal articles have an institutional email address (that is, an address from a university or research institution) (Sabel et al., 2023).

Besides publications, researchers can also buy citations or artificially inflate their own (Singh Chawla, 2024) — for instance, according to Google Scholar, Larry the cat was briefly credited with 132 citations. Because reviews remain confidential, “review mills” have also emerged: there, authors are pressured into citing specific research articles in order to boost the reviewers’ own citation counts. These reviews consist of vague, formulaic boilerplate text (Oviedo-Garcı́a, 2024).

Further Information

References

Abduh, A. J. (2023). A critical analysis of the world’s top 2% most influential scientists: Examining the limitations and biases of highly cited researchers lists. https://doi.org/10.22541/au.167435298.80209125/v1
Aczel, B., Szaszi, B., & Holcombe, A. O. (2021). A billion-dollar donation: Estimating the cost of researchers’ time spent on peer review. Research Integrity and Peer Review, 6, 1–8. https://doi.org/10.1186/s41073-021-00118-2
Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), 452–454. https://doi.org/10.1038/533452a
Bohorquez, N. G., Weerasuriya, S., Brain, D., Senanayake, S., Kularatna, S., & Barnett, A. (2024). Health and medical researchers are willing to trade their results for journal prestige: Results from a discrete choice experiment. https://doi.org/10.31219/osf.io/uwt3b
Bol, T. (2023). Gender inequality in cum laude distinctions for PhD students. Sci. Rep., 13(1), 20267. https://doi.org/10.31235/osf.io/s5b6j
Brembs, B. (2018). Prestigious science journals struggle to reach even average reliability. Frontiers in Human Neuroscience, 12, 37. https://doi.org/10.3389/fnhum.2018.00037
Brembs, B., Huneman, P., Schönbrodt, F., Nilsonne, G., Susi, T., Siems, R., Perakakis, P., Trachana, V., Ma, L., & Rodriguez-Cuadrado, S. (2023). Replacing academic journals. Royal Society Open Science, 10(7). https://doi.org/10.1098/rsos.230206
Brockman, J. (2022). Adversarial collaboration: An EDGE lecture by daniel kahneman. https://www.edge.org/adversarial-collaboration-daniel-kahneman
Byrne, J. A., & Christopher, J. (2020). Digital magic, or the dark arts of the 21st century-how can journals and peer reviewers detect manuscripts and publications from paper mills? FEBS Letters, 594(4), 583–589. https://doi.org/10.1002/1873-3468.13747
Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology, 72(12), 1461–1476. https://doi.org/10.1002/asi.24495
Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals. https://doi.org/10.48550/arXiv.2107.06751
Carlsson, R., Danielsson, H., Heene, M., Innes-Ker, Å., Lakens, D., Schimmack, U., Schönbrodt, F. D., van Asssen, M., & Weinstein, Y. (2017). Inaugural editorial of meta-psychology. Meta-Psychology, 1, a1001. https://doi.org/10.15626/MP2017.1001
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019). Correcting for bias in psychology: A comparison of meta-analytic methods. Advances in Methods and Practices in Psychological Science, 2(2), 115–144. https://doi.org/10.1177/2515245919847196
Cronbach, L. J., Snow, R. E., & Wiley, D. E. (1991). Improving inquiry in social science: A volume in honor of lee j. cronbach. Lawrence Erlbaum Associates.
Elsherif, M. M., Middleton, S. L., Phan, J. M., Azevedo, F., Iley, B. J., Grose-Hodge, M., Tyler, S. L., Kapp, S. K., Gourdon-Kanhukamwe, A., Grafton-Clarke, D., Yeung, S. K., Shaw, J. J., Hartmann, H., & Dokovova, M. (2022). Bridging neurodiversity and open scholarship: How shared values can guide best practices for research integrity, social justice, and principled education. https://doi.org/10.31222/osf.io/k7a9p
Elson, M., Hussey, I., Alsalti, T., & Arslan, R. C. (2023). Psychological measures aren’t toothbrushes. Communications Psychology, 1(1). https://doi.org/10.1038/s44271-023-00026-9
Etzel, F., Seyffert-Müller, A., Schönbrodt, F. D., Kreuzer, L., Gärtner, A., Knischewski, P., & Leising, D. (2024). Inter-rater reliability in assessing the methodological quality of research papers in psychology. https://doi.org/10.31234/osf.io/4w7rb
Fanelli, D., Schleicher, M., Fang, F. C., Casadevall, A., & Bik, E. M. (2022). Do individual and institutional predictors of misconduct vary by country? Results of a matched-control analysis of problematic image duplications. PLoS One, 17(3), e0255334. https://doi.org/10.1371/journal.pone.0255334
Fisher, Z., Tipton, E., & Zhipeng, H. (2017). Robumeta: Robust variance meta-regression. https://doi.org/10.32614/cran.package.robumeta
Forster, N., & Lund, D. W. (2018). Identifying and dealing with functional psychopathic behavior in higher education. Glob. Bus. Organ. Excel., 38(1), 22–31. https://doi.org/10.1002/joe.21897
Franzen, D. L., Carlisle, B. G., Salholz-Hillel, M., Riedel, N., & Strech, D. (2023). Institutional dashboards on clinical trial transparency for university medical centers: A case study. PLoS Medicine, 20(3), e1004175. https://doi.org/10.1371/journal.pmed.1004175
Giner-Sorolla, R. (2012). Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspectives on Psychological Science : A Journal of the Association for Psychological Science, 7(6), 562–571. https://doi.org/10.1177/1745691612457576
Gopalakrishna, G., Wicherts, J. M., Vink, G., Stoop, I., van den Akker, O., Riet, G. ter, & Bouter, L. (2021). Prevalence of responsible research practices and their potential explanatory factors: A survey among academic researchers in the netherlands. https://doi.org/10.31222/osf.io/xsn94
Greenwald, A. G. (1976). An editorial. Journal of Personality and Social Psychology, 33(1), 1–7. https://doi.org/10.1037/h0078635
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2023). The strain on scientific publishing. https://doi.org/10.48550/arXiv.2309.15884
Haustein, S., Schares, E., Alperin, J. P., Hare, M., Butler, L.-A., & Schönfelder, N. (2024). Estimating global article processing charges paid to six publishers for open access between 2019 and 2023. https://arxiv.org/abs/2407.16551
Hedges, L. V., & Vevea, J. L. (1996). Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model. Journal of Educational and Behavioral Statistics, 21(4), 299–332. https://doi.org/10.3102/10769986021004299
Heirene, R., LaPlante, D., Louderback, E. R., Keen, B., Bakker, M., Serafimovska, A., & Gainsbury, S. M. (2021). Preregistration specificity & adherence: A review of preregistered gambling studies & cross-disciplinary comparison. https://doi.org/10.31234/osf.io/nj4es
Hicks, D., Wouters, P., Waltman, L., Rijcke, S. de, & Rafols, I. (2015). Bibliometrics: The leiden manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a
Hoebel, M., Durglishvili, A., Reinold, J., & Leising, D. (2022). Sexual harassment and coercion in german academia: A large-scale survey study. Sexual Offending: Theory, Research, and Prevention, 17. https://doi.org/10.5964/sotrap.9349
Jaremka, L. M., Ackerman, J. M., Gawronski, B., Rule, N. O., Sweeny, K., Tropp, L. R., Metz, M. A., Molina, L., Ryan, W. S., & Vick, S. B. (2020). Common academic experiences no one talks about: Repeated rejection, impostor syndrome, and burnout. Perspectives on Psychological Science : A Journal of the Association for Psychological Science, 15(3), 519–543. https://doi.org/10.1177/1745691619898848
Kepes, S., Keener, S. K., McDaniel, M. A., & Hartman, N. S. (2022). Questionable research practices among researchers in the most research–productive management programs. Journal of Organizational Behavior. https://doi.org/10.1002/job.2623
Liu, C., Wang, L., Qi, R., Wang, W., Jia, S., Shang, D., Shao, Y., Yu, M., Zhu, X., Yan, S., Chang, Q., & Zhao, Y. (2019). Prevalence and associated factors of depression and anxiety among doctoral students: The mediating effect of mentoring relationships on the association between research self-efficacy and depression/anxiety. Psychology Research and Behavior Management, 12, 195–208. https://doi.org/10.2147/PRBM.S195131
Mac Giolla, E., Karlsson, S., Neequaye, D. A., & Bergquist, M. (2022). Evaluating the replicability of social priming studies. In PsyArXiv. https://doi.org/10.31234/osf.io/dwg9v
Marefat, F., Hassanzadeh, M., & Hamidi, F. (2024). Incorporating replication in higher education: Supervisors’ perspectives and institutional pressures. Account. Res., 1–22. https://doi.org/10.1080/08989621.2024.2412054
Martin, G. N., & Clarke, R. M. (2017). Are psychology journals anti-replication? A snapshot of editorial practices. Frontiers in Psychology, 8, 523. https://doi.org/10.3389/fpsyg.2017.00523
Mynatt, C. R., Doherty, M. E., & Tweney, R. D. (1977). Confirmation bias in a simulated research environment: An experimental study of scientific inference. Quarterly Journal of Experimental Psychology, 29(1), 85–95. https://doi.org/10.1080/00335557743000053
Nuijten, M. B., van Assen, M. A. L. M., Hartgerink, C. H. J., Epskamp, S., & Wicherts, J. M. (2017). The validity of the tool “statcheck” in discovering statistical reporting inconsistencies. https://doi.org/10.31234/osf.io/tcxaj
Oviedo-Garcı́a, M. (2024). The review mills, not just (self-) plagiarism in review reports, but a step further. Scientometrics, 1–9. https://doi.org/10.1007/s11192-024-05125-w
Rahal, R.-M., Fiedler, S., Adetula, A., Berntsson, R. P.-A., Dirnagl, U., Feld, G. B., Fiebach, C. J., Himi, S. A., Horner, A. J., Lonsdorf, T. B., Schönbrodt, F., Silan, M. A. A., Wenzler, M., & Azevedo, F. (2023). Quality research needs good working conditions. Nature Human Behaviour, 7(2), 164–167. https://doi.org/10.1038/s41562-022-01508-2
Reed, G., Hendlin, Y., Desikan, A., MacKinney, T., Berman, E., & Goldman, G. T. (2021). The disinformation playbook: How industry manipulates the science-policy process—and how to restore scientific integrity. Journal of Public Health Policy, 42(4), 622. https://doi.org/10.1057/s41271-021-00318-6
Riedel, N., Wieschowski, S., Bruckner, T., Holst, M. R., Kahrass, H., Nury, E., Meerpohl, J. J., Salholz-Hillel, M., & Strech, D. (2022). Results dissemination from completed clinical trials conducted at german university medical centers remained delayed and incomplete. The 2014 -2017 cohort. Journal of Clinical Epidemiology, 144, 1–7. https://doi.org/10.1016/j.jclinepi.2021.12.012
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M. (2023). Fake publications in biomedical science: Red-flagging method indicates mass production. https://doi.org/10.1101/2023.05.06.23289563
Schimmack, U. (2020). A meta-psychological perspective on the decade of replication failures in social psychology. Canadian Psychology/Psychologie Canadienne. https://doi.org/10.1037/cap0000246
Schneider, J. W., Allum, N., Andersen, J. P., Petersen, M. B., Madsen, E. B., Mejlgaard, N., & Zachariae, R. (2024). Is something rotten in the state of denmark? Cross-national evidence for widespread involvement but not systematic use of questionable research practices across all fields of research. PLoS One, 19(8), e0304342. https://doi.org/10.31222/osf.io/r6j3z
Serra-Garcia, M., & Gneezy, U. (2021). Nonreplicable publications are cited more than replicable ones. Science Advances, 7(21). https://doi.org/10.1126/sciadv.abd1705
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology. General, 143(2), 534–547. https://doi.org/10.1037/a0033242
Singh Chawla, D. (2024). The citation black market: Schemes selling fake references alarm scientists. Nature. https://doi.org/10.1038/d41586-024-01672-7
Stavrova, O., Kleinberg, B., Evans, A. M., & Ivanovic, M. (2024). Scientific publications that use promotional language receive more citations and social media mentions. In PsyArXiv. https://doi.org/10.31234/osf.io/da9xn
Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. Journal of the American Statistical Association, 54(285), 30–34. https://doi.org/10.1080/01621459.1959.10501497
Thériault, F., R. (2023). The missing majority in behavioral science dashboard. https://remi-theriault.com/dashboards/missing_majority
van Aert, R. C. M., & van Assen, M. A. L. M. (2018). P-uniform*. https://doi.org/10.31222/osf.io/zqjr9
van Noorden, R. (2023). How big is science’s fake-paper problem? Nature, 623(7987), 466–467. https://doi.org/10.1038/d41586-023-03464-x
Yeung, A. W. K. (2017). Do neuroscience journals accept replications? A survey of literature. Frontiers in Human Neuroscience, 11, 468. https://doi.org/10.3389/fnhum.2017.00468
Ziemann, M., Eren, Y., & El-Osta, A. (2016). Gene name errors are widespread in the scientific literature. Genome Biology, 17, 1–3. https://doi.org/10.1186/s13059-016-1044-7

  1. The process of obtaining a doctoral degree/title, which — depending on the discipline and institution — involves writing either a single book (monograph) or several journal articles (cumulative dissertation).↩︎

  2. One could object here that some journals only publish articles that report multiple studies. In doing so, however, these journals clearly miss their actual goal, namely the internal replication of one’s own findings. Instead, it tempts researchers to run several studies with few participants each, rather than one study with many respondents.↩︎

  3. Ethical guidelines for the publication process are available, for example, from the Committee on Publication Ethics (https://publicationethics.org/guidance/Guidelines).↩︎

  4. This refers to journals that publish articles without requiring a subscription and without a paywall, and at no cost to the authors.↩︎