Time for a prepublication culture in clinical research?

In 1969, Franz Ingelfinger wrote in The New England Journal of Medicine about the journal’s “definition of a ‘sole contribution’”.¹ The journal’s masthead had stated a clear condition for any manuscript’s consideration: “Articles are accepted for consideration with the understanding that they are contributed for publication solely in this journal.”¹ In other words, a given paper could be published exclusively in The New England Journal of Medicine, and nowhere else. This policy, known as the “Ingelfinger rule”, has had a major role in determining the ethos of publication in clinical research, and was revisited at least three times.^2–4 The editors offered justifications: assuring the novelty and newsworthiness of published papers, and that published papers met high quality standards by virtue of going through the process of peer review.

The internet came on the scene in the 1990s, and with it the opportunity for individual scientists and organisations to post manuscripts, data, and key findings in a widely available venue. But, as described by The New England Journal’s editors in 1995, “direct electronic publishing of scientific studies threatens to undermine time-tested traditions that help to ensure the quality of the medical literature”.⁴ Therefore, the editors made clear that, in their view, posting a paper, data, or key findings on the internet represented presubmission publication and would disqualify it from consideration.

And so, the Ingelfinger rule lived on. During the first decade of the 21st century, editors at various journals invoked the rule (or similar policies) to reject previously accepted papers. As recently as 2012—43 years after its initiation—findings from a survey⁵ showed that a substantial proportion of clinical scientists considered the Ingelfinger rule as active, precluding any consideration of posting full manuscripts or a significant body of their results on the internet or elsewhere before formal publication by a peer reviewed journal.

Meanwhile, beginning in the 1980s, other scientific disciplines—such as astronomy, mathematics, and computer science—were taking a wholly different approach.⁶ Authors would post preprints of their manuscripts on publicly available websites such as arXiv, where they can undergo one or more rounds of voluntary peer review. After the authors have responded (or not) to the peer reviewers’ comments, authors would then submit their papers to journals for official publication. Several other preprint servers are available, including figshare, PeerJ (which also functions as a peer-reviewed journal), F1000Research, GitHub, and ResearchGate. To date, more than 1 million e-prints are available on arXiv.

Desjardins-Proulx and colleagues⁶ cite a number of advantages for preprints. Perhaps the obvious advantage is time; under the traditional approach, scientific work does not become available to the public until after the process of submission, peer review, revision, and formal publication. Even without taking into account rejections along the way, this process can take many months, even years, a delay that can be seriously problematic for all investigators and especially early-career researchers. The problem is only becoming worse as changing practices are leading to longer times from completion of work to eventual publication.⁷ Moreover, there is a cost to society when important research findings are delayed on their way to publication. This circumstance could be particularly relevant for clinical trials, for which methods have been prespecified and registered, and results should be relatively standardised. It is therefore highly unlikely that main findings will change with additional data cleansing or follow-up. Second, venues for preprints establish provenance—who made discoveries, and when. Third, and perhaps counterintuitively, preprints allow for more robust peer review, as peer review is not limited to the 2–4 reviewers chosen by one journal editor; instead, any number of interested readers can offer comments and suggestions. For example, one of us (MSL) recently offered a voluntary preprint peer review, which even included analysis of posted data; the results of the external preprint data analysis were included in the final published version of the paper.⁸

For many disciplines, the posting of preprints for presubmission peer review has been the norm. Biology, however, has trailed behind,⁶ whereas clinical research remains well behind. Some have argued that biology’s tardiness is related to two factors: a misconception that preprints make it easier to steal ideas (when in fact preprints actually make idea theft easier to track), and the Ingelfinger rule. Desjardins-Proulx and colleagues⁶ argue that “a preprint is simply a document that allows ideas to spread and be discussed, it is not yet formally validated by the peer-review system”. Therefore, most major publishers in biology now allow, and many even encourage, preprints, including the Nature group, Public Library of Science, BMC, Elsevier, and Proceedings of the National Academy of Sciences. Moreover, sites like bioRxiv are accepting posting preprints of interest to biologists. However, to our knowledge, none of the major clinical medical journals have explicitly allowed preprints. Nonetheless, clinical journals, including The Lancet, do allow for prior posting of results through conference abstracts. It may seem counterintuitive that conference abstracts are allowed whereas fuller dissemination of clinical findings ahead of journal publication is not.

Two of us (EJT and HMK) have noted recently⁹ that the US National Institutes of Health issued a press release and held a press conference about the results of the Systolic Pressure Intervention Trial (SPRINT). The NIH had stopped the trial prematurely, after a recommendation from the data safety and monitoring board to do so given the markedly positive results. However, the NIH posted neither the main results nor the board’s report; instead, the public had to wait for publication of the final paper,¹⁰ which was based on cleaned data and reflects the views of the trial’s lead investigators.

Is it dangerous to release reports of clinical trial results prematurely? Potentially, reports that have not undergone formal peer review could be misleading.^1–4 Without peer review, and specifically peer review that is organised by a journal, clinicians and patients might be uncertain about the results and how to incorporate them into their practice and thinking. But, arguably, the same could be said even with the current system; without the benefit of systematic literature reviews and the work of guideline panels, clinicians, patients, and the public will not know how best to interpret any particular study’s findings. The published journal paper itself is unlikely to be the final word on the matter; continuing post-publication dialogue is part and parcel of knowledge acquisition. And, conversely, we might argue that in an era of open science and transparency, withholding findings from the public until a journal does its own peer review and editing could represent a kind of paternalism, which is troubling when many are calling for a democratisation of medicine and for widespread adoption of open science.¹¹ It is important to distinguish this discussion from the important and active debate about sharing individual-level patient data.¹² The current issue is specifically about the speed of reporting summary results.

The time has come for stakeholders in clinical research—funders, investigators, journals, policy makers, and most importantly patients and clinicians—to engage in an international conversation about our culture of scientific communication. Other scientific fields have come into the internet era, in which timely, global conversations about rapidly evolving scientific discoveries are stimulated. Clinical research has lagged far behind—it is time for us to catch up.