There’s a trend in medical literature that’s misleading both patients and practitioners: publication bias. The practice of positive trials having more chance of being published than negative ones, and negative trials painted in a more positive light than is warranted, means many important (albeit negative) results never reach the scientific community.
We assume (or hope, at least) doctors have access to the most up-to-date medical information. But, when key findings don’t appear in medical literature, doctors don’t have access to information that could affect the level of care patients receive. In worst-case scenarios, these biases have led to ineffective treatments, prolonged patient suffering, and wasted resources.
In an era of increased transparency and sharing, excluding negative trial results means scientific literature is unrepresentative, ultimately compromising scientific integrity.
The role of publication bias in science
Publication bias isn’t new. In his now infamous TED talk, Ben Goldacre described this longstanding issue as a “systematic flaw of the scientific basis of medicine.”
At the end of last year, an article in eLife examined how publication bias influences the way scientists present facts. The findings show that false claims can be canonized as fact, but also that this risk can be reduced through simple modifications to the publication process.
Carl Bergstrom, one of the authors, summarized: “The net effect of publication bias is that negative results are less likely to be seen, read and processed by scientific peers […] Our model showed that you need to publish more negative results – at least more than we probably are now.”
Much of the problem lies with an academic culture that supports ‘positive’ (or statistically favorable) results. Scientists want to build their status and advance their careers, which they achieve through the continual publication of high impact research outputs. And as humans, we respond well to incentives. Whether it’s an easier path to publication, a greater chance of citations, or faster career progression, there are several strong motivations swaying scientists today.
Everyone in the research-to-publication process has his or her own reasons for favoring the current system:
- Researchers prefer not to publish unpredicted ‘negative’ results
- Editors tend to discount ‘failures’ on the basis they are inconclusive and/or unimportant
- Publishers recognize that ‘failures’ have less impact and generate less interest compared to positive findings
New study shows transparency is improving
A number of measures are being taken to counter this bias. The most widely cited is the 2007 FDA Amendments Act (FDAAA), which mandated clinical trial registration and result reporting on ClinicalTrials.gov with the aim to increase transparency, responsibility and public benefit. In a context of mounting drug trial scandals, retractions and funding issues, the Act was welcomed, but it was littered with ambiguities and loopholes. These were addressed earlier this year with complementary rules, but the challenges are still present.
In July, a study published in Trials looked at impact FDAAA has had. The study compared the trials for newly approved drugs in cardiovascular disease and diabetes to those trials completed before the FDAAA took effect. The result suggested that, post-FDAAA, trials were more likely to be registered and published with findings that agreed with the FDA’s interpretation.
“Put simply, FDAAA made the clinical research supporting approval of new drugs by the FDA more transparent and, when published, less biased,” the study’s authors, Adam T. Phillips & Joseph S. Ross wrote in BioMed Central.
There are also a growing number of tools designed to detect publication bias in medical literature. Funnel plots are popular, and an alternative is the ‘p-curve analysis’, devised by Uri Simonsohn, a multidisciplinary psychologist at University of Pennsylvania. This method does not depend on a collaborative effort by authors, editors and publishers – instead it can be carried out by individuals or by small groups.
But a statistical method can only go so far and there is still a long way to go. Publication bias can occur at any stage of the research-to-publication process, so a multi-pronged approach is the only way to reduce the risk.
Luckily, attitudes towards biases in medical literature are changing. From new statistical methods and the AllTrials campaign to journals dedicated to reviewing and publishing studies with negative results, the industry is showing its commitment to greater transparency in all its activities.
Read More:
- Why are Medical Communications Teams Struggling to Manage Content?
- How the FDA is Modernizing its Review of Drugs
- 3 Ways Big Data Affects Biomedical Research
Looking for medical literature? Find the content you need in RightFind’s database of more than 140 million citations and 3+ million open access articles.