HONcode Certified

This website is certified by Health On the Net Foundation. Click to verify.

This site complies with the HONcode standard for trustworthy health information: verify here.

Peter Gill's blog

A recent article published in the BMJ raises questions about the extent and type of publication bias that exists in the literature. Publication bias is the selected publication of studies based on the results, such as only publishing studies that demonstrate a drug works while not publishing studies that demonstrate harms.

The study authors, including Ben Goldacre author of the best-seller Bad Science, explore the potential implications of study funding and high reprint orders. They contacted the editors of the top general medical journals (i.e. JAMA, Lancet, NEJM, Ann Intern Med, and BMJ) and requested information on the 20 articles with the highest number of reprint orders. After matching the articles with controls, the authors evaluated whether study funding (i.e. industry, mixed, other or none) was associated with higher numbers of reprints.

The results are telling. The Lancet led the way with a median of 126,350 reprints for the top articles with a range from 24,000 to 835,100. The BMJ was a distant second with a median of 13,248 (range 1,000 to 526,650). Unfortunately JAMA, NEJM and Ann Intern Med did not provide information.

Overall, compared with controls papers with high reprint orders were considerably more likely to be funded by the pharmaceutical industry (odds ratio 8.64, 95% CI 5.09 to 14.68). In addition the cost for reprint orders ranged from £4,002 to £1,551,794: reprints are evidently a lucrative source of supplementary income for journals.

While not designed to detect publication bias, the article highlights the importance of thinking outside the box. Evidence-based medicine is filled with cutting edge issues that are continually evolving and emerging. Do you think that a paper with potentially high reprint orders may affect an editor’s decision to publish? Should journals disclose the number of reprints for each article?

If you are keen to learn more, consider attending Evidence Live, a conference unlike any other event in healthcare, bringing together the leading speakers in evidence-based medicine from all over the world. The conference will include a session dedicated to Publication Bias at Evidence Live 2013 with an international line-up of speakers including Doug Altman, An-Wen Chan, Tom Jefferson and many more.

What do you think are undiscovered sources of publication bias? Here's your chance to share your thoughts with the experts at the University of Oxford, 25-26 March 2013.

*Note: this blog has also been posted on Evidence Live Blog.

Publication bias: big problem for children

Peter Gill
Last edited 6th May 2012

A recent study in the journal Pediatrics reported that only 29% of clinical studies in children have been published. This finding reinforces previous studies that there is significant publication bias in paediatric studies. These findings are a cause for serious concern.

What is publication bias? Essentially, it is the selected publication of studies based on the results, such as only publishing studies that demonstrate a drug works while not publishing studies that demonstrate harms.

Publication bias is a serious problem in healthcare and can have a large influence on treatment decisions by only providing limited information. Researchers have demonstrated substantial publication bias in certain areas such as the antidepressant medication reboxetine.

Several initiatives have been spearheaded to help reduce publication bias. The creation of open-access journals have shifted the focus from the importance of the results (as judged by a journal editorial committee) to the methodological rigour by which the study was completed.

But more importantly has been the creation of online trial registries, such as ClinicalTrials.gov launched in 2000. These registries serve as central databases of all the current and on-going clinical studies. Registration is optional, however in 2005 the ICMJE made registration of clinical trials as a pre-requisite of publication. Although this does not represent all journals, it sent a strong message of the importance of registration.

However despite the creation of trial registries, less than half of US based National Institute of Health (i.e. government) funded trials in children were registered on ClinicalTrials.gov. Another important finding was the lack of information included on the registries. One-third of all clinical studies terminated early did not provide any information about why they were stopped. The situation was similar for suspended studies with one quarter not providing information.

Were these studies stopped because of harms? Were the investigators no longer able to recruit children to enroll? Whatever the reason the studies were stopped, this information must be made public.

Registration of all clinical studies involving children must be made mandatory. This is the only way to minimise publication bias and increase the reporting of research. This would create massive industry uproar, but is it ethical to enroll children in a clinical study without having it publicly registered? At a minimum any trial that receives government funded must be registered.

However registration of studies is only one element of the formula. What about the dissemination of the results? Less than 10% of completed studies in children had results posted and publicly available. With the low publication rates of registered studies, and the even lower rate of posting results, how much information is still missing?

Indeed progress has been made to increase the quality and transparency of clinical studies in children but more is needed. We cannot assume that because trial registries exist that they are being used. Complacency must be replaced with compliance. It seems that more often than not, the little ones have the biggest problems.

Increasing access to journals through peer reviewers

Peter Gill
Last edited 23rd April 2012

A recent letter to the editor in the Canadian Medical Association Journal (CMAJ) highlighted a potential opportunity to increase access to subscription-based journals for individuals in low and middle-income countries. It turns out that a few journals give their peer reviewers’ free journal access or a free subscription as a thank you gift for their effort.

How widespread is this policy in the medical publishing world?

Unfortunately not very. Of the 21 journal editors contacted (including CMAJ, Lancet, BMJ, JAMA, etc.), only three actually provide reviewers with free journal access. The gift ranges from a 3-month (Lancet) to a 12-month subscription (BMJ and the Journal of Intellectual & Developmental Disability).

On the positive side, 20 out of the 21 journals were members of the Health InterNetwork Access to Research Initiative (HINARI), the World Health Organisation’s programme to provide free or low cost online access in the developing world to scientific research. This is encouraging, but HINARI is not perfect and many are still left without access.

For example, a BMJ Rapid Response highlighted that health care workers in middle-income countries such as Malaysia are often caught in the middle. Too rich for aid but not wealthy enough to afford the high cost of journal subscriptions.

However, despite the fact that most original research relevant to low and middle-income countries is open-access, the majority of the education articles, clinical reviews, news pieces and commentaries are still often behind firewalls that require payment.

The move for more open-access journals is encouraging. For example, the Howard Hughes Medical Institute, the Max Planck Society and the Wellcome Trust will be launching eLife, a new open access journal later this year.

Why don’t more journals provide peer reviewers with free subscriptions? Not only is it a symbol of appreciation for the hours of gratuitous time altruistically invested but it could be used to help others. Likely there is no pressure or demand for it. The majority of peer reviewers are already at academic institutions with subscriptions.

Has this happened to you before? What have you done with this free gift? Although few journals seem to be endorsing this policy, it may serve as a small way to increase access to those who otherwise do not have it.

In the future, if you review an article for the BMJ, the Lancet or the Journal of Intellectual & Developmental Disability (or other journals that provide free subscriptions after peer reviewing), rather than deleting the email consider who might benefit.

If you review for a journal that does not, ask the editor why not?

Twitter TrustTheEvidence.net


Search the TRIP Database

TRIP Database


Recent Comments