HONcode Certified

This website is certified by Health On the Net Foundation. Click to verify.

This site complies with the HONcode standard for trustworthy health information: verify here.

peer review

Let’s be honest. Most researchers and clinicians only read the abstracts of research studies. This is true even when they diligently search out and find the original article that inspired a news headline. A cynical colleague suggested that people only read the Tweets of someone that only read the abstract of the article. People are busy and pressed for time; skimming abstracts is an efficient way to stay up-to-date with research findings without onerously sifting through pages of details. People want the bottom line. But this approach inherently relies on journals to ensure accuracy in abstract reporting.

Well, it seems this strategy is problematic for many reasons, particularly because of “spin” or “specific reporting emphasizing the beneficial effect of the experimental treatment.” A recent study in PLoS Medicine sought to quantify this problem: the authors identified two-arm, parallel-group RCTs, searched for associated press releases, and examined both for the presence of “spin.”

What they found was concerning: 47% of press releases and 40% of abstracts contained “spin.” After completing a multivariable analysis, “spin” in the article abstract was the only factor associated with “spin” in the press release (RR, 5.6; CI, 2.8-11.0; P<0.001). Therefore, the major driver of inaccurately reported findings was written by the author. In fact, 31% of press releases misinterpreted the results from the trial, either over- (86%) or under- (14%) estimating the benefit of the therapy.

Press releases are an important part of research dissemination. A study completed earlier this year in the BMJ by Evidence Live Faculty Lisa Schwartz and Steve Woloshin found that high quality press releases by journals can influence media coverage of the associated article. Increased coverage is beneficial if the press release is accurate which relies heavily on the abstract.

People involved in conducting research understand the importance of the abstract. This is the first piece read by a journal editor once submitted, and the decision to peer review largely relies on the authors ability to ‘sell’ their study in 300 words or less. Therefore, there is an incentive for authors to (over) emphasize the main results of the study in a manner that is usually critiqued in the peer review phase. While changes may be made to the full-text article to “dumb down” the authors conclusions, it is unclear how much the abstract changes as a result of peer review.

Realistically, the previously described process is unlikely to change, and everyone is not going to start reading the full-text article, particularly the media. Therefore, the onus is on journals to take an active role to ensure accuracy in abstract reporting and press releases. If they don’t, who will? If you want to learn more about what journals are doing to tackle this problem, come and ask the editors yourself at Evidence Live 2013.

*Note: this blog has also been posted on Evidence Live Blog.

Increasing access to journals through peer reviewers

Peter Gill
Last edited 23rd April 2012

A recent letter to the editor in the Canadian Medical Association Journal (CMAJ) highlighted a potential opportunity to increase access to subscription-based journals for individuals in low and middle-income countries. It turns out that a few journals give their peer reviewers’ free journal access or a free subscription as a thank you gift for their effort.

How widespread is this policy in the medical publishing world?

Unfortunately not very. Of the 21 journal editors contacted (including CMAJ, Lancet, BMJ, JAMA, etc.), only three actually provide reviewers with free journal access. The gift ranges from a 3-month (Lancet) to a 12-month subscription (BMJ and the Journal of Intellectual & Developmental Disability).

On the positive side, 20 out of the 21 journals were members of the Health InterNetwork Access to Research Initiative (HINARI), the World Health Organisation’s programme to provide free or low cost online access in the developing world to scientific research. This is encouraging, but HINARI is not perfect and many are still left without access.

For example, a BMJ Rapid Response highlighted that health care workers in middle-income countries such as Malaysia are often caught in the middle. Too rich for aid but not wealthy enough to afford the high cost of journal subscriptions.

However, despite the fact that most original research relevant to low and middle-income countries is open-access, the majority of the education articles, clinical reviews, news pieces and commentaries are still often behind firewalls that require payment.

The move for more open-access journals is encouraging. For example, the Howard Hughes Medical Institute, the Max Planck Society and the Wellcome Trust will be launching eLife, a new open access journal later this year.

Why don’t more journals provide peer reviewers with free subscriptions? Not only is it a symbol of appreciation for the hours of gratuitous time altruistically invested but it could be used to help others. Likely there is no pressure or demand for it. The majority of peer reviewers are already at academic institutions with subscriptions.

Has this happened to you before? What have you done with this free gift? Although few journals seem to be endorsing this policy, it may serve as a small way to increase access to those who otherwise do not have it.

In the future, if you review an article for the BMJ, the Lancet or the Journal of Intellectual & Developmental Disability (or other journals that provide free subscriptions after peer reviewing), rather than deleting the email consider who might benefit.

If you review for a journal that does not, ask the editor why not?

What governments can learn from scientific enquiry

Ami Banerjee
Last edited 22nd December 2010

The last couple of weeks has been an depressing time for those of us who believe in freedom of information (FOI). As Wikileaks founder, Julian Assange, awaits his fate in Dickensian conditions for publishing secret memos which various governments believe should not have been in the public domain, I have been thinking about FOI, conflict of interest (COI) and the parallels with the world of research evidence.

In the UK, under the Freedom of Information Act, any individual, anywhere in the world, is able to make a request to a public authority for information. The information may be withheld only if the public authority considers that the public interest in withholding the information is greater than the public interest in disclosing it. The objection of the US government and others to Wikileaks has largely been that sensitive information may “threaten national or even global security”, and so should have been withheld. To a doctor without diplomatic training, this seems like a knee-jerk response to any disclosure of information which could be of interest to the public and is an insult to our intelligence as well as an overused excuse for preventing FOI. The problem is that the powers that be in global politics often have COI, which is really the reason for stopping FOI, rather than any threat to security.

FOI and COI are not new concepts in scientific research or indeed, medical care of patients. There have been several welcome moves in recent months to make research data and the research process more transparent. Undoubtedly, the age of blogs and Twitter have pressured debate and change in a way we could not have imagined. The British Medical Journal has been encouraging online rapid responses for some years, but is largely alone in inviting immediate comment and debate regarding its articles. A recent Science publication regarding “A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus” led to such a fervent online debate that even Nature, one of the most respected scientific journals, has had to admit that “Bloggers and online commentators have an important part to play in the assessment of research findings, and many researchers' blogs, in particular, contain better analyses of the true significance of a scientific finding or debate than is seen in much of the mainstream media”.

Although peer review has many strengths as a process, it has too often been the veil behind which other scientific journals hide information and COI, and it has its flaws. A Pubmed search with the terms “conflict of interest” leads to 8249 hits and “freedom of information” leads to 2583 hits, so these are clearly issues that occupy the minds of scientists. “Peer review” as a term reminds me of “national security”........In research, just as in government, COI is not just financial, although money does usually talk loudest.

Rather than resisting public pressure for greater freedom of information regarding political processes and imprisoning citizens without charge, perhaps governments would do well to invite debate and comment in the way that the scientific journals are now being forced to.

Twitter TrustTheEvidence.net


Search the TRIP Database

TRIP Database


Recent Comments