The importance of abstracts and press releases: the issue of "spin"
Let’s be honest. Most researchers and clinicians only read the abstracts of research studies. This is true even when they diligently search out and find the original article that inspired a news headline. A cynical colleague suggested that people only read the Tweets of someone that only read the abstract of the article. People are busy and pressed for time; skimming abstracts is an efficient way to stay up-to-date with research findings without onerously sifting through pages of details. People want the bottom line. But this approach inherently relies on journals to ensure accuracy in abstract reporting.
Well, it seems this strategy is problematic for many reasons, particularly because of “spin” or “specific reporting emphasizing the beneficial effect of the experimental treatment.” A recent study in PLoS Medicine sought to quantify this problem: the authors identified two-arm, parallel-group RCTs, searched for associated press releases, and examined both for the presence of “spin.”
What they found was concerning: 47% of press releases and 40% of abstracts contained “spin.” After completing a multivariable analysis, “spin” in the article abstract was the only factor associated with “spin” in the press release (RR, 5.6; CI, 2.8-11.0; P<0.001). Therefore, the major driver of inaccurately reported findings was written by the author. In fact, 31% of press releases misinterpreted the results from the trial, either over- (86%) or under- (14%) estimating the benefit of the therapy.
Press releases are an important part of research dissemination. A study completed earlier this year in the BMJ by Evidence Live Faculty Lisa Schwartz and Steve Woloshin found that high quality press releases by journals can influence media coverage of the associated article. Increased coverage is beneficial if the press release is accurate which relies heavily on the abstract.
People involved in conducting research understand the importance of the abstract. This is the first piece read by a journal editor once submitted, and the decision to peer review largely relies on the authors ability to ‘sell’ their study in 300 words or less. Therefore, there is an incentive for authors to (over) emphasize the main results of the study in a manner that is usually critiqued in the peer review phase. While changes may be made to the full-text article to “dumb down” the authors conclusions, it is unclear how much the abstract changes as a result of peer review.
Realistically, the previously described process is unlikely to change, and everyone is not going to start reading the full-text article, particularly the media. Therefore, the onus is on journals to take an active role to ensure accuracy in abstract reporting and press releases. If they don’t, who will? If you want to learn more about what journals are doing to tackle this problem, come and ask the editors yourself at Evidence Live 2013.
*Note: this blog has also been posted on Evidence Live Blog.
Ethics in media and medical research-reflections after the phone-hacking scandal
It is impossible to avoid the outrage and scandal of the News of the World if you were in the UK this week. One media organisation went too far in its pursuit of sensitive data by tapping phones, and there have been hourly revelations since then, revealing a much greater system problem. Has the media lost its way? Yes it has. The “big scoop” has become more important than respect for individuals and their lives, and the right to know has trumped the right to privacy.
As the scandal crosses the Atlantic and more people lose their jobs, the last few days made me reflect about analogies between the media and medical research, and possible lessons for EBM. Like the news headline writers, data is the major part of what medical research is about as well. It is sensitive data because the content is health information of patients. Thankfully, medical research has to go through multiple levels of ethical review before it is carried out and perhaps the rather lame Press Complaints Commission needs to be replaced with a proper code of ethics for journalism and a body that can enforce it.
The history of medical research has had plenty of ethical issues and scandals which have largely led to improvements in guidelines and the way in which research is conducted. Perhaps most importantly, the involvement of Nazi doctors in unethical research during the Holocaust led to the establishment of the Nuremberg code: ten principles of ethical conduct in medical research in 1949. The World Medical Association developed the Declaration of Helsinki to guide the medical research community regarding human experimentation. But scandals still continue to happen. The impact of Andrew Wakefield’s bogus Lancet paper about MMR has led to huge consequences for childhood immunisation across several countries. Conflicts of interest in research and clinical practice have led to inappropriate regulation of medical devices.
Medical researchers have the privilege of working in fields where their findings can genuinely impact human lives, and ethical research can be done in every area of medicine to make treatment and the patient experience better, even in end-of-life decisions in the intensive care unit.
As a researcher with an interest in epidemiology, a lot of my work involves using patient databases to formulate and test hypotheses. The aim is not “to publish or perish” and get as many papers from the data as possible, and nor is it financial reward. We as researchers should always remember why we are doing what we do: to discover more about health and disease in order to improve outcomes for patients.
Everyone has an interest in exaggeration
Think about health care communication to the public. Is it done well, does it lead to informed choice? Do you feel skeptical about the issue, and disillusioned with the current status of communication of health to a wider audience.
Today, at Kellogg College, at the University of Oxford a workshop on ‘Enhancing the Public Understanding of Health Research’ aimed to bring together folk with experience in developing and evaluating materials to help people become better users of health research. The question is, why isn’t the public informed? The two issues when we are faced with research findings, whether it be on TV, in the newspapers or in a scientific journal, are ‘Why should I believe these results,” and “What do they mean?”
Steve Woolshin and Lisa Schwarz authors of the ‘Know Your Chances’ book, talked enthusiastically, and with an array of examples, of the bigger picture and what is out there.
The issue that caught my eye was how invested we all are in exaggeration: manufacturers to sell products, academics to get their research published, journals to get their research cited and picked up by the news and for media to gain more advertising revenue. And so the cycle goes on.
For instance, Dannon's Activia, exaggerated health claims led to a US $21M fine, a further example is a study which suggests, ‘nutrient enhanced water drinks are ''''expensive lolly waters'''' with exaggerated health claims’. All sorts of exaggerations occur on a daily basis ‘Radiation health fears exaggerated, claims Oxford professor’. Just try googling ‘exaggerated health claims’.
My favourite is vitamin D which has reached rock-star status in recent years, as a potential cure for the prevention of everything.
The BBC even highlights medical journals are all part of the exaggeration phenomena: ‘Medical journals have been accused of hyping up the findings of the research that they publish.’
Am I prone to exaggeration, you bet I am! Obviously this is the best site worldwide to find the opinion on the evidence to trust.