HONcode Certified

This website is certified by Health On the Net Foundation. Click to verify.

This site complies with the HONcode standard for trustworthy health information: verify here.

Highlights of the year in Evidence-Based Medicine

Carl Heneghan
Last edited 1st January 2011

The 1st of January saw the year start with a retweet from @david_colquhoun on a scary story on Big Pharma and US healthcare. We learnt authors of BMJ articles are reluctant to respond to criticisms. Reluctant in medical speak, to cebmblog, translates in to ‘can’t be bothered’. A linked editorial highlighted ‘that all aspects of post-publication review are wanting in medical research.’ Whislt the peer review process continued to create fuss, and may finally be broken.

Shriger and Altman in a linked editorial state ‘the majority of research articles receive no critique, and, for the minority that do, authors often do not reply or reply but do not respond to the criticism.’ Will we see more criticism in 2011, probably not.

In May, Isabelle Boutron, highlighted in JAMA the reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. Guess what? Previous studies indicate the interpretation of trial results can be distorted by authors of published reports. Spin was identified in the conclusions sections for 58% of reports. Hold your hand up if you are guilty.

In terms of what drives health costs? You are twice as likely to have an x-ray, ultrasound or imaging after seeing a self-employed urologist as opposed to a salaried urologist Staying on the diagnostic theme, we learnt we should not be pooling diagnostic likelihood ratios in systematic reviews.

Small studies gave us the wrong answers, and Docs suggested women were hitting the pause button on osteoporosis drugs, which sounded very much like the HRT story.

On the drug front, we managed to give out 300,000 prescriptions for sibutramine in the UK in the last yr. Put me in charge of the drug budget, I’ll show you some cuts. Yet, nearly 1 in 10 prescribed meds were not taken by hospital inpatients. Hard to know what is the point of being in hospital but one of the major reasons was drugs were not available.

A pretty sobering statistic from 2010 is one-third of world's population is without access to surgical services

In contrast, a 10-year stroke prevention study after successful carotid endarterectomy had no surrogate, no early stopping, proper research for a change. But, bad news in that adherence statistics showed ¼ of stroke patients stopped taking their meds within 3 months and useful stroke studies remained unpublished: 1 in 5 studies were not published.

Jamie Oliver waded into the health debate @jamie_oliver: "not a day goes by I dont see a patient who could stop one med if they ate healthier." He needs to get out more, as he doesn’t see enough patients. Although his TED Prize wish: Teach every child about food was a great watch for Easter.

However, good news arrived by Xmas, despite all the snow, the FDA was aiming to keep us safe from harmful diet supplements which contain deceptively labeled ingredients, whilst the media was easily fooled by the Xmas BMJ edition. The BBC, at times, continued to produce shocking maths –why most research findings are false and, highlights for cebmblog were the prostate 'magic drug', the case of the changing headline

It’s a good job NHS choices continued to make sense of the news in 2010 : Put your feet up to slim down?

Oh, and Docs continue to be outed @ProPublicafor being paid by pharma despite government sanctions. However, they take fewer freebies, but disclosure is poor: really, 71% still accept gifts.

In addition, we learnt a bit more about ghost writers from @bengoldacre: Worst of all is the complicity of the academics And we became aware we had to be careful what you tweet ; when Bad Science clashed with Gillian McKeith.

Whilst the biggest problem may be consumer organizations: in 2010 2/3rds working with European Medicines Agency (the drug regulators) received partial or significant funding. Or is it Drug company study results are the major problem: drug company studies 85% positive, Government 50% positive. Must do more drug trials, if I want to get ahead. Or should I join the WHO: WHO admits to "inconsistencies" in its policy on conflicts of interest, whillt the BMJ editor pulled no punches WHO leadership may need to resign.

Finally the biggest change of the year saw the Avandia saga come to an end, well nearly, and four years to late. The recorded meeting between GSK and Dr Nissen in 2007 was published due to a subpoena, whilst Panorama ran the Avandia story asking why a million prescriptions were still issued in the UK.

And, after 16 yrs cebmblog thought he knew everything on calculating NNTs. Read this neat piece by Smeeth, I don't. Still more EBM to learn then for 2011, and my final thought for 2010 is: you better watch out, you better not lie. Here’s the reason I’m telling you why

Cheers cebmblog

21st century critical appraisal


Thanks for the neat summary of the past year's EBM highlights --- I am glad you passed by the opportunity to call them lowlights.

The revelations from the past year make me wonder if now is not the time to update the standard critical appraisal checklist.

Studies should now be classified by the sophistication of their designers/promoters in promoting a point of view.

Critical appraisal of naive studies requires the standard 20th century skeptical approach of EBM: look for risks of bias in randomization, allocation, maintenance, blinding, and other methodological problems that arise from unconscious cognitive biases and innocent errors.

Critical appraisal of sophisticated studies requires a more cynical 21st century approach: the RAMBO boxes will all have been ticked, so don't bother checking them; instead look for evidence of spin, bigging up, ghost writing, multiple publication of favourable results, and deliberate withholding/suppression of inconvenient results. Fraud is not on the list - yet.

Wishing you all the best for 2011, and very happy EBMing.

update checklist

Thanks hmichaelpower

agree with you on the updated checkilst almost seems quite easy to meet the traditional Rambo, particularly for Pharma

whereas issues like surrogates, composites, funding, spin currently far more important and have lots of emprical evidence

therefore be interested in rewrite aprriasal list for 2011 - BMJ here we come

email me

Cheers Carl

Twitter TrustTheEvidence.net


Search the TRIP Database

TRIP Database


Recent Comments