HONcode Certified

This website is certified by Health On the Net Foundation. Click to verify.

This site complies with the HONcode standard for trustworthy health information: verify here.


Shocking maths at the BBC - why most research findings are false

Carl Heneghan
Last edited 4th November 2010

Some studies suggest people have trouble with maths. I am having trouble understanding what Fergus Walsh as the BBC health correspondent tonight is doing reporting this stuff as major BBC news .

The study authors, quote ‘Applying a tiny electrical current to the brain could make you better at learning maths, according to Oxford University scientists.’ They ‘found that targeting a part of the brain called the parietal lobe improved the ability of volunteers to solve numerical problems.

They hope the discovery, reported in the journal Current Biology, could help people with dyscalculia, who may struggle with numbers.

Thank God they aren’t advising you to go round giving yourself shocks ‘We are not advising people to go around giving themselves electric shocks” I was thinking of wiring myself up to my bike lights and seeing if it improved my understanding of logistic regression. Oh well.

The study Modulating Neuronal Activity Changes
in Numerical Competence

TDCS is a realistic tool for intervention in cases of atypical numerical development or loss of numerical abilities because of stroke or degenerative illnesses.

I have no idea how they drew the conclusion with stroke patients. The maths that gives me an electric shock is the sample size: over 6 days, 15 healthy adults learned the association between nine arbitrary symbols.

Most of you, who read this column, will by now no the sample is so small it renders the results impossible to interpret.

This is what Ionaddis says on the issue in Why Most Published Research Findings Are False

The smaller the studies conducted in a scientific field, the less likely the research findings are to be true. Small sample size means smaller power and, for all functions above, the PPV for a true research finding decreases as power decreases towards 1 − β = 0.05. Thus, other factors being equal, research findings are more likely true in scientific fields that undertake large studies, such as randomized controlled trials in cardiology (several thousand subjects randomized) than in scientific fields with small studies, such as most research of molecular predictors (sample sizes 100-fold smaller)

It would be helpful if this sort of stuff was understood by organizations such as the BBC. Looks to me, a bit of evidence based science training wouldn’t go a miss. Maybe a bit of shock therapy would also help: better bring the bike lights then.

prostate 'magic drug', the case of the changing headline

Carl Heneghan
Last edited 12th October 2010

Earlier today the headline from the BBC read – Prostate trial drug like 'magic'. This headline concerened me for a number of reasons, as false hopes are raised by what I consider poor practice. The study was presented at this week's meeting of the European Society for Medical Oncology in Milan, Italy.

Funded by Johnson & Johnson it reports patients with advnaced prostate cancer survived on average four months longer, some did even better: I assume some did worse than this. But for those interested in the results there is no way of verifying the methods or the data. Unless someone can point me in the direction of the paper.

The following quote says it all: ‘This study was presented at a medical conference. The findings should be considered preliminary as they have not yet undergone the "peer review" process.

It hasn’t been peer reviewed, yet its making the major news. There could be major flaws in this study, which may invalidate the results. Unfortunately I can’t tell you, as I haven’t yet learnt how to do critical appraisal without the paper, whether published or unpublished.

By late afternoon the BBC had downgraded the headline to Trial confirms prostate drug promise. I'd have gone for 'prostate trial confirms conference proceedings lack of peer review

This is a worrrying trend in medicine and it’s about time something was done – important results should be released with the publication.

Oh, by the way the drug hasn’t even got a European license yet. What's magic about that.

Twitter TrustTheEvidence.net


Search the TRIP Database

TRIP Database


Recent Comments