Tags

HONcode Certified

This website is certified by Health On the Net Foundation. Click to verify.

This site complies with the HONcode standard for trustworthy health information: verify here.

autism and brain scan test: the real predictive value

Carl Heneghan
Last edited 11th August 2010

A brain scan that detects autism in adults could mean much more straightforward diagnosis of the condition, scientists say. Reported the BBC, Sky the Guardian and many more.

I had great difficulty getting hold of this paper, it wasn’t published online at the time of the press release. I managed to get a copy via Ben Goldacre at Bad Science and Evidence Matters who sent me the full text. Given this problem in getting the paper, it is highly likely no one who released the story has actually read the paper.

The news all report the headline ‘The researchers detected autism with over 90% accuracy, the Journal of Neuroscience reports.’

Sounds impressive, but this is one of the most obvious mistakes to make in interpreting a diagnostic test result. Never mind this is not the correct study type.

What has happened is the sensitivity has been taken for the positive predictive value, which is what you want to know: if I have a positive test do I have the disease?

Sensitivity: The proportion of people with disease who have a positive test.
Positive predictive value (+PV): The proportion of people with a positive test who have disease.

So, for a prevalence of 1% the actual positive predictive value is 4.5%. That is about 5 in every 100 with a positive test would have autism. Even at a prevalence of 2%, only 8.5% would be correctly identified.

Suddenly, not that great a test. This has to be one of the worst examples of misinterpreting diagnostic test results in the media I’ve ever seen.

Many problems in the paper

Yes, the paper has many problems and weaknesses, mostly related to data-mining and small samples. With 20 NTs in the study, it is not possible to say anything about the real value in the general population if ASDs are at 1%. Also, with no people with relatives with ASDs (but without being diagnosed themselves), we have no idea how it would deal with diagnostic issues either. That a test can separate autistics from NTs is not even close to enough for a real diagnosrtic test. Another issue is that media seems to see this as the first step in finding "the defect" behind ASDs, when the method used needs many differences to provide good results. If it holds up, it is more like big support for the neurodiversity concept rather than a step in the direction of finding the cause of ASDs.

I've also commented this in this blog entry: http://blog.rdos.net/?p=90

+PV

Who would suggest using this test as a screening tool for the general population with an autism prevalence of 1%?

This has to be one of the worst examples of misinterpreting diagnostic results on the internet I've seen.

William Osler IV

Written about this too

Carl,

Thanks very much for pointing me in the direction of the paper. I've written about it here: http://post.ly/rKrS

Ian

Great put-down

This is news the media want to not hear.

Michael.

Twitter TrustTheEvidence.net

tte
     

Search the TRIP Database

TRIP Database

 

Recent Comments