HONcode Certified

This website is certified by Health On the Net Foundation. Click to verify.

This site complies with the HONcode standard for trustworthy health information: verify here.

Understanding evidence-based medicine in 4 days. Lesson 4: The big picture and asking the right question

Ami Banerjee
Last edited 17th March 2010

There are several historical lessons showing why the results of studies and trials should always be viewed in the broader context of all the knowledge in that area. The most commonly used cautionary tale is that of babies lying on their side and risk of sudden infant death. The unfortunately named Dr Benjamin Spock first published his famous book, “Baby and Child Care”, in 1946. In it, he advocated lying babies on their side and sold 19 million copies. Trials as early as the mid-1980s clearly showed that there were more deaths in babies lying on their side compared with babies lying on their backs. However, scientists continued to conduct over 20 more trials which all showed the same result. If these scientists had conducted a proper systematic review and combined the results of previous analyses (meta-analysis), they would have found that further trials were totally unnecessary because the data already showed that laying a baby on its side was harmful. Instead, their trials actually led to tens of thousands of infant deaths which may have been avoided if practice had changed before 2003. Setting the results of new studies in the context of a systematic review of the results of all other relevant studies would become straightforward if systematic reviews were always done before embarking on new research. In new areas of research, such reviews should be performed as data is accumulated in order to look at overall “pooled” trends.

James Lind, a Scottish physician, is credited with performing the first systematic review in 1753, titled “Treatise of the Scurvy”. In this work, he noted,

“As it is no easy matter to root out prejudices, …. it became requisite to exhibit a full and impartial view of what had hitherto been published on the scurvy, and that in a chronological order, by which the sources of these mistakes may be detected. Indeed, before the subject could be set in a clear and proper light, it was necessary to remove a great deal of rubbish.”

His observations have stood the test of time. A systematic review must involve 4 steps: (1) a clearly formulated question; (2) finding relevant studies; (3) appraisal of quality of the studies; and (4) summary of the evidence by use. The first step is crucial, not just in systematic reviews, but in any area of evidence-based medicine. Four aspects of any study question must be clearly defined in order to make any results meaningful: (1) the population being studied; (2) the intervention or exposure being studied; (3) the comparison group used in the study; (4) the outcome that was measured in the study.

Meta-analysis just means that we are combining the numbers from individual studies or trials to give the overall effect from all available data. A meta-analysis of data can only be done if the included studies are comparable and this process will give weighting to studies with larger numbers of patients and more precise data. For an example, see my previous blog regarding aspirin in primary prevention.

The BBC this week reported that patients do not need to fast before having their cholesterol tested , and that this could save greatly on the cost and convenience of testing for cholesterol in patients. This conclusion was only possible because of a systematic review done by Cambridge researchers, published in the Journal of the American Medical Association. They looked at the available evidence for measuring cholesterol and lipids in the blood and cardiovascular risk, which involved going through the individual records of over 300 000 patients involved in 68 long-term studies. Nobody said that doing systematic reviews was always easy but if we don’t do them, we will miss the big picture.

Twitter TrustTheEvidence.net


Search the TRIP Database

TRIP Database


Recent Comments