Networks, geometry and evidence
Last week at the 19th Cochrane Colloquium in Madrid, Professor John Ioannidis from Stanford University gave a riveting talk about the geometry of the evidence. Among the hundreds of articles he has published, his 2005 paper on Why Most Published Research Findings Are False is the most downloaded technical paper from the journal PLoS Medicine with over 400,000 views. Without question, he is a leader in addressing controversial issues in biomedical research.
In his lecture, he advocated for agenda-wide views of research by using network meta-analysis. Traditional meta-analyses are useful to compare two interventions, or an intervention with placebo. But what happens when there are dozens of randomised controlled trials on many different medications for the same medical condition? For example, there are 68 antidepressant drugs to choose from, how do healthcare professionals determine which one is the most effective? In fact, Ioannidis conveyed it would probably be stupid to depend on a single meta-analysis.
Enter the network. Ioannidis has been leading the development of multiple treatment meta-analysis or network meta-analyses. In its simplistic form, the networks map out all the interventions for a known condition in a lattice or network design. The network displays the number of trials relevant for a certain intervention and illustrates how they connect (or do not connect) to each other. The pattern of the comparisons is called the geometry of the treatment network.
For example, there have been 69 trials for smoking cessation that compared nicotine replacement with no active treatment but zero trials comparing it with the drug varenicline (Champix). To make an informed decision, clinicians need information comparing interventions.
Also, when you look at funding of the clinical trials, you find that head-to-head comparisons of interventions owned by different companies are uncommon. In fact, you find many “auto-loops” showing that the majority industry sponsored trials examine a single intervention owned by the company. Worse still, when two companies sponsor the same trial, it is not due to altruistic cooperation but usually due to co-ownership of the same agents.
Although network meta-analysis offers a wider picture than a traditional meta-analysis, they combine large numbers of trials and comparisons into one academic paper, which Ioannidis pointed out is not good for researchers CV. The current framework encourages narrowly defined systematic reviews and clinical trials which demonstrate effectiveness rather than zooming out to look at the big picture. Networks, although providing a cross-section of a clinical field at one point in time, provide insight into the current evidence base and can identify where connections are missing.
Going back to the above example on smoking cessation, we need trials comparing nicotine replacement with to varenicline (Champix), not another study showing that nicotine replacement compared to placebo is effective. But the problem is that the latter is easy to publish with a large effect size, but the former will probably show no difference and not make BBC headlines.
Oxygen and heart attack - what next?
Most medical students will recognize the quote:
‘Half of what you'll learn in medical school will be shown to be either dead wrong or out of date within five years of your graduation; the trouble is that nobody can tell you which half—so the most important thing to learn is how to learn on your own.’
Dave Sackett: “Old fart from the frozen north” “Father of EBM”
The rapid assessment and treatment of a patient with a heart attack is drummed into most medical students very early on in their training. ABC: airway, breathing, circulation. Part of that resuscitation is the delivery of Oxygen to patients with a heart attack, mainly due to the fact the flow of oxygenated blood in the heart is stopped for a period of time.
The idea for providing oxygen in a heart attack is it may improve the amount of oxygen of the cells in the heart that are dying mainly due to the lack of oxygen, ultimately reducing pain and the size of the dead heart muscle. To most this will make sense in terms of pathophysiological reasoning.
Today a Cochrane review by Cabello and Burls on Oxygen therapy for acute myocardial infarction looks at the evidence from randomised controlled trials to establish whether routine use of inhaled oxygen in acute heart attack infarction improves patient-centred outcomes, in particular pain and death.
Now, here is the half of what is learnt learn that may eventually be out of date:
Three trials involving 387 patients were included and 14 deaths occurred. The pooled relative risk of death was 2.88 (95% confidence interval 0.88 to 9.39) in an intention-to-treat analysis and 3.03 (95% confidence interval 0.93 to 9.83) in patients with confirmed heart attack.
While suggestive of harm, the small number of deaths recorded meant that this could be a chance occurrence. Basically, there is no conclusive evidence from randomised controlled trials to support the routine use of inhaled oxygen in patients with acute heart attack.
The neat thing about EBM is you are never really sure of which half is out of date; this review adds to that half. As the reviewers rightly state, we need an urgent large scale trial to unpick the uncertainty.