Networks, geometry and evidence
Last week at the 19th Cochrane Colloquium in Madrid, Professor John Ioannidis from Stanford University gave a riveting talk about the geometry of the evidence. Among the hundreds of articles he has published, his 2005 paper on Why Most Published Research Findings Are False is the most downloaded technical paper from the journal PLoS Medicine with over 400,000 views. Without question, he is a leader in addressing controversial issues in biomedical research.
In his lecture, he advocated for agenda-wide views of research by using network meta-analysis. Traditional meta-analyses are useful to compare two interventions, or an intervention with placebo. But what happens when there are dozens of randomised controlled trials on many different medications for the same medical condition? For example, there are 68 antidepressant drugs to choose from, how do healthcare professionals determine which one is the most effective? In fact, Ioannidis conveyed it would probably be stupid to depend on a single meta-analysis.
Enter the network. Ioannidis has been leading the development of multiple treatment meta-analysis or network meta-analyses. In its simplistic form, the networks map out all the interventions for a known condition in a lattice or network design. The network displays the number of trials relevant for a certain intervention and illustrates how they connect (or do not connect) to each other. The pattern of the comparisons is called the geometry of the treatment network.
For example, there have been 69 trials for smoking cessation that compared nicotine replacement with no active treatment but zero trials comparing it with the drug varenicline (Champix). To make an informed decision, clinicians need information comparing interventions.
Also, when you look at funding of the clinical trials, you find that head-to-head comparisons of interventions owned by different companies are uncommon. In fact, you find many “auto-loops” showing that the majority industry sponsored trials examine a single intervention owned by the company. Worse still, when two companies sponsor the same trial, it is not due to altruistic cooperation but usually due to co-ownership of the same agents.
Although network meta-analysis offers a wider picture than a traditional meta-analysis, they combine large numbers of trials and comparisons into one academic paper, which Ioannidis pointed out is not good for researchers CV. The current framework encourages narrowly defined systematic reviews and clinical trials which demonstrate effectiveness rather than zooming out to look at the big picture. Networks, although providing a cross-section of a clinical field at one point in time, provide insight into the current evidence base and can identify where connections are missing.
Going back to the above example on smoking cessation, we need trials comparing nicotine replacement with to varenicline (Champix), not another study showing that nicotine replacement compared to placebo is effective. But the problem is that the latter is easy to publish with a large effect size, but the former will probably show no difference and not make BBC headlines.
Nothing is as good as Obecalp
It’s been tested in more clinical trials than any other drug in history. Even critics admit it is effective for mild pain, mild depression, and other ailments with subjective outcomes. It has fewer side-effects than most alternatives. And now you can buy a bottle containing 50 pills of it for £3.75 ($5.95).
It’s called Obecalp and it comes in cherry flavor.
Here’s the rub: Obecalp is ‘placebo’ spelled backwards. Their website admits: “Obecalp® is a placebo and is not intended to diagnose, treat, cure or prevent any disease, or illness”. It was designed for parents to please (‘placebo’ means ‘to please’) malingering children. But don’t placebos work because people believe they are ‘real’? If so, I’ve removed the fun – and potential benefit – of Obecalp you. Or have I?
A recent study suggests placebos work even if people know they are placebos. There are two plausible explanations for why this might be the case. The first is classical conditioning. Pavlov’s dogs salivated at the mere expectation of food after having associated the sound with imminent food, and your body might react after popping Obecalp after it has learned (over a lifetime) to associate pill popping with symptom reduction.
Another novel explanation comes from evolutionary psychology. Imagine it is the dawn of human evolution. You feel a flu coming on but a sabre-toothed tiger is attacking your clan. If your body evoked the full immune response there would be less energy for fighting the tiger. Your survival therefore depends on ignoring the flu and increasing circulation to the skeletal muscles required for fighting. (In fact, the ‘fight or flight’ response suppresses the immune system.) Thankfully, you and your fellow tribesmen scare the tiger away, and the fight or flight response subsides. Then an authority figure – an elder or shaman – tells you everything is okay. The message from the trusted figure let you know (perhaps subconsciously) that you won’t need to run or fight for a few days. You are free to lie down and let your body’s immune system get to work.
Today tigers, village elders, and shamans are rare, but people with impressive titles like ‘Doctor’ might play the same role. If an authority figure (either in person or in an advertisement) tells you that you are going to be okay, your body (again perhaps subconsciously) might get the message that it is okay to allow the full immune response to kick in.
So even now that you know what it is, Obecalp still may be better than nothing. In the worst case, nothing is as good.
Malaria vaccines: fifty percent of what?
This week there was substantial news coverage about the effects of a new malaria vaccine. This is what Fergus Walsh of the BBC said, “The first phase three results, published online in the New England Journal of Medicine, show that the vaccine cut the risk of malaria infection and several malaria - by about half.” The Guardian’s Sarah Bosely chipped in with “The vaccine had been classed as around 55-60% effective,” and allAfrica.com reported “Malaria Vaccine Will Reduce Infection in Children By Half.”
Did you interpret the result to be meaningful, you and nearly everyone else?
Surely a 50% reduction in malaria is such a big effect that the vaccine is rolling of the production line right now. There is no doubt that a vaccine for preventing malaria would be a major health advance but the question immediately on mind was, 50% of what?
Only by accessing and working your way around the original publication can you get to the real figures. Which, I have to say, it wasn’t easy to get to, and interept the data I wanted. Is debate stifled by the incisive way journals publish papers which seemingly skirt around the data that matters, leaving you ever so slightly confused and dazed?
I think it's one of the reasons most media reports just focus on the press release. The papers themselves are written in a code which is only decipherable to mere humans after numerous re reads of the article. It probably also takes too long for most to read through the studies partcularly in the modern media world of instant news.
So what does the data reveal? The results of the trial are published in the New England Journal of Medicine shows during 12 months of follow-up in the first 6000 children the incidence of the first or only episode of clinical malaria was 0.44 per person-year in the vaccine group and 0.83 per person-year in the control group giving a relative reduction of 56%.
However, when you look at severe malaria 57 of 2830 children (2.0%) in the vaccine group and in 56 of 1466 children (3.8%) in the control group had at least one episode. A relative reduction of 47%.
As you can see the two figures for the same relative effect have very different absolute effects based on the severity of the outcome.
Yet, for death there was no difference: 56 of 5949 children (0.9%) died in the vaccine group and 28 of 2974 children (0.9%) in the control group.
So are you a relative or absolute person. I’m both, but I seem to be getting more sceptical the more headlines I read. You should be too
Just a spoonful of sugar, to help the obesity epidemic go down
Teaching on an Evidence Based Practice workshop recently I came across a truly jaw-dropping projection. A participant used a clinical scenario about obesity to bring up a paper about the US Obesity epidemic.The study is three years old now, but the authors use data on adults and children from the National Health and Nutrition Examination Study (NHANES), between the 1970s and 2004. The projections are frightening for the burden and health-care costs of obesity and overweight in the US if current trends continue. By 2048, all American adults would become overweight or obese. Depending on your ethnic background it could even be worse with the authors stating that Black women and Mexican-American men are likely to reach that state even earlier. However, the authors point out that they make a number of assumptions. Firstly they assume that the increase in obesity will be at its current rate. They also ignore the effect that all future policy, environmental and behavioural changes may have. Finally they ignore the possibility that some individuals may be genetically protected from becoming obese.
In the UK the predictions are not that much better. A recent four part series on obesity in The Lancet journal included a paper co-authored by Professor Klim McPherson of The University of Oxford. Their study stated that by 2030 there would be an additional 11 million more obese adults in the UK. This would have a knock on effect of an extra 6-8 million cases of diabetes, 5—7 million cases of heart disease and stroke and approximately 600,000 additional cases of cancer. This sort of news travels fast with The Daily Mail being one of the first to report these finding to the public.
So is it all doom and gloom? Not necessarily. In the same paper the authors go on to suggest that a 1% reduction in BMI (equivalent to a 1kg loss) across the entire population could avoid up to 2 million incident cases of diabetes, nearly 2 million new cardiovascular disease cases, and 73 000–127 000 cases of cancer. The authors further infer that this could be achieved through a reduction of 20kcal/day sustained over 3 years i.e. less about a teaspoon of sugar a day. Better still, if we were all able to give up 200 to 400 kcal/day, obesity levels would drop to 1990 prevalence levels.
So small changes may be the best way to start beating this epidemic. Something to think about before adding that spoon of sugar to your next tea or coffee.
Age ain't nothing but a number
Yesterday’s news hero was 100-year old Fauja Singh who finished the Toronto Waterfront Marathon to enter the Guiness Book of Records as the oldest man to ever complete a marathon. After reading so much about obesity, lifestyle risk factors and the chronic disease epidemic, it is great to hear that this Sikh gentleman has reached his age and maintained his fitness by sticking to ginger curry and tea.
Age and the “elixir of life” seem to be the theme of the news this week. Scientists fully decoded the genome of the world’s oldest woman who died in 2004 at the age of 115 years. This lady had no signs of dementia whatsoever and her good health has led to hopes that her genome will provide clues to her longevity. A herring-rich diet may have had something to do with it.
Ageing research is big money and a big priority. At the National Institutes of Health (NIH), over US$2.5 billion will be spent on ageing research this year, and the UK has been strengthening its ageing research portfolio and networks for the past few years. Scientific journals have blossomed around the search for life-prolonging knowledge. Did you know there was a journal called “Rejuvenation Research” ?
In clinical trials and research studies, if a difference is detected between two groups of individuals, often the first step in analysis is to “adjust for age”. In plain English, that means if we get rid of the differences between two groups which are just caused by differences in the ages of the individuals in the two groups, then we can assess if there are any other differences. It has always seemed counterintuitive to me that we do not also routinely adjust for sex, ethnicity, education, socioeconomic status and any number of other risk factors which can cause differences between groups of people.
With the make-up of our societies shifting more in the direction of elderly populations with increasing chronic diseases, the focus of research is shifting to how age itself contributes to disease processes and how we might reverse age-related processes. But are we focusing too much on just one factor? Age is undoubtedly a major contributor to many diseases and their underlying development but we cannot look at age in isolation. Fauja Singh and other healthy ageing adults reach their old age due to the complex interactions between genes, environment and chance, like every disease that medical science has so far uncovered. So to just look at his genes for the answer or for everybody to start on a diet of ginger curry does not make sense to me.
Do words speak louder than actions? Reflections after the summit.
I deliberately resisted blogging about it for a whole month, partly because I wanted to see the build-up and the aftermath of it, and partly because I wanted to see what everybody else said about it. I am, of course, talking about the UN High Level Meeting for non-communicable diseases (NCDs) on 19th and 20th September. The only other occasion that the UN has met like this on a health issue is for HIV/AIDS. The stakes were and are undoubtedly high and momentum has gathered and continues to gather. Yet there is definitely a sense of anti-climax after the summit. The organisation and run-up to the meeting seems to have been dogged by logistical but more importantly, by blocks through governments and multinational companies concerned about hard outcomes from the meeting.
I followed ex-BMJ editor, Richard Smith’s blog keenly throughout the meeting. It does not seem like the key stakeholders were invited or asked for opinions in time. As he said, probably the best thing to come out of the meeting and the pre-meeting is the NCD Alliance, a robust organisation of all the important stakeholders joined for the common purpose of raising awareness.
There is no question that awareness has been raised, and the summit was covered by medical journals, including the Lancet and the BMJ. Interestingly, the major American journals were notable by their silence on the first NCD summit of its kind. Yet, these same journals are the first to publish trials of expensive therapies for cardiovascular diseases, often with major conflicts of interest. On the subject of conflicts of interest, there are particular concerns about NCDs and the way big company interests are still able to play a major role in the UN.
The newspapers and other press gave coverage, but not as much as one would hope. Some of the loudest calls for actions came from celebrities such as Jamie Oliver. Looking around the WHO website, I did not find much in the way of concrete outcomes.
During my clinical cardiology practice, I have asked doctors and patients in the last few months if they knew that the meeting was happening. Almost without exception the answer was “no”.
On the second day of the summit, Richard Smith depressingly concluded:
“The big failures for me have been the failure to raise understanding of NCDs among the wider population and—as discussed at this morning’s meeting—the failure to recognise the need for a system that has at its centre patients not professionals.” It does not come much more damning than that.
What is the point of global health and health policy if it is not engaging at all with end-users? At the BMA Global Health event last week, I could not help feeling that this is an issue across global health and health policy-makers. Words are important but so are actions, particularly if the global fight against NCDs is to succeed.