A recent study in the journal Pediatrics reported that only 29% of clinical studies in children have been published. This finding reinforces previous studies that there is significant publication bias in paediatric studies. These findings are a cause for serious concern.

What is publication bias? Essentially, it is the selected publication of studies based on the results, such as only publishing studies that demonstrate a drug works while not publishing studies that demonstrate harms.

Publication bias is a serious problem in healthcare and can have a large influence on treatment decisions by only providing limited information. Researchers have demonstrated substantial publication bias in certain areas such as the antidepressant medication reboxetine.

Several initiatives have been spearheaded to help reduce publication bias. The creation of open-access journals have shifted the focus from the importance of the results (as judged by a journal editorial committee) to the methodological rigour by which the study was completed.

But more importantly has been the creation of online trial registries, such as ClinicalTrials.gov launched in 2000. These registries serve as central databases of all the current and on-going clinical studies. Registration is optional, however in 2005 the ICMJE made registration of clinical trials as a pre-requisite of publication. Although this does not represent all journals, it sent a strong message of the importance of registration.

However despite the creation of trial registries, less than half of US based National Institute of Health (i.e. government) funded trials in children were registered on ClinicalTrials.gov. Another important finding was the lack of information included on the registries. One-third of all clinical studies terminated early did not provide any information about why they were stopped. The situation was similar for suspended studies with one quarter not providing information.

Were these studies stopped because of harms? Were the investigators no longer able to recruit children to enroll? Whatever the reason the studies were stopped, this information must be made public.

Registration of all clinical studies involving children must be made mandatory. This is the only way to minimise publication bias and increase the reporting of research. This would create massive industry uproar, but is it ethical to enroll children in a clinical study without having it publicly registered? At a minimum any trial that receives government funded must be registered.

However registration of studies is only one element of the formula. What about the dissemination of the results? Less than 10% of completed studies in children had results posted and publicly available. With the low publication rates of registered studies, and the even lower rate of posting results, how much information is still missing?

Indeed progress has been made to increase the quality and transparency of clinical studies in children but more is needed. We cannot assume that because trial registries exist that they are being used. Complacency must be replaced with compliance. It seems that more often than not, the little ones have the biggest problems.

An earlier TrustTheEvidence.net blog post on the geometry of evidence described the importance of network meta-analyses. These indirect methods of analysis compare the results from two or more studies that have one treatment in common when comparative effectiveness (CE) research is lacking.

What is comparative effectiveness research? To quote the US Federal Coordinating Council for Comparative Effectiveness Research Report to President Obama in 2009, it is defined as the:

“generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings”

Additional studies that compare one drug to placebo are not incredibly useful when we already know they work. The real challenge of evidence-based practice is determining which treatment to use when all ten available drugs are better than placebo. How do clinicians decide which one to prescribe? All too often decisions are built on studies lacking active comparators. This is not high quality care for patients.

A recent study published in PLoS ONE evaluated trials registered in ClinicalTrials.gov that focused on the top 25 topics identified as priority areas by the US Institute of Medicine (e.g. treatment of atrial fibrillation). The authors looked at studies conducted in the US between 2007 and 2010 and determined the prevalence of CE research.

Despite the importance of this research methodology, only 22% of studies were CE studies and their characteristics varied substantially based on the funding source. Primarily industry-funded studies had the shortest duration of follow-up and were more likely to report positive findings compared to studies with any government funding.

As usual, children get left out. Industry-funded studies were less likely to enroll children when compared to government or nonprofit funded trials. The lack of controlled trials in children is already a problem and there may be a perceptions among drug manufacturers that testing drugs in children brings the risk of increased liability.

The authors hypothesise that the increase in CE research will lead to an increase in the number of studies that fail to support new interventions. Not good for big pharma, but why?

First, trials with inactive comparators (i.e. placebo) are more likely to achieve favourable findings. On the contrary, CE studies tend to produce conservative results regarding the superiority of a therapy compared to other active treatments.

Second, industry-funded the majority of drug and device CE studies meaning that most were designed and conducted by the company marketing the product. There is substantial evidence that these studies are more likely to report positive findings supporting the use of a product. The PLoS ONE study provides further evidence that even in CE research industry-funded studies were more likely to report an outcome favouring the use of the intervention.

But it’s not all doom and gloom. The US has allocated $1.1 billion to CE research. This added investment of noncommercial funding will be critical to provide unbiased answers and evaluate under-studied populations (e.g. children). It’s about time we provide with stronger evidence.

Today in the BMJ we have published on the current saga of ensuring the data and the problems we have found with the inconsistencies between what is published and what is in the Clinical study reports

In the last six months I have been working with a collabaration which includes Peter Doshi, Tom Jefferson Chrsi Del Mar, Matthew Thompson and a few other folk round the world. We have been busy returning to the problems with the evidence-base of Neuraminidase Inhibitors (e.g tamiflu) used extensively last year for swine flu and this year.

We met in Oxford in September to work out the methods for the analysis, this is what Tom Clarke of Channel 4 reported about the meeting at the time:

‘An international group of scientists has today raised further concerns about the lack of evidence supporting the safety and effectiveness of the blockbuster anti-flu drug Tamiflu.’

Our proposed new approach involves compiling a complete list of drug trials (published and unpublished) and requesting full clinical study reports. We have published a full protocol of what we are trying to do, prepared by the Acute Respiratory Infections Cochrane Review Group is available This will appear in issue 1, 2011 of the Cochrane Database of Systematic Reviews, due to be released on 19 January 2011.

"We do not yet know whether sound scrutiny is feasible with a journal's resources, but we intend to use the proposed review of neuraminidase inhibitors to attempt to quantify these tasks. We will be keeping a journal of our review with a resource tally, and proposing methods for in-depth scrutiny of a whole trial programme of a new drug. It is time the media, the Cochrane Collaboration, and any reader interested in knowing what they are prescribing or are being prescribed increase the pressure on policy makers. If you swallow a medication, you need to know how it works - for real."

We are therefore proposing a new approach that would allow in-depth scrutiny of the complete set of trial data for a new drug. As a result, Roche (oseltamivir’s manufacturer) publicly pledged to make full results for ten unpublished clinical trials available for scrutiny. Yet, to date, they have failed to fulfil this promise.

We are urging you as researchers, the public, and the media to work together to put pressure on industry to embrace the ethical responsibility to release data in the public interest. We are also calling on medical journals to require submission of the most detailed report available to allow independent scrutiny.

If you’ve got time we’d like you to comment on the protocol – the more feedback the better. Comments and feedback, should be send back to: cdelmar@bond.edu.au; jefferson.tom@gmail.com