Understanding clinical risk scores in 4 days. Day 4: Treatment
Clinical trials tell us about the effects of certain drugs in certain patients in certain clinical situations. Arguably the most practical use of risk scores is to help in deciding which patients are most at risk of disease or have disease requiring most urgent treatment. In other words, risk scores can help answer the question, “Who needs treatment?”
An example we have used before is the CHADS2 score
for risk of stroke in patients with atrial fibrillation. Patients with a CHADS2 score of 2 or more should get warfarin therapy. Risk scores used in this way are easy to incorporate into treatment guidelines and are therefore translated into clinical practice fairly well.
However, we have already seen that risk scores are underused in clinical practice. In cardiovascular disease, to some extent, there are too many risk scores. These scores differ in terms of population, predictors and outcomes, which may not fit with those used by clinicians. Ideally, risk scores should be formulated, tried and tested in populations which have not received any treatment, but often patients in these populations have been treated, partially or fully and this is poorly accounted for by most rules.
Refining and re-defining of risk scores can lead to profound impact on who gets treatment, which can lead to a bigger impact on healthcare resources. For example, the CHADS2 system may not fully exclude risk of stroke in patients with a CHADS2 score of <2. Therefore, the CHA(2)DS(2)-VASc score was formulated to better define the patients with atrial fibrillation who truly are at low risk of stroke and therefore do not need warfarin. However, use of the CHA(2)DS(2)-VASc system does mean that more patients will go on warfarin treatment than with the CHADS2 system.
Another example is the impact of adding social deprivation to cardiovascular risk scores. One such score which incorporates the social gradient in cardiovascular disease is the ASSIGN score. Patients with an ASSIGN risk of 20% or more receive statin treatment. One study calculated that if the ASSIGN risk score was implemented, where a person lives would affect the decision to initiate statin treatment in the case of 15.7% of the population (aged 30-74 years), corresponding to 4.6 million in the UK.
The upshot is that risk scores are very versatile tools which can be used to guide diagnosis, prediction, prognosis and treatment in clinical practice, if the populations in which the scores were formulated and tested match your patient population, and you recognise their limitations.
Understanding clinical risk scores in 4 days. Day 3: Prognosis
Closely related and interlinked to prediction is the idea of prognosis. Literally meaning “foreseeing” in Greek, prognosis attempts to quantify the likely outcome over time for a particular patient in a particular clinical setting. For example, in terms of risk prediction, a CHADS2 score of 2 or more in a patient with AF tells us that he or she is at a large enough risk of stroke to warrant treatment with warfarin to thin the blood. 18.2% of patients with a maximum of CHADS2 score of 6 will have had a stroke at one year: this is prognostic information. Therefore, the same score can be used as both a predictive and a prognostic tool.
One of the most enduring and widely used prognostic scores in clinical practice, is the Apgar score, used to predict outcomes in newborns by a score of 1 to 10. As noted in Apgar’s seminal paper, scores of 0-3 indicate poor prognosis for newborn babies.
The prognostic information doctors and patients are most commonly interested in is survival, especially in the case of cancer. The challenge is individualising risk from epidemiologic data to a particular patient.
As in the previous 2 blogs about diagnosis and prognosis, scores are more relevant and more likely to be used in clinical practice when they are tied in a decision-making strategy, i.e. there is no point in prognostic information (other than telling the patient what their outlook is likely to be), unless the management of the patient is altered. This is nicely summarised in this 10-year old JAMA editorial:
“Medicine is an action-oriented profession in which clinicians want to relieve suffering, rather than just watch its course. Regrettably, most prognostic indices are not accompanied by decision thresholds that convert level of risk into degree of action.”
Understanding clinical risk scores in 4 days. Day 2: Prediction
As well as diagnosing disease, it may be important to predict which patients need more investigation or more treatment. The term, “clinical prediction rule”, is often used to describe the best combination of medical signs, symptoms, and other findings in predicting the probability of a specific disease or outcome. Therefore, they can be used in diagnosis as in the last article, or they can estimate the risk of a specific outcome, and therefore help in decision-making.
To form prediction rules, researchers look at a group of patients suspected of having a specific disease or outcome. They then compare the value of clinical findings available to the clinician versus the results of more intensive testing or the results of delayed clinical follow up.
Previous studies have shown that giving clinicians information in the form of risk scores in isolation may not influence clinical practice. GPs routinely underuse risk scores and have suggested that computerized risk prediction for multiple diseases simultaneously and the integration of lifestyle recommendations may improve their uptake.
Clinical prediction rules are most effective when implemented as part of a pathway within hospital policies and guidelines, as shown in the case of pneumonia and the more rigorously the rules are implemented, the more successful they are at improving outcomes.
The CHADS(2) score is an example of a clinical risk score to help in deciding whether to start warfarin in patients with atrial fibrillation to prevent stroke. Patients are classified by their clinical history, getting 1 point each for Congestive cardiac failure, Hypertension, Age>75 years, Diabetes and 2 points for Stroke. Patients with scores of 2 or more should get warfarin, as long as they have no contraindications. A recent improvement on this score, the CHA(2)DS(2)-VASc has added 3 additional factors (history of vascular disase, sex and age 65-74) to improve the prediction of risk so that only people who are really at negligible risk of stroke do not get warfarin.
Other examples of risk scores used in this way are Wells scores for deep vein thrombosis and pulmonary embolism and CURB-65 in pneumonia.
Understanding clinical risk scores in 4 days. Day 1: Diagnosis
Risk is arguably one of the most central concepts in clinical medicine. Patients want to know their risk of a particular disease or perhaps even death. Clinicians are interested in which patients are at highest risk of disease and complications. Researchers want to establish the way in which new treatments change the risk of disease. Clinical risk scores use available information to predict the risk of a particular clinical scenario. Risk scores cannot replace clinical judgment but they can guide it. There are hundreds of risk scores available to clinicians and many are available online as risk calculators and are widely used by patients. In this 4-day series, I will cover the 4 types of clinical risk score.
LESSON 1: DIAGNOSIS: IS IT DISEASE?
Some diseases are easy to diagnose because they are well-defined. For example, infection with HIV is diagnosed by a positive HIV test. However, not all diseases are so easy to diagnose or have such clearly defined criteria. Lack of diagnostic certainty can lead to inappropriate treatment and can even lead to abuse of the social benefits system by some unscrupulous parents in the case of attention deficit hyperactivity disorder(ADHD)
Diagnostic scores can be helpful in identifying disease, but also in describing severity of disease. In depression, the Beck’s depression inventory (BDI) is a score which describes the severity of depression on a continuous scale, with 0–9 indicating minimal depression, 10–18: mild depression, 19–29: moderate depression and 30–63 indicating severe depression. A score of at least 10 on the Beck Depression Inventory is the generally accepted threshold for the indication of possible depression.
The mini-mental test score (MMTS) is the most commonly used diagnostic test to assess cognitive function. Scores of 25-30 out of 30 are considered normal. NICE classify 21-24 as mild, 10-20 as moderate and <10 as severe impairment.
Diagnostic scores must be tested and validated in well-defined populations because the scores may perform differently in different patient groups. For example, the BDI above is good at diagnosing depression in post-stroke patients.
The term “clinical prediction rules” is often used to describe the use of clinical findings (history, physical examination, and test results) to make a diagnosis, OR to predict an outcome. This article from Annals in Internal Medicine sums up what you need to know about diagnostic scores before you use them on your patients.
Everyone has an interest in exaggeration
Think about health care communication to the public. Is it done well, does it lead to informed choice? Do you feel skeptical about the issue, and disillusioned with the current status of communication of health to a wider audience.
Today, at Kellogg College, at the University of Oxford a workshop on ‘Enhancing the Public Understanding of Health Research’ aimed to bring together folk with experience in developing and evaluating materials to help people become better users of health research. The question is, why isn’t the public informed? The two issues when we are faced with research findings, whether it be on TV, in the newspapers or in a scientific journal, are ‘Why should I believe these results,” and “What do they mean?”
Steve Woolshin and Lisa Schwarz authors of the ‘Know Your Chances’ book, talked enthusiastically, and with an array of examples, of the bigger picture and what is out there.
The issue that caught my eye was how invested we all are in exaggeration: manufacturers to sell products, academics to get their research published, journals to get their research cited and picked up by the news and for media to gain more advertising revenue. And so the cycle goes on.
For instance, Dannon's Activia, exaggerated health claims led to a US $21M fine, a further example is a study which suggests, ‘nutrient enhanced water drinks are ''''expensive lolly waters'''' with exaggerated health claims’. All sorts of exaggerations occur on a daily basis ‘Radiation health fears exaggerated, claims Oxford professor’. Just try googling ‘exaggerated health claims’.
My favourite is vitamin D which has reached rock-star status in recent years, as a potential cure for the prevention of everything.
The BBC even highlights medical journals are all part of the exaggeration phenomena: ‘Medical journals have been accused of hyping up the findings of the research that they publish.’
Am I prone to exaggeration, you bet I am! Obviously this is the best site worldwide to find the opinion on the evidence to trust.
Top Ten Clinical Epidemiology studies
I’ve created a list of what I think are the top ten the most influential epidemiological studies – I’m looking for feedback to see what folk think.
Therefore, if you believe there should be something different on the list then can you post a comment. Rules are it has to be a clinical study that involves people, which excludes basic sciences. In addition the study has to have subsequently influenced the field significantly.
Doll Richard, Bradford Hilly A (June 26, 1954). "The mortality of doctors in relation to their smoking habits. A preliminary report". British Medical Journal 1 (4877): 1451–55. PMID 13160495
Antiplatelet Trialists' Collaboration. Collaborative overview of randomised trials of antiplatelet therapy--I: Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients BMJ. 1994 Jan 8;308(6921):81-106. PMID 8298418
Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med. 1989;321:406-412. PMID 2473403
MRC Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment of pulmonary tuberculosis. BMJ. 1948;ii:769–783. James Lind summary
Thomas Francis, Robert Korn, et al. "An Evaluation of the the 1954 Poliomyelitis Vaccine Trials." American Journal of Public Health 45 (1955), 50 page supplement with a 63 page appendix. The salk vaccine trials
Randomized trial of cholesterol lowering in 4444 patients with coronary heart disease: The Scandinavian Simvastatin Survival Study Lancet 1994; 344 1383-1389 PMID 7968073
1998 (now retracted)
The Editors Of The Lancet (February 2010). "Retraction--Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children". Lancet 375 (9713): 445. doi:10.1016/S0140-6736(10)60175-4. PMID 20137807.
Bonadonna G, Brusamolino E, Valagussa P, Veronesi U. Adjuvant study with combination chemotherapy in operable breast cancer. Proc Am Assoc Cancer Res Am Soc Clin Oncol 1975;16:254-and N Engl J Med. 1976 Feb 19;294(8):405-10. PMID 1246307
Thomas R. Dawber, M.D., Gilcin F. Meadors, M.D., M.P.H., and Felix E. Moore, Jr., National Heart Institute, National Institutes of Health, Public Health Service, Federal Security Agensy, Washington, D. C., Epidemiological Approaches to Heart Disease: The Framingham Study Presented at a Joint Session of the Epidemiology, Health Officers, Medical Care, and Statistics Sections of the American Public Health Association, at the Seventy-eighth Annual Meeting in St. Louis, Mo., November 3, 1950.
Framingham Heart Study
On the Mode of Communication of Cholera by John Snow, M.D.
London: John Churchill, New Burlington Street, England, 1855 On the Mode of communication of cholera
Look forward to seeing what you think
you can post it to me on twitter
Are conferences any good at disseminating evidence?
The questions of how best to train and educate doctors during medical school and throughout their careers have been difficult to answer for hundreds of years. I was reflecting on this over the last couple of weeks as I attended two overseas conferences. Like most conferences, these meetings aim to bring doctors and scientists up-to-date with the latest research developments in their fields of interest within medicine and the health sciences. They both had top-level speakers and excellent programmes, and both had substantial e-learning and other online resources. Lectures and conferences are the mainstay of teaching and continuing medical education in medicine and many other disciplines, but are they good at what they set out to do? This is a question particularly pertinent to evidence-based medicine, because what it ultimately aims to do is disseminate the best evidence and enable its uptake in clinical practice.
We know that people can learn over the internet using e-modules, video-conferencing and other modalities equally well, but I think there are two reasons why conferences persist. First and foremost, in a world of compulsory continuing medical education, they offer convenience. They are the easiest (and most passive way) to disseminate evidence from research. Secondly, they offer the chance to network (and relax) with peers and opinion leaders which even market leaders like TED.com will find it difficult to emulate.
As educational budgets are cut throughout the NHS and doctors increasingly fund their own continuing medical education, the reality is that pharmaceutical or other industry-sponsored educational events are likely to grow and not decrease, and more safeguards will be needed to avoid a reduction in educational content and an increase in drug company promotional material. As postgraduate deaneries are threatened, there are fears for the standards of training of doctors, but we should be equally worried about who is going to pay for the teaching in the new NHS. The Medical Students International Network (MedSIN) is remarkably forward-thinking about trying to avoid private industry sponsorship of education in medical schools. However, in a difficult economic climate, even with increasing tuition fees, industry-sponsored undergraduate education may rear its head.
There is a role for conferences in the dissemination of research and in promoting evidence-based practice and knowledge translation, but they should not be the only way we keep up-to-date. What is the best way to learn? As some UK public health trainees, say in a recent article: “In order to retain its position as a leader in the field of public health, the UK needs to adapt its training programme to better reflect today's challenges.”