Morgenstern, J. We should stop using the Canadian CT head rule, First10EM, February 6, 2023. Available at:
https://doi.org/10.51684/FIRS.129256
In order for a clinical decision rule to be valuable it must change clinical practice, and that change must improve patient care. When assessing decision rules we often get hung up on sensitivity and specificity, but sensitivity and specificity can be misleading. Many diagnostic tests look great on paper but fail to materialize patient benefit. With a high sensitivity and low specificity, many decision rules are analogous to the D-dimer. We readily accept new rules as they are created, but do you really want more D-dimer in your life? Instead of focusing on test characteristics, we really need to focus on the clinical impact of our rules. This is why well designed implementation or impact studies are considered to reference standard in assessing the value of clinical decision rules. (Finnerty 2015) However, if that is the case, why are so many people still using the Canadian CT Head rule more than a decade after the publication of an implementation study demonstrating no benefit, and possible harm?
The paper
Stiell IG, Clement CM, Grimshaw JM, Brison RJ, Rowe BH, Lee JS, Shah A, Brehaut J, Holroyd BR, Schull MJ, McKnight RD, Eisenhauer MA, Dreyer J, Letovsky E, Rutledge T, Macphail I, Ross S, Perry JJ, Ip U, Lesiuk H, Bennett C, Wells GA. A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments. CMAJ. 2010 Oct 5;182(14):1527-32. doi: 10.1503/cmaj.091974. Epub 2010 Aug 23. PMID: 20732978
The Methods
This is a cluster-design, randomized, matched-pair controlled trial conducted in 12 emergency departments in Canada.
Patients
Stable adult patients presenting to the emergency department after minor head injury with all of the following: “blunt trauma to the head resulting in witnessed loss of consciousness, amnesia or witnessed disorientation, an initial Glasgow Coma Scale score of 13 or greater, and occurrence of the injury within the previous 24 hours.”
Intervention
The Canadian CT head rule was implemented, both via education, but also via a mandatory tool completed at the time a CT was ordered.
Comparison
No specific intervention was introduced to alter the CT scan ordering behavior of the physicians. (This group was unaware of the study.)
Outcome
The primary outcome was the proportion of eligible patients referred for a CT head.
The Results
They include a total of 4531 patients who look evenly matched across the groups.
CT usage increased from 62.8% to 76.2% after implementation of the Canadian CT head rule (absolute increase 13.3%, 95% CI 9.7-17.0%). CT usage also increased in the control sites, from 67.5% to 74.1% (absolute increase 6.7%, 95% CI 2.5-10.8%). The difference between these two increases was not statistically significant (p=0.16).
There were no missed clinically important brain injuries in either group throughout the study period.
My thoughts
This study is clearly negative. I think there are multiple biases in favour of the decision rule, and so I believe that the point estimates that suggest that the rule is harmful have a reasonable chance of being true, despite being statistically insignificant.
This trial represents the best case scenario for the Canadian CT head rule. Participants knew they were part of a study, and so were more likely to be careful in their application of the rule than in real life. There were education sessions, pocket cards, and posters. Many of these departments were involved in the original derivation and validation of the rule, and so you would expect a degree of familiarity and attachment to it. Most importantly, there was a mandatory, real-time reminder of the rule when a requisition was made for a CT. It is very unlikely that real world implementation of the rule looks anything like the idealized version in this study.
The impacts of this rule are far worse in real life, because almost no one adheres to the inclusion criteria. The Canadian CT head rule only applies to patients with blunt trauma and a witnessed loss of consciousness, definite amnesia, or witnessed disorientation. Those were the inclusion criteria, both of the original study and this cluster RCT. (Stiell 2001) In the minds of the researchers, no one would be crazy enough to even consider a CT scan in a patient who hadn’t lost consciousness, had amnesia, or had clear disorientation. No one in 2001 would consider scanning such trivial injuries. Sadly, these researchers were not able to predict the sad decline of clinical medicine. In 2023, we have forgotten their inclusion criteria altogether. In fact, we would probably purposefully ignore their criteria, refusing to apply the decision rule to a patient with significant amnesia or disorientation, because we think those patients are ‘way too high risk for a decision rule’. Instead, we apply the rule to every single patient who falls, whether they hit their head or not, which means that every patient over the age of 65 gets a CT scan of their head, because they failed a rule that never even applied to them. Therefore, the impacts of the Canadian CT head rule in real life are much much worse than they seem in this study.
The impact of the CT head rule was probably mitigated in this study because it is highly likely that at least some physicians at the control sites were familiar with the rule and used it at least some of the time. That biases the trial towards showing no effect. Usually, you would worry that the bias might potentially negate some benefit. However, considering that the point estimate suggests harm, and this bias would make the groups look more similar, it seems likely that the rule would look even more harmful if you could eliminate this source of bias.
Most importantly, their data makes it clear that the CT head rule is completely unnecessary. It is a tool designed to be 100% sensitive, which would be valuable if we were missing important injuries. However, despite having more than 4000 patients in the study, there was not a single miss in either group. Even without the CT head rule, physician judgment is more than enough to catch ever single important injury.
The Canadian CT head rule doesn’t decrease imaging and doesn’t improve our rate of diagnosis, even in idealized research settings. In the real world, this rule has almost certainly caused tremendous harm by driving increased CT rates over the last decade. No one should be using this rule in clinical practice.
Bottom line
This has been true for over a decade, but nobody should be using the Canadian CT head rule. It is one of the few decision rules that we have implementation research for, and this cluster RCT demonstrates no benefit, and I believe suggests harm, despite being the best possible scenario for the Canadian CT head rule.
Other FOAMed
Clinical decision rules are ruining medicine
References
Finnerty NM, Rodriguez RM, Carpenter CR, Sun BC, Theyyunni N, Ohle R, Dodd KW, Schoenfeld EM, Elm KD, Kline JA, Holmes JF, Kuppermann N. Clinical Decision Rules for Diagnostic Imaging in the Emergency Department: A Research Agenda. Acad Emerg Med. 2015 Dec;22(12):1406-16. doi: 10.1111/acem.12828. Epub 2015 Nov 14. PMID: 26567885
Stiell IG, Clement CM, Grimshaw JM, Brison RJ, Rowe BH, Lee JS, Shah A, Brehaut J, Holroyd BR, Schull MJ, McKnight RD, Eisenhauer MA, Dreyer J, Letovsky E, Rutledge T, Macphail I, Ross S, Perry JJ, Ip U, Lesiuk H, Bennett C, Wells GA. A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments. CMAJ. 2010 Oct 5;182(14):1527-32. doi: 10.1503/cmaj.091974. Epub 2010 Aug 23. PMID: 20732978
6 thoughts on “We should stop using the Canadian CT head rule”
I agree wholeheartedly. There are still some older ER docs around who use their clinical judgment when deciding whether or not to order a CT, but the vast majority who graduated within the last 10 years or less just want an algorithm that they can follow without thinking about whether or not it really applies to their patient. Inappropriate use of imaging is one of the big reasons why waitlist for CTs and MRs are growing ever longer.
I don’t know if this was stated in any of the original literature or if this was just the way I was taught, but I had always assumed/been taught that this rule is unidirectional.
IE, after you make a clinical decision to scan the patient you apply this rule to see whether major injury can be ruled out without CT. But you never scan, or ‘rule in’, because of this rule.
I guess the idea of not scanning when you believe it to be clinically indicated, just because a decision rule says not to, is also questionable. But given what you said above about the decline of clinical medicine and the increased use of scans I wonder if it could have utility in this way for some.
Thanks for the comment
I think one of the major reasons that we need implementation studies, is because ‘one way rules’ have such a high probability of being misused. It is easy to predict that most/many doctors will use them as two way rules instead of one way. There is definitely some advantage of breaking this research into 2 categories: perfect use and real world use. It the rule helps in perfect use, but not in real world use, then you know the rule itself isn’t the problem, and a thoughtful practitioner like yourself could focus on applying the rule perfectly with confidence that it was helping you. However, for this rule (like almost all rules) we don’t even have evidence of patient benefit with perfect use. (And this study is likely closer to perfect use than in the real world, as all physicians were required to fill out the rule whenever they ordered a scan.)
I agree. Resident here, I usually find myself bringing up the rule as a reason not to scan someone. I don’t think that saying “a study showed more CT utilization in hospitals that adopted it versus usual care” challenges this in any way, if our aim is to order less scans so as to safely decrease ED LOS, decrease exposure to radiation and decrease cost of care.
The CDC/ACEP 2008 policy recommends the New Orleans Criteria (NOC) for mild TBI and the Canadian CT Head Rule (CCHR) for minimal TBI. Mild TBI being the classic definition of TBI with LOC or transient confusion/amnesia, and minimal TBI being any head bump that did not result in neuronal dysfunction. https://www.cdc.gov/traumaticbraininjury/mtbi_guideline.html
ACEP has *just* reviewed their policy and they now endorse the CCHR as preferred over NOC regardless. They’re equally sensitive for clinically significant brain injury, but CCHR is more methodical, while NOC is more straightforward but less specific. https://www.acep.org/patient-care/clinical-policies/mild-traumatic-brain-injury2/
I also found this document as recommended DC instructions/talking points for discharging a patient you didn’t scan https://www.acep.org/patient-care/smart-phrases/ct-scans-for-minor-head-injuries/