Most medical practices are not parachutes

I was invited on EMCrit to discuss my position on idarucizumab and I fear I didn’t do a great job explaining myself. I don’t want to spend more time discussing the specifics of idarucizumab, but I think the larger problem of declaring that a therapy works without study, or declaring that it would be unethical to study a therapy because we “know it works” despite a lack of randomized control trials, is worth pursuing.There was a fantastic paper published this year in CMAJ Open entitled “Most medical practices are not parachutes” that I think applies directly the the idarucizumab debate. (Hayes et al 2018)

The metaphor of the parachute is enormously overused in medical research. Proponents of medical therapies like to conjure the image of a parachute when arguing that their particular therapy should not be subjected to a randomized trial. “We know it works”, they claim, “after all, you would never perform a randomized trial of parachutes. It wouldn’t be ethical.” These claims often harken back to a satirical article by Smith and Pell in the BMJ Christmas edition. (Smith and Pell 2003)

Unfortunately, the metaphor of the parachute is almost never applicable in medicine. Human physiology is complex, and most therapies have multiple unintended consequences that complicate the intended effect. That is not true of parachutes. More importantly, a parachute has a number needed to treat (NNT) that comes very close to 1. Although people have survived falls from planes without parachutes and there are deaths despite a parachute being used, the absolute reduction in mortality is very close to 100%. Among medical therapies, that is an unheard of benefit. In fact, in a study of over 80,000 reviews in the Cochrane database, there was only 1 therapy that reliably had a reduction in morality as large as 33%! (Pereira 2012) Empirically speaking, we just don’t have parachutes in modern medicine.

That brings us to the study by Hayes. They used Google Scholar to identify all articles that cite the parachutes paper by Smith and Pell, and then selected articles that used this citation to justify an argument that a medical therapy could not be practically or ethically studied in an RCT. They then searched for any RCTs that studied that medical therapy.

They found 35 different medical practices that had been compared to parachutes in order to justify the position that they should not be studied in an RCT. Of those, RCTs had been performed, were underway, or were planned in 22 (63%) cases. In 6 cases the RCTs supported the practice, but in 5 the RCTs demonstrated the practice was not beneficial and in another 5 cases the results were mixed. Remember, these are all medical practices that experts thought were “parachutes” – too beneficial to be subjected to RCTs – and there were as many negative results as there were positives. The practices with positives results were impressive, with NNTs between 3 and 9, but that is still a long way from the NNT of 1 of a parachute.

I think this helps explain my concerns about idarucizumab. Although the authors don’t use the word parachute, they are making the same claim. They state that (despite the lack of proper RCTs) they are so sure this new, experimental medication works that it would be unethical to study it in an RCT. (Pollack 2017) However, unlike a parachute, the physiology of the clotting cascade in critically ill bleeding patients is complex, and we should expect side effects of idarucizumab to accompany the intended effect. Furthermore, we know that unlike parachutes, the mortality reduction (if there is any) will be nowhere close to 100%. Not all patients bleeding on dabigatran die and the mortality in patients given idarucizumab was still 13% in this study. The NNT will be a long way from 1, and there will also be a number needed to harm that has to be considered. In that context, I think it is pretty clear that a RCT of idarucizumab is not just ethical, but absolutely necessary.

References

Hayes MJ, Kaestner V, Mailankody S, Prasad V. Most medical practices are not parachutes: a citation analysis of practices felt by biomedical authors to be analogous to parachutes. CMAJ open. 2018; 6(1):E31-E38. PMID: 29343497 [free full text]

Pereira TV, Horwitz RI, Ioannidis JP. Empirical evaluation of very large treatment effects of medical interventions. JAMA. 2012; 308(16):1676-84. PMID: 23093165

Pollack CV, Reilly PA, van Ryn J. Idarucizumab for Dabigatran Reversal – Full Cohort Analysis. The New England journal of medicine. 2017; 377(5):431-441. PMID: 28693366

Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ (Clinical research ed.). 2003; 327(7429):1459-61. PMID: 14684649 [free full text]