Embracing Science: #dasSMACC Hardcore EM handout

How are doctors different from naturopaths? This sounds like it should be the start of a cheesy joke, but sorry to disappoint you, I’m not funny. So seriously, how are doctors different from naturopaths?

We seem to live in the era of science denial. Pick any topic: climate change, vaccines, people who truly still believe that they earth is flat (yes they exist). In this climate, it’s pretty easy to demonize alternative practitioners. But I know a lot of alternative practitioners, and their motives are not that different from mine.

We all go to work every day hoping to help people. So do naturopaths. We suggest treatments that we believe will make people healthier. So do naturopaths. We care about our patients. So do naturopaths

So what really sets us apart? What is medicine? The one thing that sets us apart is science.

There is a reason for the traditional lab coat. Medicine 200 years ago, with its leeches and bad humors, isn’t easily distinguished from any of the alternative practices today. But we put aside our snake oils and we turned to science.

But how do our patients see medicine? Browse through any newspaper or website and the messages quickly become very confusing. Beer causes cancer. No wait, beer prevents cancer. Birth control causes cancer. No, birth control prevents cancer. Sex causes cancer. But we should ignore those articles and focus on the ones that tell us that sex prevents cancer. The legend, Dr. John Ioannidis published a great paper that discusses how everything we eat seems to both cause and prevent cancer.

Is it any wonder that so many people are confused? That it is so easy to convince people that vaccines are killing them? This is why EBM is so important. This is why we all have to understand the core concepts of EBM. Science is the foundation of our profession. We have to be able to talk about science with our patients if we are going to have any hope of maintaining trust in the face of this onslaught of pseudo-medical nonsense in the media.

So for our talk, Rory and I did a brief “year in review” literature update, but in order to ensure that we are all more comfortable reading and discussing papers, we pulled one key EBM concept from each paper. What follows is a very brief summary of those learning points.

Paper #1: Apple juice for pediatric gastroenteritis.

Freedman SB, Willan AR, Boutis K, Schuh S. Effect of Dilute Apple Juice and Preferred Fluids vs Electrolyte Maintenance Solution on Treatment Failure Among Children With Mild Gastroenteritis: A Randomized Clinical Trial. JAMA. 2016; 315(18):1966-74. PMID: 27131100

Rapid clinical summary

A single center randomized controlled trial comparing electrolyte solutions to a fluid strategy that used half strength apple juice in the emergency department and the child’s preferred fluid at home. They showed that the apple juice strategy was superior, with a number needed to treat of 12 to prevent 1 treatment failure.

Key EBM point

When reading a paper, it is always important to ask, “do the patients in this trial look like my patients?” (This is about the trial’s external validity.) In the apple juice study, there is one major caveat: the kids all had very minor dehydration. In fact, they were using the apple juice as a maintenance fluid, not for rehydration. So when applying the results, you have to ask: do those children look like the child in front of me? If you are looking after a sicker child – who is more dehydrated – these results wouldn’t necessarily apply. We don’t have data, but you might want to continue using electrolyte solutions for those kids; for rehydration.

Paper #2: Does size matter with ketorolac dosing?

Motov S, Yasavolian M, Likourezos A. Comparison of Intravenous Ketorolac at Three Single-Dose Regimens for Treating Acute Pain in the Emergency Department: A Randomized Controlled Trial. Annals of emergency medicine. 2016. PMID: 27993418

Rapid clinical summary

This is a well done RCT comparing 10mg to 15mg to 30 mg of ketorolac given IV for patients with pain in the emergency department. And the outcomes were exactly the same. You don’t get more pain relief with a higher dose.

Key EBM point

RCTs under-estimate harm. There are a number of reasons for this. Trials generally aren’t powered for rare harm events. Ketorolac is relatively safe, but it is known to have a number of rare side effects, such as Stevens Johsnson syndrome, and those just aren’t going to show up in a 240 person trial. And there are some other reasons trials miss harms. They often pick healthier patients, or they use a run-in period to exclude patients with side effects, or they just may not look (it’s expensive and difficult to follow everyone for a long time).

So although there were no harms with the higher doses in this trial, we can assume that there would be in clinical practice, and that is why the only dose of ketorolac that I use is 10mg.

Paper #3: PE in syncope

Prandoni P, Lensing AW, Prins MH. Prevalence of Pulmonary Embolism among Patients Hospitalized for Syncope. The New England journal of medicine. 2016; 375(16):1524-1531. PMID: 27797317

Rapid clinical summary

Everyone has heard about PESIT by now. It was an observational trial from Italy that indicated that 17% of patients admitted to hospital for first time syncope were diagnosed with PE. There are a number of reasons why that 17% number is not applicable to our patients, which I summarize here.

Key EBM point

When assessing any diagnostic tool, especially one that is being used for screening, we have to be wary of overdiagnosis. Overdiagnosis is not equivalent to false positives. It is not a problem with our diagnostic tests. Overdiagnosis refers to true anatomic pathology, but pathology that would have never impacted the patient. Perhaps the most common examples come from cancer: finding a very slow growing prostate cancer that the patient will never notice. In terms of the PESIT trial, it is likely that some of the PEs that were identified by screening sick, elderly patients, although actually present in the pulmonary vasculature, were not the cause of the patient’s syncope and would have never resulted in an adverse event.

Paper #4: qSOFA for sepsis

Freund Y, Lemachatti N, Krastinova E. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. JAMA. 2017; 317(3):301-308. PMID: 28114554

Rapid clinical summary

This is the first prospective validation of qSOFA. It’s performance was very similar to the retrospective derivations. The authors did all the right things: this was a prospective trial, enrolling consecutive patients presenting to the ED in whom the treating clinician was concerned for an infection. Overall, qSOFA was more accurate than SIRS, but less sensitive.

Key EBM point

There is a subtle but important difference between diagnostic and prognostic tools. Neither SIRS nor qSOFA diagnosis sepsis. They are both tools that give us a prognosis; that we are using to identify sicker patients that will require agressive intervention. From that perspective, it is easier to understand the role of qSOFA. A SIRS score of less than 2 identifies a group at low risk of mortality (2%), whereas a score greater than or equal to 2 identifies a higher risk group (11%). The qSOFA score less than 2 identifies an equally low risk group (3%), but the qSOFA greater than or equal to 2 identifies a much higher risk group (24%). At the same time, qSOFA is more specific.

I am still hesitant to change all the definitions we use clinically, because I worry about spending large sums of money to update triage systems without clear evidence that the new definitions will help our patients. However, qSOFA is a great tool because it is entirely clinical. In fact, the real lesson from the studies of qSOFA is probably that sepsis is a clincally obvious disease. You don’t need special tools to identify most septic patients, because they will have clinical features that make them obvious to the treating physician.

Paper #5 Are high prescribers to blame for the opioid epidemic?

Barnett ML, Olenski AR, Jena AB. Opioid-Prescribing Patterns of Emergency Physicians and Risk of Long-Term Use. The New England journal of medicine. 2017; 376(7):663-673. PMID: 28199807

Rapid clinical summary

This paper is a massive data dredge, which means it starts off with some major limitations. They retrospectively looked at prescriptions filled for opioids and using that data divided doctors up into high prescribers and low prescribers. They then looked at patients who saw either a high or a low prescriber and asked: did that patient use opioids for more than 6 of the next 12 months. But let me cut right to the fatal flaw of this study. They say that they were comparing high and low prescribing doctors, but they weren’t. There was no way for them to connect an individual doctor to a prescription here. All they were looking at was how likely was a patient to fill a prescription in the week after their ED visit, but there are lots of ways one of my patients might fill a script even if I didn’t give them one. So the real conclusion based on this data is that patients who go out and get themselves a prescription for opioids in week one are more likely to be using them a year later. Not exactly groundbreaking or really interesting.

Key EBM point

“Significant” has different definitions in evidence based medicine. Sometimes we mean statistically significant. Sometimes we mean clinically significant. If you look at table 1 from this paper (below), which compares the baseline demographics of the groups in this study, almost every comparison has a p value less than 0.05. In other words, they are statistically significant. But if you look at the actually numbers – age 68.8 vs 68.2, number of comorbid conditions 3.6 vs 3.5 – the differences are so small that they are meaningless. They aren’t clinically significant.

Personally, I wonder if the same can be said about the primary outcome of the study. The group of patients they say saw one of the highest intensity prescribers had a 1.5% chance of long term opioid use. If you saw one of low lowest intensity prescribers you had a 1.1% chance of of long term opioid use. A difference of 0.4%. And that is just continued opioid use at 1 year, not harm. The actual harms were 7 vs 10 in 10,000. Those numbers might be statistically significant, but are they actually clinically significant?

Paper #6 TIME IS SEPSIS

Seymour CW, Gesten F, Prescott HC. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. The New England journal of medicine. 2017; 376(23):2235-2244. PMID: 28528569

Rapid clinical summary

Using the New York State Department of Health sepsis registry database, the authors examined patients who were treated according to a pre-specified 3-hour treatment bundle within 6 hours of presentation to the Emergency Department. While these bundles varied from hospital to hospital, all participating centers were required to include blood cultures, broad spectrum antibiotics, measurement of the serum lactate level, and the administration of 30 cc/kg. Their overall conclusion is that there was an increase in the odds of death by 1.04 for every hour the bundle was delayed. Digging into the data, this association was only true for patients with septic shock, not for all comers.

Key EBM point

Whenever assessing interventions, one must always ask: at what cost? If you believe these numbers, you would have to delay ABx for over one hour in 148 patients before one additional patient dies due to your tardiness. How many patients presenting to the ED have a suspected infection? How many of these are septic? How many of these are in septic shock? Even if this benefit exists, what will it cost our system to systematically screen all patients with a suspected infection to identify the few patients who will benefit?

We don’t actually have an answer to this bigger question, but perhaps the most important takeaway from this paper is that, if time matters in sepsis, it probably only matters in the very sick septic shock patients. These patients will be clinically obvious.

Paper #7 Contrast induced nephropathy

Hinson JS, Ehmann MR, Fine DM. Risk of Acute Kidney Injury After Intravenous Contrast Media Administration. Annals of emergency medicine. 2017; 69(5):577-586.e4. PMID: 28131489

Rapid clinical summary

This is a large retrospective analysis done a single US emergency department. They included almost 18,000 patients. To get in you had to have a creatinine measured in the emergency department and then a couple days later. They compared the rate of acute kidney injury between the patients who got contrast, those who had a non-contrast CT, and another group who weren’t imaged at all. Bottom line: there was no difference at all between the groups. Contrast did not result in kidney damage.

Key EBM point

Whenever assessing the outcome of a trial, it is very important to ask: does that outcome matter to the patients. Frequently, we measure numbers or biochemical levels – things that the patient doesn’t notice. In this trial, the primary outcome was acute kidney injury, as defined by creatinine level, but a transient bump in the creatinine is something that the patient won’t notice. We call these outcomes disease oriented. What we really should be focusing on are outcomes that impact the quality or length of a patient’s life. We call these patient oriented outcomes. This distinction is important when dissecting the contrast induced nephropathy literature, because many of the prior trials demonstrating an association between contrast and kidney injury only showed a transient increase in the creatinine value without demonstrating any change in patient oriented outcomes.

Paper #8 The WOMAN trial

WOMAN trial collaborators . Effect of early tranexamic acid administration on mortality, hysterectomy, and other morbidities in women with post-partum haemorrhage (WOMAN): an international, randomised, double-blind, placebo-controlled trial. Lancet (London, England). 2017; PMID: 28456509

Rapid clinical summary

This RCT included 20,000 women diagnosed clinically with postpartum hemorrhage and randomized them to tranexamic acid or placebo. The very quick summary is that there was no change in the primary outcome of all cause mortality or hysterectomy.

Key EBM point #1

Be wary of disease specific mortality. In this study, there was no change in all cause mortality. However, they do report a statistically significant decrease in death due to bleeding (by 0.4%). But what does this actually mean? Your first question when faced with disease specific mortality should be: how did they determine cause of death. They didn’t do autopsies in this study, so there is a reasonable chance this reported difference isn’t real. However, even if it is real, when there is no change in all cause morality, any decrease in deaths from one cause must be balanced out by an increase in deaths from some other cause. There is no net benefit.

Key EBM point #2

EBM is complex, but important. This is a very large, well done trial with a negative primary outcome. However, tranexamic acid is a widely used and seemingly safe intervention. What should we do now when faced with a critically ill patient with post partum hemorrhage? The answer is: its complicated. There isn’t a clear answer. It is possible that there is still a subset of patients that could benefit. On the other hand, we don’t want to focus on treatments that aren’t going to help our patients.

These debates form the core of evidence based medicine. It can be confusing, but remember, science is the true foundation of our profession. This is why EBM can’t be left to just a few nerds; the Ken Milnes or Ryan Radeckis of the world. This is why we all have to be at least a little bit comfortable with these topics. We need to embrace science so we can make the best possible decisions for our patents and so that we can explain those decisions to them. We need to embrace science, and at the same time understand its limitations, in order to maintain the tremendous trust that our patients have placed in our hands.

---

Related articles

See this article for a discussion of why evidence based medicine is the best kind of medicine.

If you find critical appraisal daunting, check out the post “Evidence Based Medicine is Easy”.

---