Thrombolytics for acute ischemic stroke – NNT evidence review

Although I am taking a break from posting as part of my paternity leave, there were a number of projects underway before that break started. One that I was most excited about was a collaboration with Ken Milne to write an updated summary of the evidence for thrombolytics for acute ischemic stroke for the amazing theNNT.com website. While I have tackled the topic before, my original post is incredibly long, and I am happy to provide an easier read that helps to explain why this remains one of the most controversial topics in medicine.

The post is now up on theNNT, and can be found here. The editors of that website have graciously allowed me to share the article with you here as well. Our overall conclusion: the available evidence leaves us completely uncertain about whether this is a benefit from thrombolytics that could outweigh the clear harms.

Summary

This is the third NNT summary of thrombolytics for acute ischemic stroke. The first gave thrombolytics, as a class, a ‘red color recommendation: no benefit. The second gave alteplase, a single agent, a ‘green color recommendation: benefit>harm. As no relevant trials were published between the two, both author groups examined essentially the same data and arrived at opposing conclusions. We believe it would be hubris to presume this third summary will arrive at the one true answer. We focus, therefore, on the uncertainty we believe leads to conflicting interpretations.

The systematic review we chose to summarize includes 26 randomized trials of more than 10,000 participants, assessing the benefits of thrombolysis for acute ischemic stroke.¹ The authors report a 3.2% improvement in good neurologic outcome, a 5.4% increase in symptomatic intracranial hemorrhage, and a 2.5% increase in mortality. However, we question the certainty implied by these summary numbers.

There are multiple relevant systematic reviews with varying methods but similar findings. In one widely cited review, Emberson and colleagues reported only on alteplase (a problem we discuss below) and find a 5% improvement in neurologic outcomes, a 5.5% increase in intracranial hemorrhage, and a 1.4% increase in 90 day mortality that was not statistically significant.² The 2014 Cochrane review by Wardlaw and colleagues arrives at similar conclusions with significant improvement in neurologic outcomes, increased intracranial hemorrhage, and increased mortality.³ Thus our conclusions and discussion are unchanged by choice of review, and reflect our belief that pooling data on this topic is overly simplistic and masks profound uncertainty.

Caveats

One of the great conceptual difficulties of summary statistics like the number-needed-to-treat (NNT) is the implication of certainty. A major strength is its simplicity, making complex research easier to understand. A weakness, however, is also simplicity, hiding the complexity of research, ignoring confidence intervals, and obscuring biases. For most topics, these details are more important than any individual number.

Sources of uncertainty:

Conflicting individual trial results: A major source of uncertainty is the differing results of trials. Among 26 trials in this systematic review, 24 research groups found no benefit in their selected primary outcome.¹ Moreover, the two that claim a benefit (NINDS part 2 and ECASS III)^4,5 had baseline imbalances that may explain the difference. In re-analyses adjusting for imbalances in both these trials the benefits disappear.^7,8 However, in some re-analyses of NINDS-2 the benefit is maintained, adding to the uncertainty.^9,10

Clinical heterogeneity of individual trials: The 26 trials are clinically heterogeneous, enrolling stroke patients of differing demographics, treatment times, stroke severities, anatomic territories, and thrombolytic agents. The author of the first NNT summary felt this was too much heterogeneity for appropriate pooling, a position supported by the major differences in conclusions drawn depending on which studies an author group chooses to include.

Selective emphasis on trials claiming benefit: It is circular and erroneous logic to claim efficacy for thrombolytics based on the trial characteristics of the two positive trials. First, there is legitimate debate about whether they were truly positive. Second, selectively highlighting positive results is a form of the “Texas sharpshooter fallacy”.¹¹ For instance, because both NINDS and ECASS III used alteplase, some have suggested alteplase is a superior agent.^4,5 On close inspection this logic falters: few trials have compared thrombolytic agents head to head, nine additional trials of alteplase are negative, and systematic reviews consistently find no heterogeneity of effect between agents.^3,5,6 Moreover, in evaluating drug efficacy, establishing a class effect is generally a prerequisite for debating or comparing individual agents.¹² Therefore, while it may increase complexity, we believe it is a mistake to exclusively examine data from the agent used in the two trials that claimed benefit.

Likewise, while there are theoretical reasons to think early treatment is better, this has not been directly tested and is not strongly supported by data. Neither Donaldson et al. nor the Cochrane review find an interaction between time to treatment and effect.^1,3 IST-3, the largest trial of thrombolytics for stroke, found better outcomes among those treated after 4.5 hours than in patients treated at 3-4.5 hours from onset. Again, we feel it is best to consider this literature as a whole rather than using time windows selected based on outlying (i.e. positive) results. 

Individual trial bias: Bias is a major source of uncertainty in all scientific research. Importantly, using the GRADE tool,¹³ Donaldson et al. rate the risk of bias as “serious” for all outcomes.¹ One notable source is the outcome scales used, for instance the modified Rankin score. This score is known to be subjective with poor inter-rater reliability and questionable validity. When trained neurologists examine the same patients there is substantial variability in Rankin score assignments.^14,15 Compounding the problem, some trials assessed patients by phone or mail, a choice certain to increase variability and imprecision. For example in IST-3, which contributes nearly 40% of subjects in the Donaldson meta-analysis,¹⁶ results were obtained using telephone and mail follow-up, and non-blinded. This subjectivity is important, because removing this trial from the pooled analysis removes any finding of benefit.

Early stoppage and low power: Because larger trials are weighted more heavily in a meta-analysis, early termination (which reduces trial size) can significantly affect results. Five thrombolytic trials were stopped early for harm or futility.^17–21 Together these would have enrolled more than 2,000 additional subjects who, had they been included, may have neutralized or even reversed findings from the two trials claiming benefit, NINDS2 and ECASS III (combined n=1,445).^4,5 Furthermore, while over 10,000 subjects were enrolled in stroke trials, some individual trials for acute myocardial infarction enrolled far more, and in aggregate those trials included more than 60,000.^22,23 The comparatively small number of participants in stroke trials means chance findings like baseline imbalances are both more likely and more influential, furthering uncertainty.

Harms and overlapping outcomes: In contrast to the heterogeneous data on benefits, the data on harms are more certain. Exact numbers vary based on definitions and whether one focuses on fatal, symptomatic, or any hemorrhage, but an increase in intracranial hemorrhage is certain. More importantly, there is also an increase in mortality with thrombolytics.^1,3

Prior NNT summaries directly compared the number of patients with intracranial hemorrhage to the number with a good neurologic outcome, which can be misleading. Any harms due to intracranial hemorrhage are incorporated into a final neurologic outcome assessment, and therefore good functional outcome is far more important than intracranial hemorrhage.

There is, however, a difficult comparison to consider between the chance of neurologic benefit and the increased risk of death. Thrombolytics appear to increase mortality;^1,3 however many patients may be willing to accept this risk for an increased chance of functional improvement. Shared decision making for this already very difficult decision is made even harder by the fact that a mortality harm is more certain than any benefit.

Financial conflicts: A final important source of uncertainty in this research arises from financial conflicts of interest, which are unfortunately common in medical research. Although they do not always invalidate findings, they complicate interpretation because studies with financial conflicts are more likely to report positive findings.^24–27 Such conflicts are well documented in both the original thrombolysis research and the subsequent guidelines²⁸ and are known to affect the quality and conclusions of systematic reviews. This may be an unconscious contributor to the divergent conclusions seen for this topic.²⁹

Like all such work, our summary has inherent limitations. Not included in this review, for instance, are newer trials using advanced imaging to select patients for thrombolysis,^30–33 trials we felt should not be combined with studies using thrombolytics based largely on clinical criteria. Also of note, the Donaldson review uses a different cut-off on the modified Rankin Score (≤3) than others. The authors argue being able to walk and care for one’s self independently is the threshold most would consider a good outcome. While this cut-off may seem unconventional, other reviews find similar harms and benefits using different cutoffs, suggesting to us that cutoff points are not an important differentiator of conclusions.^2,3

The uncertainty in these data may help explain differing views of thrombolysis for stroke: trials are heterogenous with many sources of bias; rare positive findings are marred by baseline imbalances but continue to have outsize influence; outcomes were assessed using imprecise, subjective scales; early terminations tilt the data; outcomes overlap; and conflicts abound. There is no simple summary. We find the only appropriate conclusion is uncertainty.

Summaries of the literature cannot answer questions the literature cannot answer. Thrombolytics for stroke has been among the most debated topics in medicine for three decades, perhaps due to underlying data deficiencies. The Donaldson et al. meta-analysis suggests a potential functional outcome benefit which must be balanced against increased mortality. However, the authors warn us that the findings are at serious risk of bias, and pooling results introduces more bias. Three NNT summaries have now arrived at three conclusions. This question will not be answered by re-analyzing the same data. We need additional, carefully executed trials. 

In the meantime, it is difficult to offer clear guidance. Clinicians should be aware of the weaknesses in these data, and patients deserve to know as well. We believe the data are not compelling enough for specific recommendations. Shared decision making is essential, and physicians are left with the difficult task of guiding patients through a decision without a clear answer.

References

1.         Donaldson L, Fitzgerald E, Flower O, Delaney A. Review article: Why is there still a debate regarding the safety and efficacy of intravenous thrombolysis in the management of presumed acute ischaemic stroke? A systematic review and meta-analysis. Emerg Med Australas 2016;28(5):496–510.

2.         Emberson J, Lees KR, Lyden P, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet 2014;384(9958):1929–35.

3.         Wardlaw JM, Murray V, Berge E, del Zoppo GJ. Thrombolysis for acute ischaemic stroke. Cochrane Database Syst Rev 2014;(7):CD000213.

4.         National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med 1995;333(24):1581–7.

5.         Hacke W, Kaste M, Bluhmki E, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med 2008;359(13):1317–29.

6.         Mann J. Truths about the NINDS study: setting the record straight. West J Med 2002;176(3):192–4.

7.         Alper BS, Foster G, Thabane L, Rae-Grant A, Malone-Moses M, Manheimer E. Thrombolysis with alteplase 3-4.5 hours after acute ischaemic stroke: trial reanalysis adjusted for baseline imbalances. BMJ Evid Based Med 2020;

8.         Hoffman JR, Schriger DL. A graphic reanalysis of the NINDS Trial. Ann Emerg Med 2009;54(3):329–36, 336.e1-35.

9.         Ingall TJ, O’Fallon WM, Asplund K, et al. Findings from the reanalysis of the NINDS tissue plasminogen activator for acute ischemic stroke treatment trial. Stroke 2004;35(10):2418–24.

10.       Kwiatkowski T, Libman R, Tilley BC, et al. The impact of imbalances in baseline stroke severity on outcome in the National Institute of Neurological Disorders and Stroke Recombinant Tissue Plasminogen Activator Stroke Study. Ann Emerg Med 2005;45(4):377–84.

11.       Smith G. Standard deviations: flawed assumptions, tortured data, and other ways to lie with statistics. 2015.

12.       McDonald CJ. Medical Heuristics: The Silent Adjudicators of Clinical Practice. Ann Intern Med 1996;124(1_Part_1):56.

13.       Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336(7650):924–6.

14.       Quinn TJ, Dawson J, Walters MR, Lees KR. Reliability of the modified Rankin Scale: a systematic review. Stroke 2009;40(10):3393–5.

15.       Zhao H, Collier JM, Quah DM, Purvis T, Bernhardt J. The modified Rankin Scale in acute stroke has good inter-rater-reliability but questionable validity. Cerebrovasc Dis 2010;29(2):188–93.

16.       IST-3 collaborative group, Sandercock P, Wardlaw JM, et al. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet 2012;379(9834):2352–63.

17.       Albers GW, Clark WM, Madden KP, Hamilton SA. ATLANTIS trial: results for patients treated within 3 hours of stroke onset. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. Stroke 2002;33(2):493–5.

18.       Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g) : results of a double-blind, placebo-controlled, multicenter study. Thromblytic therapy in acute ischemic stroke study investigators. Stroke 2000;31(4):811–6.

19.       Donnan GA, Davis SM, Chambers BR, et al. Streptokinase for acute ischemic stroke with relationship to time of administration: Australian Streptokinase (ASK) Trial Study Group. JAMA 1996;276(12):961–6.

20.       Multicenter Acute Stroke Trial–Europe Study Group, Hommel M, Cornu C, Boutitie F, Boissel JP. Thrombolytic therapy with streptokinase in acute ischemic stroke. N Engl J Med 1996;335(3):145–50.

21.       Randomised controlled trial of streptokinase, aspirin, and combination of both in treatment of acute ischaemic stroke. Multicentre Acute Stroke Trial–Italy (MAST-I) Group. Lancet 1995;346(8989):1509–14.

22.       Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico (GISSI). Lancet 1986;1(8478):397–402.

23.       Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Lancet 1988;2(8607):349–60.

24.       Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003;290(7):921–8.

25.       Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003;289(4):454–65.

26.       Perlis RH, Perlis CS, Wu Y, Hwang C, Joseph M, Nierenberg AA. Industry Sponsorship and Financial Conflict of Interest in the Reporting of Clinical Trials in Psychiatry. AJP 2005;162(10):1957–60.

27.       Bhandari M, Busse JW, Jackowski D, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 2004;170(4):477–80.

28.       Lenzer J. Alteplase for stroke: money and optimistic claims buttress the “brain attack” campaign. BMJ 2002;324(7339):723–9.

29.       Hansen C, Lundh A, Rasmussen K, Hróbjartsson A. Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality. Cochrane Database Syst Rev 2019;8:MR000047.

30.       Davis SM, Donnan GA, Parsons MW, et al. Effects of alteplase beyond 3 h after stroke in the Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET): a placebo-controlled randomised trial. Lancet Neurol 2008;7(4):299–309.

31.       Ringleb P, Bendszus M, Bluhmki E, et al. Extending the time window for intravenous thrombolysis in acute ischemic stroke using magnetic resonance imaging-based patient selection. Int J Stroke 2019;14(5):483–90.

32.       Ma H, Campbell BCV, Parsons MW, et al. Thrombolysis Guided by Perfusion Imaging up to 9 Hours after Onset of Stroke. N Engl J Med 2019;380(19):1795–803.

33.       Thomalla G, Simonsen CZ, Boutitie F, et al. MRI-Guided Thrombolysis for Stroke with Unknown Time of Onset. N Engl J Med 2018;379(7):611–22.