Morgenstern, J. EPIC-HR: Some underwhelming data on Paxlovid, First10EM, April 12, 2022. Available at:
My initial evidence update for Paxlovid (nirmatrelvir/ritonavir) contained exactly no evidence, because nothing was published, despite recommendations at that time from multiple organizations. Sadly, two years into the pandemic, a large portion of the medical community still seems willing to act based solely on drug company press releases. The EPIC-HR trial has now been published, so we can at least discuss the science, which unfortunately is a bit of a mess. At the end of the day, I am no more certain that Paxlovid truly helps than I was before this trial was published.
Hammond J, Leister-Tebbe H, Gardner A, Abreu P, Bao W, Wisemandle W, Baniecki M, Hendrick VM, Damle B, Simón-Campos A, Pypstra R, Rusnak JM; EPIC-HR Investigators. Oral Nirmatrelvir for High-Risk, Nonhospitalized Adults with Covid-19. N Engl J Med. 2022 Feb 16:NEJMoa2118542. doi: 10.1056/NEJMoa2118542. Epub ahead of print. PMID: 35172054 NCT04960202
This is a phase 2/3 double-blind, placebo controlled RCT.
Adult patients with confirmed, symptomatic COVID-19, within 5 days of symptoms onset, and at least one high risk feature for progression to severe disease.
Exclusions: COVID vaccination, prior COVID infection, anticipated need for hospitalization within 2 days, receipt of convalescent plasma.
Paxlovid (nirmatrelvir plus ritonavir 300mg/100mg) orally every 12 hours for 5 days.
The primary outcome was a composite of all cause mortality and disease specific hospitalization at 28 days. They used a modified intention to treat analysis that only included patients who received the drug within 3 days of symptoms onset, despite their inclusion criteria.
They enrolled 2246 (of a planned 3000) patients. The median age was 46, 51% were male, 80% had a BMI over 25, 39% were smokers, and 33% had hypertension. 66% presented within 3 days of symptom onset.
For their composite primary outcome of hospitalization due to COVID-19 or death, there was a significant reduction with treatment (0.7% vs 6.5%). The results look similar in their secondary analysis looking at all patients (up to 5 days after symptom onset): 0.8% vs 6.3%.
They do not present any statistics around the overall mortality numbers. There were 0 deaths in the treatment group and 9 (1.3%) in the placebo group.
They don’t present overall hospitalization numbers at all.
There is not a difference in adverse events reported, but when adverse events are occurring in 25% of both groups, and can be difficult to pick out a smaller signal of harm from a novel, experimental agent.
Despite the fact that this is a double-blind RCT with promising results which have a lot of people excited, this trial has many problems that leave me very uncertain about the results.
This trial is completely run by Pfizer. The lead author, senior author, and many of the other authors are employees of Pfizer. Therefore, there is a massive conflict of interest that needs to be considered. As I discuss in my EBM bibliography, industry sponsored (let alone designed and run) studies are much more likely to find positive results for their own products. There is empirical evidence that these industry studies are biased or cheating. (Eg, Lexchin 2003) Therefore, we must always downgrade our confidence in studies with such obvious conflicts of interest. If, without the conflict of interest, you read this study and thought there was a 60% chance that Paxlovid works, the conflict of interest means that you probably need to drop that estimate to less than 40%, and perhaps much more. (Of course, those numbers are entirely made up.) Replication is always essential in medicine, but much more so when the person running the trial is also trying to sell you something.
It has been a recurrent problem in COVID research, but this trial uses the incredibly biased and completely irrelevant outcome of “COVID-19 related hospitalizations”. This transforms a relatively objective outcome into a completely subjective one. It means that patients can be admitted to hospital for severe drug reactions, and simply be ignored. And it means that the manuscript provides us with a meaningless number. It doesn’t tell us how many people were admitted to hospital, which is the outcome that doctors and patients actually care about. It only tells us the subset of admissions that they thought were due to COVID (which we all know is incredibly hard to distinguish). I find this especially problematic in this trial, because they don’t even present the all-cause hospitalization numbers. (They don’t even list the real hospitalization numbers in the supplement.) There is no way to know from this manuscript whether paxlovid decreased total hospitalizations, which is the only purported benefit. (For that matter, based on this publication, it is entirely possible that Paxlovid actually increases hospitalizations. Based on the numbers of “COVID related hospitalizations”, and the numbers of adverse events listed, I doubt this is an issue, but there is just no way to know.)
Compared to the completely irrelevant, subjective, and biased outcome of disease specific hospitalization, the 1.3% reduction in mortality would definitely be clinically important. I don’t understand why they don’t present statistics for this finding. (When I plug the numbers into an online calculator, I get a p value of 0.002 for this outcome.) In my mind, this number is really the only promising part of the published manuscript. The other sources of bias still limit my certainty, and it only applies to a very select group of patients who mostly don’t look like the patients I am seeing, but the potential mortality benefit is a finding worth pursuing in future research.
They used a modified intention to treat analysis that only included patients who received the drug within 3 days of symptoms onset, despite including patients up to 5 days after symptom onset. Their statistical plan is not described anywhere in their trial registry, and they don’t really explain why they limited their statistical analysis to patients within 3 days of symptoms despite including patients out to 5 days. I have never seen a study adjust their inclusion criteria within their statistical analysis like that, and in an industry run study, it makes me concerned about potential p-hacking.
The trial was stopped early, enrolling 2246 of the planned 3000 patients. (The number gets even smaller, because they only included 1379 patients in their modified intention to treat analysis.) They did stop at a planned interim analysis, but early termination means that the benefit is likely to be overestimated.
There are a number of secondary outcomes listed in the trial registry, such as duration of symptoms, severity of symptoms, hospital length of stay, and ICU length of stay, which are not listed in the manuscript, which is a form of publication bias. (Considering that Pfizer is the one writing up these results, you can bet that these results would have been listed if they had made Paxlovid look good.)
As I discuss in the longer philosophy interlude in the budesonide article, we must account for pretest probability when assessing new studies. During the pandemic, we have thrown so many different chemicals as COVID-19, we should expect a number of studies to be positive by chance alone. (The graph below from a recent publication in Nature gives a good sense of the low pretest probability required for any individual agent.). Therefore, a single positive study will never be completely convincing. When you add the fact that this is an industry run study with a primary outcome that is inherently problematic, I think that there is less than a 50% chance that these findings are replicated in independent high quality research.
Ultimately, I am just not sure what to do with this data. Outside of a pandemic, I think it is very clear that we would not recommend this drug at this time, and instead demand more evidence. An industry run trial with this many problems is simply not enough to safely base practice on. However, the pandemic may change the math, and the mortality finding might be important. (I honestly don’t know if the pandemic should change our decisions. It worries me to abandon our core EBM principles. If we are using different calculations during the pandemic, I think it is essential that we make that completely clear about it, because otherwise these practices will become forever entrenched, even when the pandemic is finally over.)
Considering the high probability that future independent research refutes these findings, we should be very careful in deciding who should receive this medication. If you wanted to argue that we shouldn’t be prescribing it at all, it probably wouldn’t take much to convince me that you are right. However, presuming that we are going to be prescribing it based on these uncertain but promising results, the significant uncertainty means we really should limit prescriptions to the population in whom it was studied. The data doesn’t seem strong enough to extrapolate to other, unstudied populations. Therefore, until further studies are published, I think the only patients who should be considered for paxlovid are unvaccinated adults with confirmed COVID-19, within 5 days of symptoms onset, and with at least one high risk criteria for progression to severe COVID-19.
There is a second RCT that has been reported through a press release. Based on their trial registry, the EPIC-SR trial looked at standard-risk patients, and focused on symptom resolution rather than hard outcomes. (Sounding like Tamiflu.) I will probably cover it when it is published, but based on the fact that it is essentially a mirror to this study, run by Pfizer, I expect many of the problems to be the same.
(I think we would do well to remember how much money was spent during H1N1 on the ‘miracle drug’ Tamiflu, and just how well that turned out.)
Considering the financial conflict of interest and multiple significant sources of bias, I think this is low quality evidence. The only benefit claimed is in a biased, subjective, and completely non-patient oriented outcome. Outside of a pandemic, I would definitely not prescribe Paxlovid. Whether you think this evidence is strong enough to warrant treatment will depend a lot on how much you think a pandemic changes the rules of evidence based medicine.
Hammond J, Leister-Tebbe H, Gardner A, Abreu P, Bao W, Wisemandle W, Baniecki M, Hendrick VM, Damle B, Simón-Campos A, Pypstra R, Rusnak JM; EPIC-HR Investigators. Oral Nirmatrelvir for High-Risk, Nonhospitalized Adults with Covid-19. N Engl J Med. 2022 Feb 16:NEJMoa2118542. doi: 10.1056/NEJMoa2118542. Epub ahead of print. PMID: 35172054
Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003 May 31;326(7400):1167-70. doi: 10.1136/bmj.326.7400.1167. PMID: 12775614