I hate guidelines. I shouldn’t. In theory, summaries of the medical literature that are accessible to practicing clinicians could only be good. Unfortunately, in current practice, medical guidelines are too often biased, unscientific, overreaching, or misleading.
I started residency as a slave to guidelines. I read them all. I strictly followed every recommendation. But it didn’t take me long to become disillusioned.
My first insight into the fallibility of guidelines was on the topic of pediatric urinary tract infections. I started my family medicine residency rigorously following the current guidelines, meaning that a lot of febrile children were catheterized, and subsequently sent for painful medical procedures like the VCUG. (Roberts 1999) However, when I actually read the evidence for myself, I realized that almost everything I had been doing was wrong. I had been hurting children.
This pattern repeated itself over and over again. I would practice according to the newest guidelines, but whenever I took the time to read the evidence, I would be shocked to find that I wasn’t providing my patients with the best care. I was hurting patients; guidelines were hurting my patients.
Problems with guidelines
There are many problems with medical guidelines as they are currently produced and used.
Perhaps the biggest problem is that they are treated as gospel. People often act as if guidelines set the standard of care. Funding decisions and clinical bench marks are based on guidelines, often with little consideration of the underlying quality of evidence. (Lenzer 2013) Physicians often feel compelled to follow guidelines, even when they disagree with the recommendations. (Lenzer 2006; Woolf 1999) Unfortunately, because guidelines try to provide a single answer from complex and uncertain data, and different guidelines committees have different biases and conflicts, different guidelines on the same topic can often provide significantly different recommendations, provoking confusion and anxiety among clinicians and patients alike. (Feder 1994; Burgers 2002; McAlister 2007)
Scientific evidence is always complex and nuanced. There is almost never a single, easy answer. Evidence based medicine requires that we use clinical judgement and consider patient values. All too often, this complexity is lost in guidelines.
If the evidence does not support straightforward conclusions, pretending it does is worse than admitting uncertainty” (Lenzer 2013)
Guidelines rarely highlight scientific uncertainty. I have only seen confidence intervals presented in a clinical guideline one time. It is essentially unheard of for guidelines to discuss reasonable alternative options to their “final recommendation”. Although it is unlikely that every panel member perfectly agreed on every recommendation, discussions and debates are never included, so guidelines come across as infallible, unquestionable, or definitive. (Sniderman 2009)
Furthermore, too many guidelines are biased from the outset. Financial conflicts of interest are still rampant in medicine, including in guideline development. (Brindsley 2013; Choudhry 2002; Gale 2011; Holloway 2008; Kung 2012) Although the Institute of Medicine has recommended that guidelines eliminate (or reduce) conflicts of interest, one survey indicated that 70% of committee chairs and 90% of committee co-chairs have financial conflicts of interest. (Kung 2012) We know that financial conflicts, even if small, will significantly sway judgement. There is evidence that conflicted guidelines lead to drastically different recommendations. (George 2014) At their worst, some guidelines are little more than marketing tools for industry.
Guidelines also tend to severely limit the diversity of opinions expressed. Committees are comprised of a relatively small number of academics with similar backgrounds. (Woolf 1999) Community physicians are rarely represented. The unique needs and insights of rural communities are not included. The difficulties of low and middle income countries are frequently ignored. Guidelines present recommendations as if medical decisions are solely the purview of physicians, and generally overlook the insights of patients. (Lenzer 2013) These issues are made worse when the already small guideline committees are filled with individuals who all started with similar opinions – known as “panel stacking”. (Lenzer 2013; Sniderman 2009)
Although there are many problems with our current guidelines, I still like the idea of easy to read, accessible summaries of the evidence for practicing clinicians. It is simply not feasible for clinicians to read every relevant study – not even me. We need summaries to have any hope of practicing evidence based medicine, but we need to drastically change how those summaries are produced and presented.
I think the most important step is to divide current guidelines into two completely separate documents. Currently, guidelines mix a few recommendations supported by strong evidence with a large amount of expert opinion. There is value in both, but they shouldn’t be in the same document.
In any current guideline, there are only a handful of very strong recommendations, based on multiple, consistent RCT level evidence. These are recommendations, like aspirin for STEMI, that we think should be followed almost all of the time. These very strong recommendations should be pulled out into a separate document, which would probably never be longer than 1 page. These would be the minority of medical practices with a high degree of certainty, and would be reasonable targets for attempts to limit (but not eliminate) variability in practice.
There is still a role for expert opinion and low quality, mixed, or biased evidence. However, mixing it into guidelines is often misleading or confusing, resulting in strong recommendations or funding decisions being based on low quality evidence. In areas that we don’t have strong evidence, I am fine with expert opinion, as long as it is clearly labelled as such at the outset, and it is clear that such recommendations in no way represent a standard of care.
In either case, it is important that guidelines not present their recommendations as “rules” that must be followed. Limitations of the science must be emphasized. Patient variations and values need to be discussed. It should be clear that it is appropriate for clinicians to deviate from guidelines based on their clinical judgment and depending on patient values.
Next, guidelines must embrace and discuss the uncertainty of science. Far too often, complex questions are simplified and presented as if there is a single strong answer. Weaknesses in the underlying evidence must be highlighted. All recommendations should be presented not just with absolute risk reductions or NNTs, but also confidence intervals. Potential harms must be discussed right next to potential benefits. Dissenting opinions from the guidelines committee, if they exist, must be published.
Financial bias is a huge problem in all of science, but is clearly problematic in guidelines. I believe that individuals with financial conflicts of interest should be completely banned from guideline committees. (This is a rather controversial statement. Others would argue for disclosing and managing conflicts of interest, but I think there is good evidence that disclosure can backfire, (Cain 2003) and maintain the firm position that individuals producing guidelines, especially considering the power guidelines currently have in medicine, should have no vested interest in the recommendations.)
The selection of guideline committees also needs to be addressed. Guidelines should be primarily about science, and therefore should be predominated by science experts (methodologists) rather than content experts. (Content experts are still necessary, but should be the minority). Guideline selection also needs to be more representative, including community practitioners, clinicians from rural settings, and representatives of low and middle income countries (as well as many other minority voices I am leaving off the list). Involving patients also makes a lot of sense.
Although a lengthy list, these are not difficult changes to make. Unbiased and nuanced summaries of clinical evidence are already being produced (just look at the FOAMed community). If the general medical community demands more from guidelines, I am optimistic that they will improve.
The ILCOR vision
And now is the time for us to demand more. Although there are many problems with our current medical guidelines, they are not a lost cause. But we all need to get involved if we want guidelines to be helpful tools that improve patient care.
Next month, I am going to be attending the annual ILCOR meeting. As far as guidelines go, I think the ILCOR recommendations (that get translated into guidelines by multiple agencies, such as the AHA and ERC) are pretty good. I always read them. But I was thrilled to hear that they are constantly trying to improve their process.
Drs Robert Neumar and Gavin Perkins, the current co-chairs of ILCOR, wants to change the way that the general medical community interacts with ILCOR and their process of generating consensus on science and treatment recommendations (CoSTRs) that are the foundation for international guidelines on cardiac arrest resuscitation and first aid. After a long conversation with Bob, I am pretty excited about the vision. (Aside from being invited to speak at their meeting, I have no ties to ILCOR. I am paraphrasing Dr. Neumar.)
At the core of the ILCOR philosophy are 2 ideas:
- It is better to draw on the expertise and wisdom of the entire medical community than just a limited number of hand-picked experts.
- Guidelines should not be created behind closed doors. The process should be open and transparent.
The current task forces that generate ILCOR CoSTRs are made up of 17 content experts with diverse methodologic, scientific and clinical expertise from across the world. Although this represents an incredible wealth of knowledge and expertise, a limited number of people can only provide a limited number of viewpoints. ILCOR wants to expand the range of voices that shape their treatment recommendations. Doctors working in the community often have important insights that differ from those in academic centers. Rural communities need a say on the impact and practicality of these guidelines in their setting. The same goes for low and middle income countries. Not every voice can be present at the meeting, but every voice can be heard.
ILCOR wants input at all stages of guideline development. Before the process even begins, the community could help decide what topics are most important to examine. During the systematic review, the community might help identify or critique important research. Once the guideline is written in draft form, the community should be able to comment and criticize. And once the guideline is finalized, the community can be involved in dissemination, discussing how it should be applied in practice, and pointing out when it needs updating.
ILCOR also wants to make this entire process more transparent.
These goals seem to align very well with the general ethos of FOAMed and the medical social media community.
In a few weeks time, I am going to speak at the ILCOR meeting on the ways that they could use social media, or potentially partner with existing FOAMed resources, to improve their guideline process. Now is your chance to shape this process. Leave comments, and I will bring your ideas to the ILCOR committee.
- ILCOR wants community feedback on its guidelines. How would you use social media or FOAMed to encourage that feedback?
- How would you improve the guideline creation process, especially considering the desire to be transparent and interactive?
- Do you have a FOAMed site? Any thoughts on how partnerships with FOAMed would improve the quality and usability of guidelines?
- The most recent ILCOR CoSTRS (both final and draft) are on the ILCOR website. How would you improve this website to encourage feedback and discussion?
Bindslev JB, Schroll J, Gøtzsche PC, Lundh A. Underreporting of conflicts of interest in clinical practice guidelines: cross sectional study. BMC medical ethics. 2013; 14:19. [pubmed]
Burgers JS, Bailey JV, Klazinga NS, et al. Inside guidelines: comparative analysis of recommendations and evidence in diabetes guidelines from 13 countries. Diabetes care. 2002; 25(11):1933-9. [pubmed]
Cain DM, Loewenstein GF, Moore DA. The Dirt on Coming Clean: Perverse Effects of Disclosing Conflicts of Interest. SSRN Journal. 2003.
Choudhry NK, Stelfox HT, Detsky AS. Relationships between authors of clinical practice guidelines and the pharmaceutical industry. JAMA. 2002; 287(5):612-7. [pubmed]
Feder G. Management of mild hypertension: Which guidelenes to follow? BMJ : British Medical Journal. 1994;308(6926):470-471.
Gale EA. Conflicts of interest in guideline panel members. BMJ (Clinical research ed.). 2011; 343:d5728. [pubmed]
George JN, Vesely SK, Woolf SH. Conflicts of interest and clinical recommendations: comparison of two concurrent clinical practice guidelines for primary immune thrombocytopenia developed by different methods. American journal of medical quality : the official journal of the American College of Medical Quality. 2014; 29(1):53-60. [pubmed]
Holloway RG, Mooney CJ, Getchius TS, Edlund WS, Miyasaki JO. Invited Article: Conflicts of interest for authors of American Academy of Neurology clinical practice guidelines. Neurology. 2008; 71(1):57-63. [pubmed]
Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet institute of medicine standards: Two more decades of little, if any, progress. Archives of internal medicine. 2012; 172(21):1628-33. [pubmed]
Lenzer J. NIH secrets: study break. New Republic 2006 Oct 6. http://www.newrepublic.com/article/national-institutes-health-fred-geisler-spinal-cord.
Lenzer J, Hoffman JR, Furberg CD, Ioannidis JP, . Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ (Clinical research ed.). 2013; 347:f5535. [pubmed]
McAlister FA, van Diepen S, Padwal RS, Johnson JA, Majumdar SR. How evidence-based are the recommendations in evidence-based guidelines? PLoS medicine. 2007; 4(8):e250. [pubmed]
Roberts KB. A Synopsis of the American Academy of Pediatrics’ Practice Parameter on the Diagnosis, Treatment, and Evaluation of the Initial Urinary Tract Infection in Febrile Infants and Young Children Pediatrics in Review. 1999; 20(10):344-347.
Sniderman AD, Furberg CD. Why guideline-making requires reform. JAMA. 2009; 301(4):429-31. [pubmed]
Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ (Clinical research ed.). 1999; 318(7182):527-30. [pubmed]
Justin Morgenstern. I hate guidelines (but they can improve and you can help), First10EM, 2018. Available at: