My transition from medical student to practicing diagnostician was marked by one key realization: doctors don’t make definitive diagnoses. Many think that we do. Our patients are certainly under that illusion. But even at the best of times, the physician’s job to to determine the probability of disease. Continue reading “Communicating diagnostic uncertainty”
I don’t know how much of what I “learned” in medical school I have since forgotten. It is a lot. Probably more than I remember. I did great on exams, but then it was time to move on to a new semester and piles of new information took priority over the old. There was no time to review or consolidate.
At the time, this just seemed like the way that learning was done. It wasn’t much different from my undergraduate routine: cram for an exam, get the marks, and move on to another topic. This was the way medical school was structured. The expert educators behind my medical school curriculum obviously knew what was best for me – right?
But where is all that knowledge now? Why did it feel like I had to start all over again in residency? Why do the residents I teach now, fresh out of medical school and close to that wealth of information, so often struggle?
We spend so much time trying to learn medicine, but we never really learn how to learn. This post is basically a review of the book “Make It Stick” by Peter Brown, Roddy Roediger, and Mark McDaniel.1 I wish I had been given this book before starting medical school. Actually, it would have been more beneficial before starting university, or even high school. It explains clearly why learning seemed so easy but ephemeral. I had excellent marks throughout high school and university – but if you made me take an exam from any of my past courses right now, I would almost certainly fail. Is that really learning?
These are the key lessons I wish I had learned long ago: Continue reading “Making it Stick”
Survival lessons from the book “Deep Survival: Who lives, who dies, and why” adapted for the ED
“Deep Survival: Who Lives, Who Dies, and Why” by Laurence Gonzales is a book about life and death. It explores disasters – at the top of mountains, in the middle the ocean, in the most remote wilderness – and examines why some people live while others die. Gonzales concludes with book with some advice designed to help readers survive. As I read his advice, I was struck by how applicable it is to the practice of emergency medicine. These are his survival lessons, adapted for the ED: Continue reading “Survival lessons for the emergency department”
I got to sit down in person with Ken Milne this week while at the great SkiBEEM conference and talk about evidence based medicine. I know – life just can’t get much better. The episode of The Skeptics Guide to Emergency Medicine that we recorded about the HEAT trial was just released today. Make sure to check it out!
The episode is: SGEM#146: The HEAT is On – IV Acetaminophen for Fever in the ICU
I briefly covered the HEAT trial in the October 2015 edition of articles of the month:
Turning down the heat: can acetaminophen save lives?
For some reason, people just love to hate on fever. It is present when people are sick, so it must be bad, right? We better rush to treat it. This is a randomized, double blind trial of 690 adult ICU patients with a fever and suspected infection, comparing acetaminophen 1 gram IV every 6 hours to placebo. Not surprisingly (unless you actually believed treating fever was helping patients) there was no difference in the primary outcome of ICU free days. There was also no difference in mortality at 28 or 90 days.
Bottom line: Tylenol is great, but it isn’t needed for febrile patients
Although this brief description is sufficient for spreading the word about new evidence, I think it is incredibly important to develop and in-depth understanding of the trials you use to shape your practice. The structured critical appraisal done on the SGEM is an amazing way to dissect this evidence and learn about EBM at the same time.
The most important question I get asked: how do I used this evidence? When just looking at the HEAT trial alone, I think the SGEM conclusion is appropriate: “We would agree with the authors’ conclusion that intravenous acetaminophen to treat fever in ICU patients thought to be due to an infection did not affect the number of ICU-free days.” However, this conclusion is pretty narrow. If you try to place this trial within the context of everything else we know about treating fever, I think it is safe to say that routinely treating infectious fever is unlikely to alter any clinically important outcomes. However, acetaminophen is safe and effectively manages symptoms, so it is very reasonable to use it for symptom control. (I certainly do.)
How is the BMJ different from MAD Magazine? Well, I have never found a hidden picture by folding the pages of the BMJ, and although the editorials are occasionally pithy, they really can’t compare to the entertainment value of Spy vs Spy. On the other hand, the BMJ is considered a trustworthy collection of important medical information. Both are just words on paper written by people, so why is the BMJ considered more trustworthy? Peer review is supposed to be the answer. Nothing is published in the BMJ without first being reviewed for quality and truth by a panel of experts. But how well does this process of peer review work?
“Dismally” is probably the first word that comes to my mind. Peer review can’t possibly address the systemic problems we have fostered by allowing science to be done by those with a vested interest in the outcomes, but even for evaluating the papers at face value peer review seems to be a poor tool. We rely on content experts, but generally not methodology experts – and its the methodology that makes or breaks a paper. Richard Smith, a former editor of BMJ, had this to say about peer review: “It is slow, expensive, ineffective, a lottery, biased, incapable of detecting fraud and prone to abuse”. (Smith 2004)
Maybe theory and expert opinion don’t convince you (good), but how about some empirical evidence? Baxt and and colleagues (1998) rather mischievously sent out a purposefully flawed manuscript to reviewers. There were both major and minor flaws in the paper. How good were the reviewers? More than 2/3rds of the major errors were never identified by the reviewers. Not great.
Reviewers aren’t even consistent. Peters and Ceci (1982) took 12 papers that had already been published in major psychology journals and just resubmitted them to the same journals, but using different author names and institutional affiliations. Only 3 of the 12 were recognized as duplicates. Of the 9 that were not recognized as duplicates, 8 papers were rejected despite the fact the same paper had been accepted just 1-3 years previously.
There are a lot of things we need to fix about the way that medical research is currently being done. I will probably rant more about those in the future, but the purpose of today’s rant is to discuss the most important solution we have until we are willing to consider a complete overhaul of the entire medical research system: post-publication review.
Peer review is not perfect. It probably has its role, but there is no way that 2 or 3 reviewers can catch everything. Should peer reviewers be trained methodologists? For sure. Should they be required to perform simple tasks, like comparing the manuscript to the initial trial design on ClinicalTrials.gov? Absolutely. Can we expect them to be perfect? Of course not.
However, as soon as a trial is published, the number of reviewers jumps for 2 or 3 to a couple thousand. We can and should have our say about the quality of published research. That is one of the major advantages of the FOAM movement. Look at the brilliance of Ryan Radecki at EM Lit of Note, Rory Spiegel at EM Nerd, Anand Swaminathan (@EMSwami), Salim Rezaie (@srrezaie), and so many more.
But you don’t have to be one of those celebrities to have your say. If you want to take an active and important role in reviewing and critiquing the emergency medicine literature, it is easy to get involved. I would suggest checking out and participating in Ken Milne’s Hot Off the Press project at theSGEM. He takes a brand new publication, does the normal (excellent) SGEM critical appraisal, and then opens the floor to comments from the audience that can be directed directly at the author of the paper. All this while cutting the knowledge translation window down to less than 1 month. Pretty incredible.
The most recent SGEM HOP looked at this systematic review on femoral nerve blocks in the CJEM. Have a read and leave your comments for Ken and the lead author Brandon Ritcey over at the SGEM.
Get involved. Join the discussion. I think that the most important outcome of the FOAM revolution is bound to be the changes we will see in dissemination and analysis of medical literature through our worldwide, asynchronous #FOAMed journal club. Post publication peer review is incredible and is here to stay.
Smith Richard. Travelling but never arriving: reflections of a retiring editor BMJ 2004; 329 :242
Baxt WG, Waeckerle JF, Berlin JA, Callaham ML. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Annals of emergency medicine. 32(3 Pt 1):310-7. 1998. PMID: 9737492
Peters DP, Ceci SJ. Peer-review practices of psychological journals: The fate of published articles, submitted again. Behav Brain Sci. 5(02):187-. 2010.
Ioannidis JP. Why most published research findings are false. PLoS medicine. 2(8):e124. 2005. PMID: 16060722
Millard WB. The Wisdom of Crowds, the Madness of Crowds: Rethinking Peer Review in the Web Era. Annals of Emergency Medicine. 57(1):A13-A20. 2011. [free full text]
Berger E. Peer review: A castle built on sand or the bedrock of scientific publishing?. Annals of Emergency Medicine. 47(2):157-159. 2006. [free full text]
Some of my favorite quotes from Dr. William Osler
Do you need a break from the daily grind? Maybe even from the regular deluge of FOAM flooding your inbox? Maybe what you really need is a good dose of Canada’s hero and most quotable doctor, William Osler. It’s almost impossible to read anything he said and not be inspired to get back to helping patients, teaching students, and just generally enjoying this profession. Here are some of my favorite quotes. (These are all quotes attributed to Osler somewhere. I never fact checked over the years and I certainly don’t have sources for any of them. If the wisdom comes from another source, it doesn’t make it any less wise.)
“The good physician treats the disease; the great physician treats the patient who has the disease.”
“One special advantage of the skeptical attitude of mind is that a man is never vexed to find that after all he has been in the wrong.”
“The first duties of the physician is to educate the masses not to take medicine.”
“One finger in the throat and one in the rectum makes a good diagnostician.”
“The greater the ignorance the greater the dogmatism”
“Avoid wine and women — choose a freckly-faced girl for a wife; they are invariably more amiable.”
“No human being is constituted to know the truth, the whole truth and nothing but the truth; and even the best of men must be content with fragments, with partial glimpses, never the full fruition”
“It is much more important to know what sort of a patient has a disease than what sort of a disease a patient has.”
“The very first step towards success in any occupation is to become interested in it.”
“To study the phenomena of disease without books is to sail an uncharted sea, while to study books without patients is not to go to sea at all.”
“There is no disease more conducive to clinical humility than aneurysm of the aorta.”
“Medicine is a science of uncertainty and an art of probability.”
“Look wise, say nothing, and grunt. Speech was given to conceal thought.”
“It is much simpler to buy books than to read them and easier to read them than to absorb their contents.”
“Varicose veins are the result of an improper selection of grandparents.”
“To confess ignorance is often wiser than to beat about the bush with a hypothetical diagnosis.”
“To it, more than to anything else, I owe whatever success I have had — to this power of settling down to the day’s work and trying to do it to the best of one’s ability, and letting the future take care of itself.”
“Throw away all ambition beyond that of doing the day’s work well. The travelers on the road to success live in the present, heedless of taking thought for the morrow. Live neither in the past nor in the future, but let each day’s work absorb your entire energies, and satisfy your wildest ambition.”
“Perhaps no sin so easily besets us as a sense of self-satisfied superiority to others.”
“In science the credit goes to the man who convinces the world, not to the man to whom the idea first occurs.”
“A physician who treats himself has a fool for a patient.”
“Things cannot always go your way. Learn to accept in silence the minor aggravations, cultivate the gift of taciturnity and consume your own smoke with an extra draught of hard work, so that those about you may not be annoyed with the dust and soot of your complaint.”
“The best preparation for tomorrow is to do today’s work superbly well.”
Research participants are offered a contract: assume the risks of research for the betterment of humanity. We are routinely breaking that contract…
Dear Sir/Madam, we would like you to take part in this research study. We hope this new therapy might prove to be helpful, but research is needed because we are unsure. It is possible it could turn out to be harmful. We are therefore asking you to volunteer, to take this risk, not for your own benefit, but for the benefit of future patients. We thank you very much for your contribution to the greater good, to medicine, and to our understanding of the world.
This is the contract we offer to research participants. We are offering them an unproven therapy that may prove beneficial, but based on history is actually more likely more likely to have no effect or even cause harm. The trade off is a contribution to future patients through improved understanding.
What these individuals are never told is how frequently we break this contract. We allow them to take the risk of an experimental therapy and then we hide the results of the study so that no one can ever benefit. Sometimes the results are hidden for nefarious reasons – the results indicate a therapy is not beneficial, but the researcher makes money from its sale, so the results are purposefully set aside. Other times, the loss of results is more innocent – academics didn’t have time to write them up or the results didn’t seem interesting so journals didn’t want to publish them.
Whatever the reason, unpublished research results represent a breach of the contract made with research participants and is morally reprehensible. Patients are willing to risk their lives for the betterment of medicine and we repay them by hiding results that don’t interest us or affect a company’s bottom line. The resulting void leaves the medical community worse informed than before the research was conducted. I think that this is the largest systemic violation of ethics that exists in medicine today.
How can we fix this? The most important fix is the least likely to be enacted: we should take the responsibility for research completely out of the hands of those with skin in the game. If we care about patients and getting accurate research results, we should never allow pharmaceutical companies to test their own products and then sell us their sciency sounding marketing as if it is valid research. However, this is big business, worth trillions of dollars, and it is unlikely that lowly doctors or patients will be able to create change. This is the realm of politicians and businessmen, not reason and science.
However, there are things you can do as an individual. If you have ever participated in medical research, you should call the researcher and ask where the results are published. If they aren’t published, ask why, and when you can expect to see them. Remind them of their obligation to you as a research participant. You participated specifically so others would benefit, so you have a right to see the results published.
I would also recommend checking out the work done by “AllTrials”. This group is dedicated to transparency in research and ensuring that the results of all trials, past and future, are freely available to be reviewed. As an individual, you can sign their petition here. If you are feeling generous, you can even donate to the cause (I have no affiliation with this group). If you are a member of any patient, nursing, or physician groups, suggest that your organization sign on to show their support.
This is a major issue that is irreparably degrading the quality of current medical science and systematically infringing on the rights of research participants. We must do better.