Making it Stick (How to learn better)

I don’t know how much of what I “learned” in medical school I have since forgotten. It is a lot. Probably more than I remember. I did great on exams, but then it was time to move on to a new semester and piles of new information took priority over the old. There was no time to review or consolidate.

At the time, this just seemed like the way that learning was done. It wasn’t much different from my undergraduate routine: cram for an exam, get the marks, and move on to another topic. This was the way medical school was structured. The expert educators behind my medical school curriculum obviously knew what was best for me – right?

But where is all that knowledge now? Why did it feel like I had to start all over again in residency? Why do the residents I teach now, fresh out of medical school and close to that wealth of information, so often struggle?

We spend so much time trying to learn medicine, but we never really learn how to learn. This post is basically a review of the book “Make It Stick” by Peter Brown, Roddy Roediger, and Mark McDaniel.¹ I wish I had been given this book before starting medical school. Actually, it would have been more beneficial before starting university, or even high school. It explains clearly why learning seemed so easy but ephemeral. I had excellent marks throughout high school and university – but if you made me take an exam from any of my past courses right now, I would almost certainly fail. Is that really learning?

These are the key lessons I wish I had learned long ago:

Learning is deeper and more durable when it is effortful

Learning is deeper and more durable when it’s effortful. Learning that’s easy is like writing in sand, here today and gone tomorrow.”¹

If there is one lesson that encompasses everything else in the book, it is that true learning is effortful. When learning is easy, it is often superficial and soon forgotten. This might seem obvious when written down, but the value of effort was never obvious to me. I always thought that the easiest way to get information into my head was the best way. I learned the order of the prime ministers of Canada by listening to a recording that I played on repeat while I slept. And it worked! I passed my exam easily. (I never considered that the effort it took to record the tape might have had something to do with my retaining the information.)

We all know that “cramming” for a test might help you remember the material for the next day, but does not efficiently promote long term retention. Unfortunately, we are poor judges of how well we are learning (at least for the long term.) Re-reading and massed practice (practicing the same type of problem over and over again until you’ve ‘got it’) are techniques that almost all of us have used in school that turn out to be poor learning strategies for long term retention. However, Brown presents a number of studies illustrating that, even when faced with lower test scores, students swear that massed practice is the best way for them to learn.

Rereading text and massed practice of a skill or new knowledge are by far the preferred study strategies of learners of all stripes, but they’re also among the least productive.”¹

As I progressed through the book, I recognized the various studying habits I had employed during medical school. Unfortunately, they were all given as examples of what not to do.  They were easy and seemed effective at the time, but judging by my degree of retention less than 10 years out of medical school, I was probably wrong. The techniques that Brown promotes, and that I wish I had been taught, are: retrieval practice, periodic practice, and interleaving.

Retrieval practice

Retrieval is the act of recalling something from your memory. Flashcards are an example (which, unfortunately, I always thought were a little bit nerdy.) The effort required to bring back concepts you have learned results in stronger connections and longer memory. Retrieval is also essential in identifying areas of weakness that can be address later.

This is contrasted with the more common and easier technique of reviewing or re-reading material. This was always my favorite technique. It is easy to glance over a page and tell yourself that you know it. Even when I couldn’t recall exactly where isocitrate dehydrogenase fit into the Kreb cycle, I could glance down at the page and say, “Of course – I knew that.”

There is good evidence that practicing retrieval makes learning stick better than re-exposure to the original material, but how long should you wait to practice retrieval? Long enough that the information isn’t still sitting on the tip of your tongue. It should take some effort. Some forgetting between practice sessions might actually be a good thing. To be most effective, retrieval must be repeated, but spaced far enough that instead of mindless repetition, it requires cognitive effort.

The act of retrieving learning from memory has two profound benefits. One, it tells you what you know and don’t know, and therefore where to focus further study to improve the areas where you’re weak. Two, recalling what you have learned causes your brain to reconsolidate the memory, which strengthens its connections to what you already know and makes it easier for you to recall in the future.”¹

Periodic practice

As I mentioned above, to really make the most out of retrieval, you need to space out your studying or practice sessions. When you space out your sessions, retrieval is more difficult. This often makes the practice session seem less productive, but, in fact, the opposite is true.

 When you space out practice at a task and get a little rusty between sessions, or you interleave the practice of two or more subjects, retrieval is harder and feels less productive, but the effort produces longer lasting learning and enables more versatile application of it in later settings.”

Most the the learning I have done in my life has been focused around one time events: exams in school and conferences as a practicing physician. It is far to easy to study intensely around these events, feel accomplished, and then never return to the material agian. I imagine I am not the only person who has noticed (too late) that this technique does not promote long term retention. It is essential that you have some mechanism that ensures that you review new material regularly.

Interleaved or varied practice

Massed practice (focusing on mastering a single type of problem at a time) leads to immediate gains that are evident. However, the rapid forgetting that follows is far less obvious. When I repeat a phone number 3 or 4 times in a row, I will remember it long enough to make the phone call. If you ask me for the number the next day, though, you are out of luck.

Learning that is spaced out and mixed with other types of learning is more difficult, but results in longer retention. Although there is no way that I could memorize a random 16 digit number, the fact that I have been forced to reproduce my credit card numbers at many different times in many different situations means that I can easily recall the numbers of multiple different cards.

Interleaving has another important advantage: by switching between different types of problems instead of focusing on just a single problem, interleaving can help learners discriminate between these problem types, learn to select the right type of solution, and understand the problem within a broader context. I think this is essential in medical education. We often get very focused on teaching a single type of problem. “Today, we are going to master the approach to chest pain”.  Or even worse, “today, we will learn about the presentation of pulmonary embolism”. However, real patients don’t present like they do on exams. If I only ever practice trauma resuscitation in the setting of an ATLS course (where all the cases presented are trauma), I am setting myself up to miss the diagnosis of trauma in the more subtle, intoxicated altered mental status patient. (More on this concept in a later post.)

Testing

I hate being tested. I expect to perform well on tests, so getting a good grade does nothing to boost my confidence. However, a failing grade could deal a crushing blow to the bedrock of confidence and presumed intelligence on which I base my daily performance.

We are all subject to various biases. The Dunning-Kruger effect tells us that as learners we will often over-estimate our knowledge or skill.² Therefore, testing is essential to calibrate what we have actually learned with what we think we have learned. (Testing does not have to be formal. Individual, private retrieval practice is an excellent, non-threatening way to test your own knowledge, if done correctly.)

Testing, although loaded with emotional baggage, is essential at all levels of performance. When discussed at conferences, we often discuss the Dunning-Kruger graph as if it is a single curve – the path towards excellence in our field. In reality, the mastery of a profession is built on thousands of these curves. I may have moved well towards the right in my understanding and management of anaphylaxis, but find myself sitting proudly atop “Mount Stupid” in the area of bedside ultrasound. Testing is what allows me to identify those shortcomings and continue to improve.

Striving

To become an expert, you must constantly strive to surpass your current level of ability. Striving, by its very nature, will result in setbacks. If you do not experience setbacks, you are not setting your goals high enough. These setbacks are essential because they provide information about what is required to gain mastery.

Desirable difficulties

A recurring theme above is that learning requires effort. The more cognitive effort required, the more likely learning will stick. It needs to be hard. Retrieval is harder then review. Interleaving is harder than massed practice. Spaced repetition feels hard, but works. Borrowing a term from psychologists Elizabeth and Robert Bjork, Brown calls these short term impediments that result in stronger long term learning “desirable difficulties”.³

It is important to note that these desirable difficulties are all hard in a very specific way: they require cognitive energy to be expended on the topic to be learned. There are lots of other ways to make learning hard that would not have the same positive effect. Yelling at someone, or putting them on the spot – the classic “pimping” in medicine – definitely makes recall more difficult, but because the difficulty is external to the material being learned it doesn’t aid in learning. A text full of difficult or esoteric language will make learning more difficult, but probably won’t help retention. (However, difficult words might force some people to stop and rephrase things in their own words, which would actually help learning.)

It is important that learners understand the concept of desirable difficulties. Elizabeth and Robert Bjork note that if “the learner does not have the background knowledge or skills to respond to them successfully, they become undesirable difficulties.”³ It does our learners no good if we just start using the concept of interleaving in their teaching, or throwing extra tests at them. We have to first explain these concepts. To improve learning, we must first teach learning.

Robert Bjork talking about desirable difficulties

Learning is stronger when it matters

Learning is stronger when it matters, when the abstract is made concrete and personal”¹

I wasn’t sure if I should include this lesson. I think we all know that we learn better when we can see the value of learning. This is why we have always emphasized the importance of reading around cases. We learn, because that information is necessary to treat the patient in front of us.

However, I am not sure we do a great job connecting knowledge with its real value in medicine. During my clerkship rotations, I never really learned surgical techniques, because I knew I wasn’t going to be a surgeon. I wasn’t aware that I might be able to apply plastic surgery techniques in the emergency department, or that some day I could be tasked with opening a chest or performing a resuscitative hysterotomy.

As teachers, we can definitely do a better job illustrating the value of knowledge, but learners can also take on this responsibility. Reading about congenital heart disease can be painful. However, if you think about the possibility of working in a single coverage hospital and having a child arrive in the middle of a Tet spell, all of a sudden you have a very strong drive to learn. Imagination can help learning.

Is intelligence inherent?

I am not going to answer any questions about nature versus nurture here. If you made it into medical school, you are smart, but your beliefs about intelligence can have profound impacts on your learning.

There are studies (primarily limited to younger children) that illustrate that if after a test you praise a person’s intelligence, in the future they will focus on easier tests because they think that performance is an indication of their intelligence. Their learning will stagnate. However, if you praise their effort, they will continuously take harder and harder tests. In the long run, students who had their effort praised over their intelligence performed better in school.

I see the consequences of this all the time when teaching medical students. They are often afraid to put their neck out; to answer questions. They have made it through life primarily by being intelligent. Getting one more question right won’t shift the needle a lot. On the other hand, getting a question wrong, especially in front of their peers, could hurt their image.

But I am not asking learners questions to assess how intelligent they are. I am asking questions to examine their current state of expertise, so that I can focus my teaching on areas of need. I don’t care how intelligent you are. I care how well you are currently performing in the care of emergency department patients. Intelligence (alone) does not make great physicians. Excellence arises from a commitment to continuously improving performance. 

Paradoxically, a focus on performance trips up some star athletes. Praised for being “naturals,” they believe their performance is a result of innate gifts. If they’re naturals, the idea goes, they shouldn’t have to work hard to excel, and in fact many simply avoid practicing, because a need to practice is public evidence that their natural gifts are not good enough to cut the mustard after all. A focus on performance instead of on learning and growing causes people to hold back from risk taking or exposing their self-image to ridicule by putting themselves into situations where they have to break a sweat to deliver the critical outcome.”¹

Summary

I wish I had been exposed to this book years ago. Had I learned how to learn before being thrown into medical school’s abyss of endless knowledge, I would be a better doctor today. I think “Make It Stick” is a book every medical student should read, but honestly, it is valuable no matter what your level. I only scratched the surface here. My best advice is just to get the book for yourself.

This post is rather theoretical. If you have reached this point and aren’t really sure what to do with this information, don’t despair. Next week, I will have a follow-up post that highlights some of the practical advice given to integrate retrieval, periodic practice, and interleaving into your own education.

Peter Brown interviewed on the Optimize podcast

Some key concepts of the book by Brian Johnson on the Optimize podcast

A longer an more in-depth talk by Robert Bjork

This post on Cult of Pedagogy explains all of the above, but does a much better job of it.

References

1. Brown, Peter C., Henry L. Roediger, and Mark A. McDaniel. 2014. Make it stick: the science of successful learning.
2. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of personality and social psychology. 77(6):1121-34. 1999. PMID: 10626367
3. Bjork EL and Bjork RA. Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In: Gernsbacher MA et al (eds). Psychology and the real world: Essays illustrating fundamental contributions to society. New York: Worth, 2009.
4. Roediger HL, Karpicke JD. The Power of Testing Memory. Perspect on Psych Science. 1(3):181-210. 2006. [article]