Blog | Thursday, November 15, 2018

Philosophical musings on tests in medical school and beyond

I recently read the column “On Being A Doctor” in the Annals of Internal Medicine. The story, “Murky Water,” tells the story of a classmate who committed suicide after failing Step 1 of the United States Medical Licensing Examination by one question. Over the last 12 hours I have pondered this story, as well as an article in the same issue titled “The MCAT's Restrictive Effect on the Minority Physician Pipeline: A Legal Perspective.”

What do standardized tests tell us? In college, as a psychology major I took a course in psychometrics. I understand the principles behind designing tests that produce bell shaped curves.

Test scores measure retention of information and the ability to use that information to understand and answer questions. We all know colleagues who are particularly good at taking such tests (even when they do not know the material that well) and other colleagues who consistently underperform on those tests.

Clinician educators see students who do very well during the first two years of medical school and on step 1, but who struggle during clinical rotations. We also see “bottom quartile” students who blossom during their clinical years.

As I have pondered the value of these tests. What are the dimensions of master physicians? The Closler initiative (moving us closer to Osler) lists four large dimensions:
1. Connecting with patients
2. Clinical reasoning
3. Passion in the medical profession
4. Lifelong learning in clinical excellence

Of all these, perhaps #4 could be tested. But even then we have a disconnect. What is the goal of our education? We should understand physiologic and pathophysiologic processes; we should do likewise for relevant anatomy; we should understand laboratory testing and the use of imaging; we should understand our pharmacological options as well as other therapeutic options and be able to weight the risks and benefits of each.

The key word here is understanding. As an educator, too often learners can regurgitate lists of potential diagnoses, but they lack a true understanding necessary for clinical reasoning and decision making. Our education which unfortunately is dominated by the tests and subsequent test scores too often implicitly devalues learning to understand when learning to do well on the test is the goal.

These are not original thoughts. But they are thoughts too often ignored in medical education. We say we want to produce physicians who fit the Closler initiative but then we talk about Step 1, 2, and 3 incessantly. We judge residency applications partly on these scores. We judge our residencies by pass rates on specialty boards.

I suppose tests are necessary evils. But I wonder about the unintended consequences of psychometrically pure tests. We produce some great physicians, but we could do better. We can never reform the curriculum when “the test” represents a high stakes hurdle. We need a different approach to evaluation. We all really know that, but we seem helpless to address the problem.

But on each rotation I work hard to help the students and residents understand what we are doing rather than just memorizing an algorithmic approach. The learners seem to like this approach and sadly find it refreshingly different. And that saddens me.

The story that influenced this post haunts me. How can a test lead to such a tragic end? Yet I understand the implications of these tests. I see it in our students, our residents and practicing physicians. And I do not like it.

db is the nickname for Robert M. Centor, MD, MACP. db stands both for Dr. Bob and da boss. He is an academic general internist at the University of Alabama School of Medicine, and the former Regional Dean for the Huntsville Regional Medical Campus of UASOM. He still makes inpatient rounds regularly at the Birmingham VA and Huntsville Hospital. His current titles are Professor-Emeritus and Chair-Emeritus of the ACP Board of Regents. This post originally appeared at his blog, db's Medical Rants.