Blog | Monday, January 15, 2018

The Emperor of Performance has no clothes


Our greatest and worst attribute is seeing and saying that the Emperor has no clothes. It requires intellectual honesty, a willingness to overcome confirmation bias and perhaps some hubris. Often we make others uncomfortable with the raw honesty of such proclamations.

For years many blogs have decried performance measures. I have written about this problem for over 10 years. Currently I serve on ACP's performance measure committee. In this capacity I have reviewed well over 100 performance measures. Most performance measures have the potential for harming patient care. The committee has given a thumb down to many; you can find them here categorized by disease.

We have argued about performance measures. The best positive argument is that one can use them to assess their patient population. But almost all agree that using them as a payment strategy has dangers and some believe that current strategies are just schemes to decrease physician payment.

As I was considering this problem and old Beach Boys song entered my thoughts: “Wouldn't it be Nice?” With apologies to Brian Wilson:
Wouldn't it be nice if we could measure physician quality?
Wouldn't it be nice if patients had one disease that we could treat perfectly?
Wouldn't it be nice if patients believed our recommendations and could afford their treatments?
Wouldn't it be nice if patients had no side effects?

Of course we do not live in a perfect world. So here are some of the problems of performance measures as a payment strategy:

Performance measures can only measure part of one dimension of quality. And even in that dimension we do not measure performance with all patients. Donabedian stated: “Which of a multitude of possible dimensions and criteria are selected to define quality will, of course, have profound influence on the approaches and methods one employs in the assessment of medical care.”

We can measure management of some diseases, but only common ones. This means that we must exclude a proportion of patients from any measurement.

We cannot measure diagnostic accuracy, and without diagnostic accuracy performance measurement is useless. Multiple times I have had patients admitted to the hospital on perfect treatment for presumed systolic dysfunction, but since the patients really had COPD, OSA and right side heart failure, the treatment was inappropriate. Yet the treating physician received a perfect score for treating a disease that the patient did not have. So you say, why not create diagnostic measures. Researchers and clinicians smarter than us have tried without success.

Performance measures clearly have unintended negative consequences. The added cost to the health care system involved in documenting these measures is immense. These costs are both financial and time costs. Anything that distracts physicians from their patients decreases patient care quality. We all know the 4-hour pneumonia rule story. We have heard of the overtreatment induced with aggressive hemoglobin A1c targets for all patients.

Performance measures are almost never tested prior to adoption. The Emperors of Performance Measures just know that we cannot improve quality without metrics. And they know that metrics are therefore good. Unfortunately these Emperors do not understand patients. Here are a few of the problems:

Patients rarely have one disease, and the other diseases may influence how we treat and prioritize management. Patients have different belief systems. Pediatricians in Mississippi have much higher vaccination rates that those in Davis, Calif. Patients have differing financial situations that impact their ability to buy medications.

We know from the recent Annals of Internal Medicine article that physicians treating underserved populations have worse performance scores than those who work in wealthy suburbs. We know from the NHS studies in Great Britain that focusing on some performance measures leads to deterioration of those measures not required.

Performance measurement as a flawed concept. The Emperors never do a premortem analysis. Practicing physicians all understand the flaws. Imposing these measures without understanding the unintended consequences is akin to practicing medicine without a license. Primum non nocere.

Fortunately, physicians and researchers are finally focusing on this problem. Unfortunately, it is not clear that the Emperors are listening. Perhaps if we scream louder.

db is the nickname for Robert M. Centor, MD, FACP. db stands both for Dr. Bob and da boss. He is an academic general internist at the University of Alabama School of Medicine, and the former Regional Dean for the Huntsville Regional Medical Campus of UASOM. He still makes inpatient rounds regularly at the Birmingham VA and Huntsville Hospital. His current titles are Professor-Emeritus and Chair-Emeritus of the ACP Board of Regents. This post originally appeared at his blog, db's Medical Rants.