Blog | Friday, June 17, 2016

The sad corruption of the performance measurement movement


This rant reflects conversations with experts, but it does not qualify as good reporting. I will take some liberties, and have not done careful checking of information. Nonetheless I believe everything that I am typing.

Too many decision makers in the performance measurement field ignore the evidence. While the initial impetus for performance measurement came from the belief that with measurement we could improve adherence to evidence based treatments or testing. If one truly believes in evidence based medicine, then the evidence should be indisputable and we should even have evidence that measuring performance actually helps patients.

But too many who have influence in the decision making for performance measures do so without understanding evidence, the concept of unintended consequences, or even the clinical situations involved. Too often groups lobby for their favorite goals, without really understanding what adopting a performance measure means.

For the near future we are stuck with performance measures and payments influenced by our performance. Hospitals already influence patient care so that their payments do not suffer.

Those who do this are often practicing medicine without a license. While they likely do not really understand that concept, that is the case de facto.

The famous 4-hour pneumonia rule, now rescinded, unfortunately still influences practice. Too many patients get labeled as having community acquired pneumonia on the basis of incomplete data and problem representations that do not match the illness script. This tendency started because hospitals received payments based on how quickly pneumonia patients received antibiotics (first within 4 hours, and later within 6 hours). Thus they “encouraged” the emergency department to start antibiotics presumptively if the patient might have CAP.

Anyone who has studied test characteristics understands the problem that ensued. When you increase sensitivity (and that was what emergency departments under the urging of hospital administrators did), you must decrease specificity. You could not avoid overuse of antibiotics.

Performance measurement decisions too often result from political influences (not from professional politicians but rather from health care organizations ranging from subspecialty societies to hospital organization). As a physician who tries to make patient centered decisions, the more that I hear about how performance measures are actually chosen, the sadder and more disgusted I become.

Perhaps we should always remember that the road to hell is paved with good intentions. Perhaps I am being a bit melodramatic, but I really do believe that unless we transform the performance measurement movement, we will continue to harm rather than benefit patient care.

We need prospective evaluations of careful considered performance measures. These evaluations should resemble phase 3 drug trials, looking for benefits and harms. To do less is a shame.

We should remember that the famous Deming quote is almost always misquoted:

The problem is that Deming actually wrote, “It is wrong to suppose that if you can't measure it, you can't manage it—a costly myth” (my emphasis added)—the exact opposite. Deming consistently cautioned against requiring measurement to guide management decisions, observing that the most important data needed to manage often are unknown and unknowable.

db is the nickname for Robert M. Centor, MD, FACP. db stands both for Dr. Bob and da boss. He is an academic general internist at the University of Alabama School of Medicine, and is the Regional Associate Dean for the Huntsville Regional Medical Campus of UASOM. He still makes inpatient rounds over 100 days each year. This post originally appeared at his blog, db's Medical Rants.