Blog | Friday, July 22, 2016

Medical statistics and the art of deception


“There are 3 kinds of lies: lies, damned lies, and statistics.”

There is much truth in this quotation of uncertain provenance. We see this phenomenon regularly in the medical profession. We see it in medical journals when statistics are presented in a manner that exaggerates the benefit of a treatment or a diagnostic test. Massaging numbers is raised to an art form by the pharmaceutical companies who will engage in numerical gymnastics to shine a favorable light on their product. It's massaging, not outright mendacity. The promotional material that pharmaceutical representatives present to doctors is riddled with soft deception.

A favorite from their bag of tricks is to rely upon relative value rather than absolute value. Here's how this works in this hypothetical example.

A drug named Profitsoar is tested to determine if it can reduce the risk of a heart attack. Two thousand patients are participating in the study. Each patients receives either Profitsoar or a placebo at random. Here are the results. Of 1,000 Profitsoar patients, 4 had heart attacks. Of 1,000 placebo patients, 6 had heart attacks.

As is evident, only 2 patients were spared a heart attack by the drug. This is a trivial benefit, as only 6 of 1,000 patients in the placebo group suffered a heart attack. This means that taking the drug provides no meaningful protection for an individual patient. However, the drug companies will highlight the results in relative terms to package the results differently. They will claim that Profitsoar reduced heart attack rates by 33%, which would lure many patients and a few doctors to drink the Kool Aid.

Check out this promotional piece below which was recently mailed to me about Uceris, a steroid that I use at times for colitis patients.

See how low the actual remission rates are for the drug. Only 18% of patients responded to the drug, a small minority, and the placebo rate was 6%. No worries. Just brag that Uceris is 3 times more effective than placebo!

Is this a lie? Not exactly. Is it the truth? Technically yes.

Most physicians are tuned into this deception. I know from my own patients that the public is easily seduced by this slick presentation of data. The next time you see a TV ad for a medication, which will be about 5 minutes after you turn on the TV, see if you can spot the illusion. You'll have to watch quickly and repeatedly. Like all skilled magicians, these guys are expert at distraction and sleight of hand. Hint: Whenever you hear the word “percent,” as in “35% of patients responded …,” you should pay particular attention.

When we used to see a woman sawed in half on stage, we knew it was a trick even if we couldn't explain how it was done. I've taken you behind the curtain here. Let's make it a fair fight between us and illusionists.

This post by Michael Kirsch, MD, FACP, appeared at MD Whistleblower. Dr. Kirsch is a full time practicing physician and writer who addresses the joys and challenges of medical practice, including controversies in the doctor-patient relationship, medical ethics and measuring medical quality. When he's not writing, he's performing colonoscopies.