Blog | Tuesday, May 28, 2013

Scientists should embrace criticism

Some argue that the practice of medicine is more art than science, an antiquated and false dichotomy. Even were it true, modern medicine rests on a foundation of serious science. Most advances in medicine come through scientific discovery and statistical analysis of data.

A lot goes into keeping this system running well and, aside from money, one of the most important factors in medical science is doubt. There's a lot that goes into any advancement in science, a lot of places where things can go wrong. And the way we analyze statistics, a certain number of results will be incorrect despite being statistically "true". Any new finding, large or small, requires testing and re-testing, until the preponderance of the evidence points in one direction.

There have always been threats to this system, a system that has helped wipe out infectious diseases, increase our longevity and quality of life. The one on my mind today is the abandonment of doubt.

We see this in the centuries-old phenomenon of quackery. Before medical science was understood, people tried all manner of cures, most relying on false understandings of biology. The removal of ill humours by bleeding or purging, or the exorcism of evil spirits, and many other practices relied on a superstitious understanding of health and disease and a failure to understand statistics.

Some early uses of statistics in medicine hinted at the revolution to come. In 18th century England, Charles Maitland tested smallpox inoculation on prisoners, then orphans, keeping track of successes and failures and deaths. In the 19th century, Ignaz Semmelweis used statistics to find that hand washing could reduce deaths from childbed fever.

Semmelweis suffered no lack of skepticism from the medical community (his insanity didn't help him much) but as microbiology advanced, the reasons for his success became understood and widely accepted.

Quacks have never favored statistics. Quacks operate by playing to the fears and desires of their marks. More important than numbers are anecdotes, testimonials of how their latest potions or pills made someone better. I like to think of quacks as residing outside the scientific process, a peripheral annoyance, but they have a huge influence on our culture and economy. Look at any example of a "weight-loss miracle" presented by Dr. Oz. He may not endorse the products, but without any need for scientific evidence, quacks can say, "As Seen on Dr. Oz!" and sell, sell, sell.

The other threat to doubt comes from within the scientific establishment. It is seen in phenomena such as "publication bias," where "interesting" positive results are more likely to be published than negative results, giving a treatment a false veneer of success. Another is disturbing because it involves the corruption of individuals, the people we rely on to do the real work behind our discoveries.

The excellent blog, "Retraction Watch" works to bring to light studies and papers that have been published and then withdrawn or given an "asterisk." Normally, a scientific paper isn't retracted because it is wrong. Remember, doubt drives science, and finding out something is wrong is just as important as finding it is right. The problem comes when a paper's data is found to be in some way corrupted, either intentionally by manipulation of data, or simply by disastrous mistakes missed by editors. It's not always easy to prove actual "faking" of data, although the evidence is often clear, so a retraction or other sort of asterisk is a caution, not a legal action. It is an opportunity for the scientific community to police itself and potentially save money and time wasted in following red herrings.

To add a layer of "ick," there seems to be a trend of researches who have suffered retractions lashing out at the messenger. Twice so far this year, Retraction Watch has been threatened with legal action simply for reporting facts. The first regarded Dr. Bharat Aggarwal, a researcher who is under investigation by MD Anderson and who has been the subject of a number of retractions. The second is Ariel Fernandez, a researcher who had a paper given an "expression of concern," a sort of shot across the bow that there are questions about his work.

Both of these men lashed out at Retraction Watch for simply reporting the truth, a truth that helps the scientific process along. Doubt is the null hypothesis, the default state. It is up to a researcher to show why we should believe their suppositions. If they can't do it with good science, then their ideas may not be ripe, they may be wrong, or not yet provable. But to abandon doubt, to replace it with legal threats, essentially saying, "believe what I wrote, and if you don't, don't tell" corrupts this process. It also makes researchers who work may simply be wrong look like they might have something to hide.

Doubt isn't a weapon to be used until it slaughters all creative thoughts. It is a tool to help us avoid the very human instinct to believe what appears to be true, and what we wish to be true. Scientists, most of all, should know this.

Peter A. Lipson, ACP Member, is a practicing internist and teaching physician in Southeast Michigan. After graduating from Rush Medical College in Chicago, he completed his internal medicine residency at Northwestern Memorial Hospital. This post first appeared at his blog at Forbes. His blog, which has been around in various forms since 2007, offers "musings on the intersection of science, medicine, and culture." His writing focuses on the difference between science-based medicine and "everything else," but also speaks to the day-to-day practice of medicine, fatherhood, and whatever else migrates from his head to his keyboard.