American College of Physicians: Internal Medicine — Doctors for Adults ®

 
Advertisement
Tuesday, February 9, 2016

Randomized trials and how we know what we know

I suppose I might be more expert in randomized controlled trials if I had ever had the actual opportunity to fetch a pail of water without 1 when my foot caught on fire, as I've said I would do. I can't say I'm sorry that hasn't happened.

I feel qualified to opine on the topic just the same. I have designed, conducted, and published dozens of such trials. I have written 2 textbooks about them, too, 1 addressing details of methodology, the other addressing both that, and its application to clinical decisions. I know a thing or 2 about randomized trials.

So here's the punch line: I know a thing or 2 without a need for randomized trials, too.

There is a fantasy taking over the world of nutrition, especially acute in the aftermath of the contentious Dietary Guidelines release, that nobody really knows anything. The arguments are made at times by seemingly expert people, although we often find they are either not the experts they pretend to be, or are badly conflicted. Or, sometimes, both.

One of the shibboleths with which this camp routinely differentiates itself is the contention that all reliable knowledge, in science, at least, results from randomized, controlled trials (RCTs). Implied, if not stated, is that RCTs are not just necessary and better, but presumably, infallible. The argument continues that such trials are glaringly absent in nutrition, and then finishes with the flourish: We therefore know nothing about nutrition. I can only guess how much Big Food loves this sequence.

It is, however, nonsense, from start to finish. We know plenty about the basic care and feeding of Homo sapiens, in part from excellent RCTs, but by no means limited to them.

For starters, RCTs do only a very specific job, although admittedly, they can do it uniquely well. They are designed to answer questions when there is considerable uncertainty about the best or right thing to do. In the absence of such uncertainty, RCTs quickly bog down in ethical problems. We have, for instance, no RCTs of treating gunshot wounds to the chest or abdomen, versus watching them bleed to see what happens. We have no RCTs of actual vs. sham emergency surgery in this circumstance, or comparisons of trauma surgery to Gregorian chants.

Similar reasoning extends well beyond the bounds of the emergency department. We have no RCTs of spraying water on a house fire vs. watching it burn to see which saves more of a family's possessions. We have no RCTs of spraying water vs. spraying gasoline.

These silly examples aren't as silly as they seem. They point out 2 serious flaws in the RCT fantasy: (1) for ethical reasons, you simply cannot always run a RCT, and (2), when you do run 1, the answer is only ever as good as the question.

Randomization, technically, is a methodologic defense against something called confounding, which is the influence of an overlooked variable. For instance, if one compares coffee drinkers to non-drinkers and finds more emphysema in the former group, it suggests that coffee might cause emphysema. If, however, coffee drinkers smoke far more often than non-drinkers, it would account for the finding without indicting coffee. Coffee is an innocent bystander. There are innumerable variations on this theme; randomization is a robust, albeit imperfect, defense against them.

Blinding, as in “double blind,” is a defense against bias. The idea is that if no one knows who is in which group, no one can contrive the results—intentionally, or otherwise—to correspond with expectations or hopes. However, it's rather difficult to blind people randomly assigned to, say, beef vs. broccoli. They tend to notice the difference. The technique is very useful, but not universally applicable. It is most important when the outcomes are least definitive, and most subjective. If, for instance, the outcome is survival rate, you can imagine the difficulty, not to mention legal problems, in contriving it.

Finally, controls, or as they are generally known, placebos, serve to distinguish specific from non-specific effects. If, for instance, you compare a pain pill to nothing, and pain gets better with the pill, it might be due to specific effects of the pill. However, it might be due, in part, to people getting “something” expecting to get better, and people getting “nothing” expecting no such luck. These expectations map to a complex physiologic response that can, itself, relieve pain and exert other effects. Placebos and control groups guard against mistaking the effects of expectation for the effects of a given treatment.

So, RCTs have decided strengths. But they have rather profound limitations, too. They tend to require rather large treatment effects in relatively short periods of time. If we are looking for effects over a lifetime, in a study of, say, longevity, and feel we need a RCT, then our RCT will need to last 100 years. Those aren't done very often.

The strict stipulation of inclusion and exclusion of RCTs makes them quite robust in one way, but very contrived in another. The result is that: what happens in a RCT may stay with the RCT. In other words, people who agree to participate and play by the trial rules may look too little like the rest of the world to tell us much of anything applicable to it. And as noted, ethics alone preclude RCTs in many circumstances.

Lastly, RCTs can get it wrong, badly wrong. This can happen because the trial is flawed in some way, or the question is misguided; or it can happen because the results are sound, but misinterpreted by scientists, the media, or a bit of both. I won't repeat the tale here, but colleagues and I discovered that what we thought we knew about hormone replacement at menopause based on observational trials was a bit wrong in one direction, while what we thought we learned from subsequent RCTs was at least as wrong in a different direction.

Just about everything currently passing for wisdom about RCTs and nutrition is wrong. The claim that we have no RCTs is wrong; we have many, and some quite dazzling. The claim that other forms of evidence are inevitably lesser is wrong; sometimes other data sources are all there is. Blue Zone populations have not been randomly assigned to live as they do, but how absurd to ignore their shining example for that reason. Results at the level of whole populations over a span of generations trump just about anything we could hope for in even the most lavish of RCTs. The idea that RCTs are themselves infallible is every bit as silly as the questions they are sometimes designed to answer.

And finally, and most importantly: You don't need me or anyone else to tell you that you know some things pretty darn well in the utter absence of evidence from randomized trials. Just ask yourself what you would do about it if your foot ever caught on fire.

David L. Katz, MD, FACP, MPH, FACPM, is an internationally renowned authority on nutrition, weight management, and the prevention of chronic disease, and an internationally recognized leader in integrative medicine and patient-centered care. He is a board certified specialist in both Internal Medicine, and Preventive Medicine/Public Health, and Associate Professor (adjunct) in Public Health Practice at the Yale University School of Medicine. He is the Director and founder (1998) of Yale University's Prevention Research Center; Director and founder of the Integrative Medicine Center at Griffin Hospital (2000) in Derby, Conn.; founder and president of the non-profit Turn the Tide Foundation; and formerly the Director of Medical Studies in Public Health at the Yale School of Medicine for eight years. This post originally appeared on his blog at The Huffington Post.

Labels: , , , ,

0 Comments:

Post a Comment

Subscribe to Post Comments [Atom]

<< Home

This is a printer-friendly version of this page

Print this page  |  Close the preview

Advertisement

Share

 

Contact ACP Internist

Send comments to ACP Internist staff at acpinternist@acponline.org.

Blog log

Members of the American College of Physicians contribute posts from their own sites to ACP Internistand ACP Hospitalist. Contributors include:

Albert Fuchs, MD
Albert Fuchs, MD, FACP, graduated from the University of California, Los Angeles School of Medicine, where he also did his internal medicine training. Certified by the American Board of Internal Medicine, Dr. Fuchs spent three years as a full-time faculty member at UCLA School of Medicine before opening his private practice in Beverly Hills in 2000.

And Thus, It Begins
Amanda Xi, ACP Medical Student Member, is a first-year medical student at the OUWB School of Medicine, charter class of 2015, in Rochester, Mich., from which she which chronicles her journey through medical training from day 1 of medical school.

Auscultation
Ira S. Nash, MD, FACP, is the senior vice president and executive director of the North Shore-LIJ Medical Group, and a professor of Cardiology and Population Health at Hofstra North Shore-LIJ School of Medicine. He is Board Certified in Internal Medicine and Cardiovascular Diseases and was in the private practice of cardiology before joining the full-time faculty of Massachusetts General Hospital.

Zackary Berger
Zackary Berger, MD, ACP Member, is a primary care doctor and general internist in the Division of General Internal Medicine at Johns Hopkins. His research interests include doctor-patient communication, bioethics, and systematic reviews.

Controversies in Hospital Infection Prevention
Run by three ACP Fellows, this blog ponders vexing issues in infection prevention and control, inside and outside the hospital. Daniel J Diekema, MD, FACP, practices infectious diseases, clinical microbiology, and hospital epidemiology in Iowa City, Iowa, splitting time between seeing patients with infectious diseases, diagnosing infections in the microbiology laboratory, and trying to prevent infections in the hospital. Michael B. Edmond, MD, FACP, is a hospital epidemiologist in Iowa City, IA, with a focus on understanding why infections occur in the hospital and ways to prevent these infections, and sees patients in the inpatient and outpatient settings. Eli N. Perencevich, MD, ACP Member, is an infectious disease physician and epidemiologist in Iowa City, Iowa, who studies methods to halt the spread of resistant bacteria in our hospitals (including novel ways to get everyone to wash their hands).

db's Medical Rants
Robert M. Centor, MD, FACP, contributes short essays contemplating medicine and the health care system.

Suneel Dhand, MD, ACP Member
Suneel Dhand, MD, ACP Member, is a practicing physician in Massachusetts. He has published numerous articles in clinical medicine, covering a wide range of specialty areas including; pulmonology, cardiology, endocrinology, hematology, and infectious disease. He has also authored chapters in the prestigious "5-Minute Clinical Consult" medical textbook. His other clinical interests include quality improvement, hospital safety, hospital utilization, and the use of technology in health care.

DrDialogue
Juliet K. Mavromatis, MD, FACP, provides a conversation about health topics for patients and health professionals.

Dr. Mintz' Blog
Matthew Mintz, MD, FACP, has practiced internal medicine for more than a decade and is an Associate Professor of Medicine at an academic medical center on the East Coast. His time is split between teaching medical students and residents, and caring for patients.

Everything Health
Toni Brayer, MD, FACP, blogs about the rapid changes in science, medicine, health and healing in the 21st century.

FutureDocs
Vineet Arora, MD, FACP, is Associate Program Director for the Internal Medicine Residency and Assistant Dean of Scholarship & Discovery at the Pritzker School of Medicine for the University of Chicago. Her education and research focus is on resident duty hours, patient handoffs, medical professionalism, and quality of hospital care. She is also an academic hospitalist.

Glass Hospital
John H. Schumann, MD, FACP, provides transparency on the workings of medical practice and the complexities of hospital care, illuminates the emotional and cognitive aspects of caregiving and decision-making from the perspective of an active primary care physician, and offers behind-the-scenes portraits of hospital sanctums and the people who inhabit them.

Gut Check
Ryan Madanick, MD, ACP Member, is a gastroenterologist at the University of North Carolina School of Medicine, and the Program Director for the GI & Hepatology Fellowship Program. He specializes in diseases of the esophagus, with a strong interest in the diagnosis and treatment of patients who have difficult-to-manage esophageal problems such as refractory GERD, heartburn, and chest pain.

I'm dok
Mike Aref, MD, PhD, FACP, is an academic hospitalist with an interest in basic and clinical science and education, with interests in noninvasive monitoring and diagnostic testing using novel bedside imaging modalities, diagnostic reasoning, medical informatics, new medical education modalities, pre-code/code management, palliative care, patient-physician communication, quality improvement, and quantitative biomedical imaging.

Informatics Professor
William Hersh, MD, FACP, Professor and Chair, Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, posts his thoughts on various topics related to biomedical and health informatics.

David Katz, MD
David L. Katz, MD, MPH, FACP, is an internationally renowned authority on nutrition, weight management, and the prevention of chronic disease, and an internationally recognized leader in integrative medicine and patient-centered care.

Just Oncology
Richard Just, MD, ACP Member, has 36 years in clinical practice of hematology and medical oncology. His blog is a joint publication with Gregg Masters, MPH.

KevinMD
Kevin Pho, MD, ACP Member, offers one of the Web's definitive sites for influential health commentary.

MD Whistleblower
Michael Kirsch, MD, FACP, addresses the joys and challenges of medical practice, including controversies in the doctor-patient relationship, medical ethics and measuring medical quality. When he's not writing, he's performing colonoscopies.

Medical Lessons
Elaine Schattner, MD, FACP, shares her ideas on education, ethics in medicine, health care news and culture. Her views on medicine are informed by her past experiences in caring for patients, as a researcher in cancer immunology, and as a patient who's had breast cancer.

Mired in MedEd
Alexander M. Djuricich, MD, FACP, is the Associate Dean for Continuing Medical Education (CME), and a Program Director in Medicine-Pediatrics at the Indiana University School of Medicine in Indianapolis, where he blogs about medical education.

More Musings
Rob Lamberts, MD, ACP Member, a med-peds and general practice internist, returns with "volume 2" of his personal musings about medicine, life, armadillos and Sasquatch at More Musings (of a Distractible Kind).

Prescriptions
David M. Sack, MD, FACP, practices general gastroenterology at a small community hospital in Connecticut. His blog is a series of musings on medicine, medical care, the health care system and medical ethics, in no particular order.

Reflections of a Grady Doctor
Kimberly Manning, MD, FACP, reflects on the personal side of being a doctor in a community hospital in Atlanta.

The Blog of Paul Sufka
Paul Sufka, MD, ACP Member, is a board certified rheumatologist in St. Paul, Minn. He was a chief resident in internal medicine with the University of Minnesota and then completed his fellowship training in rheumatology in June 2011 at the University of Minnesota Department of Rheumatology. His interests include the use of technology in medicine.

Technology in (Medical) Education
Neil Mehta, MBBS, MS, FACP, is interested in use of technology in education, social media and networking, practice management and evidence-based medicine tools, personal information and knowledge management.

Peter A. Lipson, MD
Peter A. Lipson, MD, ACP Member, is a practicing internist and teaching physician in Southeast Michigan. The blog, which has been around in various forms since 2007, offers musings on the intersection of science, medicine, and culture.

Why is American Health Care So Expensive?
Janice Boughton, MD, FACP, practiced internal medicine for 20 years before adopting a career in hospital and primary care medicine as a locum tenens physician. She lives in Idaho when not traveling.

World's Best Site
Daniel Ginsberg, MD, FACP, is an internal medicine physician who has avidly applied computers to medicine since 1986, when he first wrote medically oriented computer programs. He is in practice in Tacoma, Washington.

Other blogs of note:

American Journal of Medicine
Also known as the Green Journal, the American Journal of Medicine publishes original clinical articles of interest to physicians in internal medicine and its subspecialities, both in academia and community-based practice.

Clinical Correlations
A collaborative medical blog started by Neil Shapiro, MD, ACP Member, associate program director at New York University Medical Center's internal medicine residency program. Faculty, residents and students contribute case studies, mystery quizzes, news, commentary and more.

Interact MD
Michael Benjamin, MD, ACP member, doesn't accept industry money so he can create an independent, clinician-reviewed space on the Internet for physicians to report and comment on the medical news of the day.

PLoS Blog
The Public Library of Science's open access materials include a blog.

White Coat Rants
One of the most popular anonymous blogs written by an emergency room physician.

Powered by Blogger

RSS feed