Blog | Thursday, April 24, 2014

QD: News Every Day--Most doctors, med students couldn't answer this question

One in 4 physicians and medical students correctly answered a single question testing their ability to correctly interpret the false positive rate of a diagnostic test result, a research letter found.

The researchers replicated a 1978 study by asking a convenience sample of physicians, house officers, and students the same question: “If a test to detect a disease whose prevalence is 1/1,000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?”

The convenience sample at a Boston-area hospital assessed answers of 24 attending physicians, 26 house officers, 10 medical students, and 1 retired physician, across a range of clinical specialties. Results appeared online April 21 at JAMA Internal Medicine.

Only 14 doctors and students provided the correct answer of 2%. The most common answer was 95% (27 respondents). Among the range of answers, from 0.005% to 96%, the median was 66%, which authors noted is 33 times larger than the true answer.

Responses in the replication study were not significantly different from the 11 of 60 correct responses (18%) in the original study. Also, in both studies the most common answer was 95%. Respondents often knew to compute positive predictive value but accounted for prevalence incorrectly.

The researchers noted that understanding positive predictive value “is particularly important when screening for unlikely conditions, where even nominally sensitive and specific tests can be diagnostically uninformative.”

The authors suggested incorporating aids into ordering software, and even using point-of-care tools like smartphones. They also suggested revising premedical education standards to incorporate training in statistics instead of calculus, which is seldom used in clinical practice.

In an accompanying editor’s note, Joseph S. Ross, MD, MPH, ACP Member, noted that disease prevalence matters for testing, as does test sensitivity and specificity.

“The persistent inability of physicians to reliably manage this cognitive exercise implies that our educational programs need to do a better job at teaching numeracy skills,” he wrote. “Because imprecise diagnostic decision making is leading to excessive testing, patient harm, and excessively costly care, we must raise the bar and master these cognitive skills.”