Gray BM, Vandergrift JL, Lipner RS. — American Board of Internal Medicine
Presented: Academy Health Annual Research Meeting, June 2017
Research Objective: To examine the relationship between diagnostic ability, as measured by a cognitive exam, and 90-day mortality for Medicare beneficiaries.
Study Design: We studied 1,605 primary care internists who initially certified in 2000 and completed the American Board of Medicine's (ABIM) Internal Medicine Maintenance of Certification (MOC) exam some 10 years later. Exam questions assess physicians using clinical vignettes and were categorized by ABIM's Internal Medicine exam committee by tasks. Our measure of cognitive diagnostic ability was percent correct for questions related to diagnostic reasoning or appropriate use of tests.
For each physician, we obtained Medicare data for all of patients who had an outpatient visits with them over the 2007-2012 period. We then restricted our study population to all patient visits occurring in the year after the generalist completed the Internal Medicine MOC exam. To better measure associations with diagnostic ability, we limited beneficiary visits to 221,078 where there had not been an outpatient visit with any provider in the previous 90 days. Our outcome measure was death within 90 days of an outpatient visit with a generalist in our sample (i.e., 90-day mortality). This sort of short-term outcome may be particularly sensitive to diagnostic ability.
We estimated the cross-sectional relationship between 90-day mortality and exam performance using a logit regression model. Our explanatory variables of interest were the percentage of diagnosis/testing questions correct and the percentage of non-diagnosis/testing questions correct (these were mostly questions related to treatment or care decisions). Control variables accounted for patient and physician demographics (patient age, race/ethnicity, eligibility for Medicaid, zip-code median income, 10 Health and Human Services region indicators; physician practice type (e.g., academic, solo), sex, and US-medical school); two sets of lagged risk adjusters (18 chronic condition indicators and CMS’s hierarchical condition category index); and year indicators and explanatory variables that accounted for difference in exam questions across exam forms (exam form indicators). To account for correlated errors due to multiple beneficiary visits per physician, we applied Huber-White physician cluster adjusted standard errors.
Population Studied: Medicare beneficiaries treated by mid-career non-subspecializing internists.
Principal Findings: Overall beneficiary 90-day mortality rate was 0.8% and physicians answered 75.0% (SD=8.2%) of diagnostic/testing questions correctly. These questions made up 55% of the exam. We found that answering 10% more diagnostic/testing questions correctly was associated with a statistically significant 8.5% reduction in 90-day mortality (P=.015; 95% confidence interval (CI) -16.0% to -1.4%) but a highly insignificant association with percent correct on other questions (2.8% increase (P=.838; 95% CI -5.2 to 10.1)).
Conclusions: Diagnostic ability as measured through a cognitive exam is associated with short-term mortality of Medicare beneficiaries.
Implications for Policy or Practice: Although preliminary, evidence for this study suggests that interventions targeted at improving diagnostic cognitive ability (e.g., studying for MOC exams or other forms of continuing education) may be an effective means to improve health outcome but more research is need to determine if improvements in this ability results in improvements in outcomes.
For more information about this presentation, please contact Research@abim.org.