Hess BJ, Weng W, Lynn LA, Holmboe ES, Lipner RS. — American Board of Internal Medicine
Presented: 14th Ottawa Conference, May 2010
Background: Many stakeholders believe assessing physicians' practice performance is essential for improving quality. However, little research has focused on how to set fair performance standards using evidence-based measures by which physicians could be held accountable for public reporting.
Summary of Work: The ABIM Hypertension Practice Improvement Module was used to collect data from 10 clinical and two patient experience measures. Eight internist/subspecialist experts applied the Angoff method to judge how minimally competent physicians would perform on individual measures. Experts then rated each measure's relative importance; the Dunn-Rankin method was applied to establish scoring weights for the final composite measure.
Summary of Results: 657 physicians abstracted 20,131 charts and received 18,706 patient surveys. Composite measure scores were reliable (.94) and normally distributed. The expert panel successfully established reasonable performance criteria and scoring weights, yielding a standard of 42.50 out of 100 points, which classified 4.1% of physicians as not providing acceptable care. Decision consistency on competence was high (.97).
Conclusions: The standard-setting method yielded a credible and defensible standard whose outcome was reasonable and reproducible.
Take-Home Message: Performance of practicing physicians can be reliably assessed using patient-care data, and a fair performance standard can be determined.
For more information about this presentation, please contact Research@abim.org.