Jump to start of content

Having ABIM Physician Portal Sign In Problems?... Expand/Collapse the ABIM alert.

If you're experiencing sign in issues, please email us at request@abim.org and provide your name and ABIM ID. We'll respond with information to complete the sign in process. We apologize for the delay and appreciate your patience.

Breadcrumb trail:

Assessing decision accuracy and consistency of a medical certification examination.


Song H, Sares TA, Zhu R, Lipner RS. — American Board of Internal Medicine

Presented: National Council on Measurement in Education Meeting, April 2011

Abstract: Accuracy and consistency of pass-fail decisions are of primary interest to high-stakes certification testing programs. For medical certification programs, in particular, decision accuracy and consistency may affect patient safety as well as a physician's career advancement. This study evaluates the applicability of Rudner's decision accuracy and Huynh's decision consistency methods for ABIM certification and Maintenance of Certification (MOC) examinations during transition from classical test theory (CTT) to item response theory (IRT) scoring. The pass-fail decision accuracy and consistency were determined from simulated true and estimated abilities and provided a nonparametric method to cross-validate those model-based estimates. Results verified that Rudner's IRT-based decision accuracy procedures are convenient, efficient and “in line” with IRT scoring classification logic. Results also supported the proposition that the Huynh procedure is applicable for estimating pass-fail decision consistency for exams in the IRT framework via a simple conversion between the IRT and the raw score standards.

For more information about this presentation, please contact Research@abim.org.