Arnold GK, Clauser JC. — American Board of Internal Medicine
Presented: Joint Statistical Meeting, August 2017
Abstract: A crucial part of medical certification or MOC examinations is content coverage of those exams. Content details of an exam are its blueprint. Recent policy changes in MOC programs call for more frequent, shorter test forms. This requirement poses a problem: how well does any form with dozens of test questions adequately cover the blueprint with its many hundreds of testing points?
Imagine a blueprint as a finite population of discrete, testing points and a test form as a sample drawn from that population. Since the form is shorter, both blueprint and form are stratified into larger overlapping content groups. Assessment of form to blueprint coverage and fit is possible. We describe the process.
Using 15 subspecialty exam blueprints, we simulate test form lengths by drawing a series of multinomial, random samples from those blueprints until the combined coverage of the forms matches the blueprint specifications. We use Chi-square, Kolmogorov-Smirnov, Anderson-Darling and EDF statistics described by Zhang (2001) to assess goodness-of-fit of the simulated forms to the subspecialty blueprints and estimate sensitivity of the statistics to form length.
For more information about this presentation, please contact Research@abim.org.