Guille R, Zhu R, Zhang Y, Song H, Lipner RS. — American Board of Internal Medicine
Becker KA, Sun L. — Pearson VUE
Presented: National Council on Measurement in Education Meeting, April 2011
Background: The study proposed a computer adaptive multi-stage test design with asymmetric early termination in comparison to the traditional linear test design for a medical certification test program. The objectives of the study were (1) to permit the majority of high ability examinees to terminate early, thus reducing their testing time, and (2) to evaluate how the standard errors of subscores for low ability examinees changed. The item response data were simulated based on the empirical ability estimates from a representative sample of 6,287 test takers and 3-PL item parameters from 180 multiple choice items. Two multi-stage panels were created by automated test assembly for each objective respectively: a panel of three stages and another of four stages. The results suggested that the first goal was attainable. With regard to the second objective, the results showed that while it retained, it did not remarkably improve the subscore reliability for low ability examinees.
For more information about this presentation, please contact Research@abim.org.