Sares TA, Zhu R. — American Board of Internal Medicine
Presented: American Educational Research Association Annual Meeting, April 2012
Abstract: Computer-based testing makes possible the integration of innovative forms of digital media such as image zooming for enhancing the fidelity of the examination. The objective of this study was to compare item statistics and response times with and without the image zooming feature, and to examine candidates' post-exam reactions to the image zooming feature. We utilized data from a medical certification exam that was administered to 508 candidates. The medical exam included 20 image zooming items that were administered previously without the zooming feature. The logistic regression differential item functioning (DIF) method (Swaminathan & Rogers, 1990) provided a model-based approach for testing the impact of zoomed image item variants on item performance. The non-parametric Kruskal-Wallis H test was conducted to examine item response time. The results suggested DIF in five of the 17 zooming items with two items presenting interactions between ability and zooming, and one item uniformly easier with zooming. Two items were easier with less time spent, and one item was harder with no significant change in time spent. This study provides useful insights for other health certification agencies seeking to increase exam fidelity.
For more information about this presentation, please contact Research@abim.org.