Dugosh JW, Grosso LJ — American Board of Internal Medicine
Wiley D. — ACS Ventures
Presented: Association of Test Publishers, March 2015
Abstract: It is critical for successful testing programs to maintain item pools that are balanced against content and statistical specifications and are large enough to support test administration and security demands. A plan to ensure the quality of the pool is critical to meet this goal. The organizations leading this session have either maintained the rigor of their own high-stakes assessments or managed the review of pools for clients in education, certification and licensure.
Pool quality can be evaluated by several criteria: balance (deviation from target at every blueprint level), depth (availability of variants of items at every level); currency (items reflecting current best evidence), blueprint specificity (exam forms consistently meeting content and statistical standards); focus (items measuring appropriate cognitive ability); and statistical attributes (items meeting desired statistical characteristics).
Representatives from the second program have worked with partners in education and credentialing to implement policies and procedures to ensure pool quality and identify long-term needs for their item pools. Tools used have included data visualization techniques that have aided in communication with key constituents.
For more information about this presentation, please contact Research@abim.org.