Herrschaft BA. — American Board of Internal Medicine
Parshall CG. — CBT Measurement
Presented: Association of Test Publishers Conference, March 2016
Abstract: The user experience involves the full range of a person's interactions with and attitudes toward a product or service. This includes basic usability elements such as the ease of understanding and using the system, and the larger framework of value and utility to the user. In the case of users of a computer-based test, the experience of an examinee during a test can impact his or her attitude toward the sponsoring organization for years to come. More importantly, to ensure the validity of the measurement of a candidate's true ability, the user experience should not introduce any elements that interfere with the construct validity of the assessment.
The American Board of Internal Medicine (ABIM) recently undertook initiatives aimed at improving different aspects of the user experience of candidates sitting for the medical certification exams. In one initiative, a series of user interface designs were developed and revised in order to make use of the larger screen real estate available on new, 24" widescreen monitors at the test centers. In the second initiative, a medical decision support tool was iteratively developed in conjunction with physician item writers to improve the fidelity to practice of the exam. In this instance, the goal was to incorporate a decision support tool on the exam that physicians often use in practice without needing access to external resources during the assessment.
Both of these initiatives were co-created with physicians, to ensure that the user experience encompassed in the final designs met their needs. As part of this co-creation agenda, a usability study with ABIM diplomates was planned and conducted to learn more about the effectiveness of these two aspects in improving the exam user's experience, and to find ways to refine and improve the design and functionality of the widescreen interface and decision support tool.
This presentation will review the basic elements of software usability, as well as some of the fundamental concepts related to the user experience. These will be addressed within the context of the design processes used in both initiatives. The design of the usability study to test both initiatives will be reviewed to depict the application of the concepts of the user experience to support an assessment program. Video clips from the usability sessions will illustrate some of the key findings of the study.
For more information about this presentation, please contact Research@abim.org.