Reddy S. — American Board of Internal Medicine
Presented: Society of General Internal Medicine Conference, May 2009
Background: A large body of research evidence has demonstrated that trainees lack competence in medical interviewing, physical examination and informed decision-making. In addition, many quality and safety processes of care are not adequately performed. Supervising physicians are required to evaluate trainees’ clinical skills, have a responsibility to provide the learner with sufficient and timely feedback, and to ensure that clinical care provided by the learner is of high quality and safe. Direct observation should be the primary means of assessing these competencies. Unfortunately, research shows direct observation and systematic review of care delivered by trainees occurs infrequently.
Content: The American Board of Internal Medicine developed the Web-based Clinical Supervision Practice Improvement Module (CS PIM), which enables diplomates to fulfill their Evaluation of Practice Performance requirement for Maintenance of Certification (MOC). The module provides a Mini-CEX-based framework for evaluating trainee clinical skills and patient care using direct observation. The module also asks the physician to identify and document up to three things the learner did well, three things the learner could improve, feedback given, and any changes the trainee made between clinical encounters (if applicable). A medical record audit facilitates review of the trainee note for accuracy and performance on core patient safety measures.
Design: A summary report is generated after completion of at least 10 observation-audit cycles that the physician uses to reflect upon and to develop a personal action plan aimed at improving his or her own direct observation and supervision skills. A beta test was carried out in 2008 to determine how well the CS PIM facilitated the evaluation of trainees.
Evaluation: Eighty-five diplomates enrolled in the beta test and to date 41 completed the module and a feedback survey (48%). Beta testers spent an average of 53% (20%-100%) of their time in patient care activities; 74% (10%-100%) of this time was spent supervising and teaching trainees. All participants (100%) rated the module as very good or excellent in facilitating their observation and evaluation skills. All indicated that they would recommend the module to a colleague and 90% would use it outside of MOC for teaching and supervision. As a result of completing the module, 95% of supervising physicians indicated having made changes to their evaluation strategies and 45% made changes to their own clinical practice. Means scores on the seven mini-CEX domains ranged from 6.4 - 7.2 (9-point scale), but faculty provided ratings from 2-3 (poor) to 9 (superior) on all seven categories. Finally, module navigation and data transfer via the Internet presented no difficulty for 95% of users.
Summary: This CS PIM was well received by supervising physicians as a means of observing and documenting trainee clinical skills. More importantly, it resulted in changes in the way they observed and assessed trainees, and enabled participation in quality improvement through the evaluation of clinical care provided by their trainees. The CS PIM successfully integrates a validated direct observation measure and a key patient safety component into a user-friendly, Web-based option for supervising physicians in MOC.
For more information about this presentation, please contact Research@abim.org.