TOTB-09: A USE OF THINK ALOUD METHOD TO ASSESS COMPETENCY
Iman Ghaderi, MD, MSc; University of Arizona

What problem in education is addressed by this work?:
In the current model of operative performance assessment, observation of performance by faculty or trained raters and automated measurements such as motion analysis are well-studied methods. While it has been shown they can produce valid assessments of actual performance, they fail to capture what occurs in the mind of a surgeon. The current methods of measuring cognitive abilities are multiple-choice questions, standard clinical caveats and questions with “if-then” structure, which does not provide us enough information about cognitive competency of surgeons in action. Our previous study showed that the "think aloud" method can be used to capture the thought process of a surgeon in a simulated setting and use of "protocol analysis" verbalizations during and after watching videos of critical portions of a procedure could differentiate expert from novice surgeons.

Describe the intervention:
The "think aloud" method can potentially be used to determine if trainees have cognitive elements of competence appropriate to their level of training. We propose use of the "think aloud" as a feasible method to assess surgeons' cognitive competence. The critical cognitive elements of common surgical procedures will be identified by a group of experts to be used as benchmarks.

Describe how this intervention could be applied at other institutions. Please specifically comment on identified barriers that could exist and how they could be overcome:
Implementation and use of "think aloud" method is easily reproducible. Although the "protocol analysis" of verbalizations can be challenging during initial implementation phase in other institutions, accurate use of this method can be facilitated with specific training.