PS4-05: WHAT DO SURGEONS THINK? MAPPING SURGEONSâ PERCEPTION TO TECHNICAL SKILL THROUGH TALK ALOUD NARRATIVES.
Kendall Free, Anand O Malpani, Gregory D Hager, Shameema Sikder, S. Swaroop Vedula; The Johns Hopkins University
Introduction: Presumably, surgeons’ assessment of technical skill is driven by their observations of the surgical field. Our objective was to systematically capture surgeons’ thinking or cognitive process regarding a procedure and explore its characteristics across levels of technical skill.
Methods: One surgeon watched videos of capsulorhexis, a critical step in cataract surgery, and freely narrated their thoughts on their observations. We posed no restrictions or specified any scope for the surgeon’s narration, only that it may only be interrupted by very brief pauses and it should last for at least 5 minutes or the duration of the capsulorhexis, whichever was longer. Of narratives on 10 videos that we captured, three, two, one, and four videos were performed by a faculty, PGY-2, PGY-3, and PGY-4, respectively. One expert surgeon (unaware of identify of operating surgeon) assigned a skill class label for each video in three ways - appointment status, i.e., expert (faculty)/novice (trainee); and using the capsulorhexis indices and the first and last global skill indices in ICO-OSCAR:Phaco. For the latter indices, an expert was when the score on at least one index was five (on a Likert scale of two to five), and at least four on the other index. One annotator tagged each narrative with codes pertaining to activities in capsulorhexis and concepts for feedback. We analyzed frequency of tagged codes by levels of technical skill across activities in capsulorhexis represented as a word cloud.
Results: Our analyses show that the observing surgeon's thoughts were driven towards activities commensurate with experience, i.e., inserting cystotome and engaging cystotome for PGY-2/3 and other steps for PGY-4. When using structured rating scales, the narratives on novice instances were focused upon the initial activities in capsulorhexis, whereas the narratives one expert instances spanned all activities.
Conclusions: Our work demonstrates a methodology to capture surgeons’ cognition and map it to intraoperative technical skill assessments. Surgeons’ perception of aspects of a procedure that may need feedback appeared to be influenced by expectations on activities performed by faculty and trainee surgeons.
