Skip to Content
Authors Rooney DM, Hungness ES, Darosa DA, Pugh CM
Author Profile(s)
Journal Surgery Volume: 151 Issue: 6 Pages: 796-802
Publish Date 2012 Jun
PubMed ID 22652120

The purpose of this study was to compare faculty ratings between live versus video-recorded resident performances and faculty versus skills coaches’ ratings of video-recorded resident performances.PGY1 residents were observed, video-recorded, and rated during a Verification of Proficiency examination on 4 stations (ie, suturing, laparotomy, central line, and cricothyroidotomy). One surgeon and 2 trained skills coaches independently rated each video-recorded performance (N = 25). The chi-square test was used to compare checklist ratings. Analysis of variance was used to compare global ratings. Intraclass correlations were used to evaluate inter-rater agreement.There were no statistical differences in faculty checklist ratings for live versus video-recorded performances (P > .05), and we found a nearly perfect interrater agreement, intraclass correlation coefficient (ICC) = 0.99 (P < .001). When comparing faculty versus skills coaches’ ratings on video-recorded performances, we found no differences for the global or checklist ratings. Inter-rater agreement was moderately high for the global ratings, ICC = 0.71 (P <. 0.01, 95% confidence interval 0.23-0.96), and nearly perfect for the checklist ratings, ICC = 0.99 (P < .001, 95% confidence interval 0.94-1.00).When assessing residents’ performances, use of video-recorded performance ratings and skills coaches may be viable alternatives to live ratings performed by surgical faculty. Copyright © 2016 The Board of Regents of the University of Wisconsin System