Resident surgical skills web-based evaluation: a comparison of 2 assessment tools Journal Article uri icon

Overview

abstract
  • BACKGROUND: Evaluation of surgical skill competency is necessary as graduate medical education moves toward a competency-based curriculum. This study by the American Board of Orthopaedic Surgery (ABOS) and the Council of Orthopaedic Residency Directors (CORD) compares 2 web-based evaluation tools that assess the level of autonomy that is demonstrated by residents during surgical procedures in the operating room as measured by faculty.
    METHODS: Two hundred and ninety-four residents from 16 orthopaedic surgery residency programs were evaluated by 370 faculty using 2 web-based evaluation tools in a crossover design in which residents requested faculty review of their surgical skills before starting a case. One thousand, one hundred and fifty Ottawa Surgical Competency Operating Room Evaluation (O-Score) assessments, which included a 9-question evaluation of 8 steps of the surgical procedure, were compared with 1,186 P-score evaluations, which included a single-question summative evaluation. Twenty-five different surgical procedures were evaluated.
    RESULTS: There were no significant differences in rates of resident requests or faculty completion of the 2 scores. The most common surgical procedures that were assessed were total knee arthroplasty (n = 254, 11%), carpal tunnel release (n = 191, 8%), open reduction and internal fixation (ORIF) of stable hip fractures (n = 170, 7%), ORIF of simple ankle fractures (n = 169, 7%), and total hip arthroplasty (n = 166, 7%). Both instruments disclosed significant differences in competency among entry, intermediate, and advanced-level residents. The findings support the construct validity of the evaluation method. The survey results indicated that >70% of the faculty were confident that use of either the P-score or the O-score allowed them to distinguish a resident who can perform the surgery independently from one who needs additional training.
    CONCLUSIONS: This research has led to the modification of the O-score and the P-score into a combined OP-score instrument. The ABOS envisions that the OP-score instrument can be used with an expanded number of surgical procedures as a required element of residency training in the near future.
    CLINICAL RELEVANCE: This study allows the profession of orthopaedic surgery education to take a leadership role in the measurement of competence for surgical skills for orthopaedic surgeons in residency training, an important clinically relevant topic to the practice of orthopaedic surgery.

  • Link to Article
    publication date
  • 2019
  • Research

    keywords
  • Analysis of Variance
  • Clinical Competence/*standards
  • Competency-Based Education/methods
  • Feasibility Studies
  • Humans
  • Internet
  • Internship and Residency/methods/*standards
  • Orthopedic Procedures/*standards
  • Orthopedics/*education/standards
  • United States
  • Additional Document Info

    volume
  • 101
  • issue
  • 5