Development of a learning portfolio to assess the competency of anesthesia residents in Thailand

Background: Recently, a competency-based educational system has been recommended for anesthesia residents in Thailand, instead of a structure-and-process-based medical system. Learning portfolios have become popular and reliable in health profession and education to assess competency and performance in clinical practice. Objectives: Develop a portfolio for learning improvement in first year anesthesia residents in Thailand, and validate this portfolio as a competency evaluation and to identify strength and weakness of implementation. Methods: A learning portfolio was developed from Thai Medical Council general competencies, academic activities, and performance assessment in several modalities. Twenty-four first year anesthesia residents and eight mentors were enrolled for this study. One staff mentored three residents and rated their competencies in portfolios, twice, four-months apart. Content validity was assessed by six content experts. Concurrent validity of portfolio was determined by agreement with faculty global rating and in-training examination. Inter-rater reliability of portfolio was evaluated by five faculties that rated 24 residents. Practicality was commented upon by all mentors and residents in the questionnaire and semi-structure, open-ended questions. Results: All content experts accepted that this portfolio could assess general competencies of the first year anesthesia residents. Concurrent validity of portfolio was demonstrated by high overall agreement with faculty global rating and in-training examination. Inter-rater reliability was good. The majority of mentors and residents (>70%) agreed with the benefit of portfolio based on learning development and competency assessment. However, half of residents were not satisfied with the burden from portfolio. Conclusion: The present learning portfolio provided benefit in learning improvement. It was a valid and reliable tool in competency assessment, but a burden, in the views of the residents.