Medical Council of Canada

Identifying gender bias in performance assessment (2018-2019) – Evan Tannenbaum

Identifying gender bias in performance assessment (2018-2019) – Evan Tannenbaum

Identifying gender bias in performance assessment

Investigator

Tannenbaum, E.

Co-investigators

Tavares, W.
Kuper, A.

Introduction

Performance-Based Assessments (PBAs) in Health Professions Education (HPE) are intended to be unbiased, however recent evidence suggests otherwise. Differences in the scoring of PBAs exist between men and women, regarding numerical scores and the language used in narrative comments. Sex-role stereotypes have been shown to affect our perceptions of performance, yet we do not yet understand the how such biases influences raters’ judgment and scoring in PBA. Understanding the mechanism of gender bias in PBA is key to establishing ways to improve equity and validity in our decisions around admissions, promotion, and remediation in HPE.

Study Objective

To identify gender bias in PBA and determine whether or not such bias is implicit.

Methods

This randomized double-blinded trial will require participant raters to grade, using a modified Mini-CEX, trainee-performance on scripted clinical performance videos designed to elicit a gender bias in scores (i.e., numerical scores and narrative comments). Participants will be randomized to one of two sets of videos that are identical except for the gender of the trainee. Between-group analyses of scores and narrative comments will be performed to detect gender bias. Participants will also complete a Lexical Decision Task (LDT), to test for the presence of implicit gender bias which will be correlated to the results from the Mini-CEX. Data collection will be facilitated by an online platform.

Implications

Gender bias represents a threat to the validity and reliability of PBAs. This study will add to our understanding of how gender can influence a rater’s judgement and scoring of performance. This study will allow us to determine whether or not bias in assessment is associated with implicit gender bias, which may assist us in the design of future rater training interventions to mitigate such bias, and improve our ability to rely on PBAs.

Publication

Study is still ongoing.