Recipient: 2015-2016 – Shiphra Ginsburg

There is growing appreciation of the value and importance of written comments in assessments of clinical competence. This coincides with a movement towards competency-based medical education, which may entail the collection of large numbers of evaluation forms based on frequent, low-stakes assessments. Our own research, and that of others, suggests that the content of comments, while important, may tell only part of the story; it seems that how comments are written may have a significant effect on how they are interpreted by others, including program directors, residents and other faculty. Our objective is to develop a deeper understanding of the nature and influence of comment writing styles used by faculty in internal medicine (IM) programs in Canada and the US. Specific questions include: Do faculty display consistent, individualistic styles when writing ITERcomments? What are these styles? Do they correlate with faculty factors (e.g., gender, rank, years of experience) and/or resident factors (e.g., PGY level, gender, or performance)? Are these findings locally specific or generalizable across programs? To answer these questions written comments from four IM programs, labelled by relevant faculty and resident variables, will be coded using three complementary approaches (involving language analysis software, linguistic frameworks and theoretically derived thematic codes). The resulting database will thus contain each faculty’s completed ITERs, rated on several theoretically relevant dimensions. Analyses will explore for differences within faculty, between faculty and between programs, and will involve Generalizability Theory, cluster analyses and multiple regressions. Results will include a taxonomy of writing styles and an understanding of how these styles correlate with faculty and resident variables. This research can critically inform faculty development programs, residents’ ability to learn from their assessments and committees and organizations responsible for judging the competence of trainees and physicians.