Why multi-source feedback (MSF)?
Commonly used to evaluate executives and professionals in areas other than medicine, multi-source feedback (MSF) involves looking at performance with a 360-degree perspective. Recognizing the importance of providing physicians with feedback about their professional behaviours, regulatory authorities, professional organizations and health systems began adopting MSF for physicians. MSF collects data from those interacting with a physician and provides feedback on the physician’s actual performance in practice.
Research shows that valid data can be generated with a sufficient number of respondents, and that physicians will use the feedback to contemplate and implement meaningful improvements in their practice.
MSF is one of the better tools to provide feedback and guide performance in interpersonal behaviours such as communication, professionalism and collaboration. It is not intended to evaluate physicians’ medical expertise.
Research that monitors and improves the tool and process
The MCC identifies and undertakes research to evaluate and inform improvements to the MCC 360 tool and process, with advice and oversight from the Research Advisory Committee and the Research and Development Committee. The research agenda supports program improvements to tools and processes, informed by aggregate data. In consultation with program groups, the MCC may undertake additional research topics.
The MCC is gathering validity evidence to report on reliability of the MCC 360 results. It is interested in changes and improvements physicians make to their practices based on the feedback in their MCC 360 report, as well as what drives change within medical practice and what makes a physician receptive to feedback.
Physicians undergoing MCC 360 are reporting high levels of satisfaction, particularly with the patient feedback and the feedback facilitation process. The majority feel the experience promoted improvements to their practice.
Read the Survey results post-MSF+ from the College of Physicians & Surgeons of Alberta (2019) and the following oral presentations:
Roy, M., Streefkerk, C. & Lockyer, J. (2018, April). From PAR to MCC 360: A revised national multi-source feedback (MSF) tool. Oral presentation at the Canadian Conference on Medical Education (CCME), Halifax, N.S.
Roy, M., Streefkerk, C., Lockyer, J. & Sargeant, J. (2018, August). MCC 360: A national multi-source feedback program in the world of CBME. Oral presentation at the Association for Medial Education in Europe (AMEE), Basel, Switzerland.
Roy, M., Streefkerk, C. & Touchie, C. (2018, October). MCC 360: A national MSF program for assessing the roles of communicator, collaborator, and professional. Oral presentation at the Coalition for Physician Enhancement, Washington, D.C.
Further quantitative and qualitative research evaluation of the program is underway:
• MCC 360 program evaluation: focus groups with facilitators and physicians to capture participant feedback; a quantitative look at rating data; and qualitative analysis of comment data, action plans and facilitation reports. It also includes a six month follow -up with physicians. This is a collaborative research project with the College of Physicians & Surgeons of Alberta.
• The MCC is collaborating with four universities, in English and French, to adapt the tool and processes to fit the undergraduate medical education context
• The MCC is adapting the tool to fit more challenging contexts such as episodic care, or cases where the patient is not able to provide feedback and where that feedback would be coming instead from a caregiver or a substitute decision-maker (such as pediatrics, dementia patients, or palliative care)
MCC 360 is recommended and intended to be used for formative purposes and is not meant to provide a pass/fail decision about a physician, or to rank-order physicians.
As this is intended to be a formative tool, reliability is insufficient for high-stakes decision-making. Most MCC 360 reports result in performance assessment scores that are skewed to the positive end of the spectrum because most physicians going through the process are good physicians. In addition, MCC 360 focuses specifically on three CanMEDS roles (communicator, collaborator, professional) and therefore focuses on a small component of a physician’s practice.
MCC 360 should be considered one assessment tool in a broader “performance assessment toolbox.” Administrators may choose to offer physicians additional data from other sources, such as prescribing data, patient volume or other practice data, alongside their MCC 360 report. The College of Physicians & Surgeons of Alberta, for example, engages physicians in MCC 360 as only one feedback component of their quality improvement initiatives.
For further reading and resources on multisource feedback research, please consult our National Guidelines.