Why multi-source feedback (MSF)?
Commonly used to evaluate executives and professionals in areas other than medicine, multi-source feedback (MSF) involves looking at performance with a 360-degree perspective. Recognizing the importance of providing physicians with feedback about their professional behaviours, regulatory authorities, professional organizations and health systems began adopting MSF for physicians. MSF collects data from those interacting with a physician and provides feedback on the physician’s actual performance in practice.
Research shows that valid data can be generated with a statistically significant number of respondents, and that physicians will use the feedback to contemplate and implement meaningful improvements in their practice.
MSF is one of the better tools to provide feedback and guide performance in interpersonal behaviours such as communication, professionalism and collaboration. It is not intended to evaluate physicians’ medical expertise.
Research that monitors and improves the tool and process
Operating under the MCC 360 Governance Committee, the Research Working Group identifies, proposes and undertakes research to evaluate and inform improvements to the MCC 360 tool and process. MCC 360’s research agenda supports program improvements to tools and processes, informed by aggregate data. Studies to be undertaken by the MCC are outlined by the Research Working Group.
The MCC is gathering validity evidence to report on reliability of the MCC 360 results. It is interested in changes and improvements physicians make to their practices based on the feedback in their MCC 360 report, as well as what drives change within medical practice and what makes a physician receptive to feedback.
Physicians undergoing MCC 360 are reporting high levels of satisfaction, particularly with the patient feedback and the feedback facilitation process. The majority feel the experience promoted improvements to their practice.
Further quantitative and qualitative research evaluation of the program will be undertaken.
MCC 360 is recommended and intended to be used for formative purposes and is not meant to provide a pass/fail decision about a physician, or to rank-order physicians.
As this is intended to be a formative tool, reliability is insufficient for high-stakes decision-making. Most MCC 360 reports result in performance assessment scores that are skewed to the positive end of the spectrum because most physicians going through the process are good physicians. In addition, MCC 360 focuses specifically on three CanMEDS roles (communicator, collaborator, professional) and therefore focuses on a small component of a physician’s practice.
MCC 360 should be considered one assessment tool in a broader “performance assessment toolbox.” Administrators may choose to offer physicians additional data from other sources, such as prescribing data, patient volume or other practice data, alongside their MCC 360 report. The College of Physicians & Surgeons of Alberta, for example, engages physicians in MCC 360 as only one feedback component of their quality improvement initiatives.
For further reading and resources on multi-source feedback research, please consult our National Guidelines.