Frequently asked questions for MCC 360

Most physicians tend to receive a good result. Highlighting those physicians that are in the bottom 10% is likely meaningless and could cause some unintended consequences such as emotional distress or focusing on issues that really are not problematic but fall below the “norm.”^1,2,3

To avoid this pitfall, MCC 360 lays out the data for the physician to look at their strengths and opportunities of improvement relative to themselves by rater group. In this way, we are hoping to emphasize the intended purpose of MCC 360: quality improvement.

1. Hill, J. J., Asprey, A., Richards, S. H., & Campbell J. L. (2012). Multisource feedback questionnaires in appraisal and for revalidation: a qualitative study in UK general practice. British Journal of General Practice, 62(598), 314–321.
2. DeNisi, A. S., & Kluger, A. N. (2000). Feedback effectiveness: Can 360-degree appraisals be improved? Academy of Management Executive, 14(1), 129–139.
3. Sargeant, J., Mann, K., Sinclair, D., van der Vleuten, C., & Metsemakers, J. (2007). Challenges in multisource feedback: intended and unintended outcomes. Medical Education, 41(6), 583–591. 

Physicians do not often directly observe each other’s performance in the medical expert role, so feedback about these behaviours may not be perceived as credible and is not subsequently used to inform practice improvement.¹

Medical expert is more reliably assessed and accepted using other tools (e.g., chart review, stimulated chart recall, prescribing patterns, outcome metrics, etc.). MCC 360 is addressing a pan-Canadian request to develop a multi-source feedback tool to focus on the roles of collaborator, communicator, and professional.

Although MCC 360 (and multi-source feedback in general) is not suitable to be used in isolation for summative assessment or quality assurance purposes, combining MCC 360 data about communicator, collaborator, and professional roles with data assessing the medical expert role measured by other tools is appropriate and could provide a more holistic assessment of a physician’s performance.

1. Sargeant, J., Mann, K., Sinclair, D., van der Vleuten, C., & Metsemakers, J. (2007). Challenges in multisource feedback: intended and unintended outcomes. Medical Education, 41(6), 583–591.

• A rigorous multipronged evidence-informed approach was used to develop and evaluate MCC 360.
• A pan-Canadian expert panel developed the surveys and report. The panel included representation from physicians, medical education researchers, medical regulatory authorities, and hospitals.
• The existing evidence on multi-source feedback and lessons learned from previous multi-source feedback tools were heavily consulted and used.
• Focus groups with physicians, nonphysician co-workers, and patients were held to gather feedback on the surveys and report.
• An extensive pilot was conducted that included the collection and evaluation of
  • MCC 360 data
  • post-MCC 360 evaluation surveys
  • focus groups with physicians and facilitators
  • facilitator reports of their discussion with physicians
  • physician action plans for quality improvement
  • six-month follow-up survey with physicians regarding implementation of their plans
• Results of our pilot indicate the following:
  • Most ratings and comments are quite positive.
  • Most comments elaborate and specify aspects of behaviours targeted by the rated items.
  • Each physician receives several narrative comments from colleagues, co-workers, and patients.
  • Physicians place great value on narrative comments and facilitation.
  • Physicians are using their MCC 360 data to create and enact plans for improvement.
  • Both physicians and facilitators make use of both quantitative and qualitative data in creating physician action plans.

Quantitative feedback alone is often interpreted as lacking meaning and specificity. The addition of narrative comments can play an important role in providing context to numerical ratings through the description of specific examples and contextual factors related to a behaviour. In addition, feedback recipients report being more satisfied with feedback that includes narrative and report greater tendency to use such feedback to make future improvements.^1,2,3

1. Brutus, S. (2009). Words versus numbers: A theoretical exploration of giving and receiving narrative comments in performance appraisal. Human Resource Management Review, 20(2), 144–157.
2. Overeem, K., Lombarts, M. J., Arah, O. A., Klazinga, N. S., Grol, R. P., Wollersheim, H. C. (2010). Three methods of multi-source feedback compared: A plea for narrative comments and coworkers’ perspectives. Medical Teacher, 32(2), 141–147.
3. Vivekananda-Schmidt, P., MacKillop, L., Crossley, J., & Wade, W. (2013). Do assessor comments on a multi-source feedback instrument provide learner-centred feedback? Medical Education, 47(11), 1080–1088.