Adaptability of MCC teams in a time of complexity: Experiences we learn and grow from

In February, Dr. Maureen Topps, Executive Director and CEO of the Medical Council of Canada (MCC), participated in the 2022 Perspectives Symposium event hosted by Touchstone Institute. This years’ topic was Perspectives on virtualization and Dr. Topps presented on the ever-changing environment experienced, the adaptability of MCC teams and the lessons learned from delivering high-stakes assessments during the pandemic.

The pandemic accelerated the use of virtual communication tools in health care and delivery models for high-stakes assessments. Prior to 2020, the MCC was already increasingly investigating and adopting virtualization, opting in 2008 for the technology-assisted delivery of the Medical Council of Canada Evaluating Examination (MCCEE) that was offered as a computer-based exam in Canada and internationally. At the onset of the pandemic in 2020, our stakeholders called for the MCC to rapidly adopt platforms such as remote proctoring and virtual processes to support exam delivery. Two years later, and with some successes and some challenges behind us, these exam delivery options have improved considerably.

As a self-regulated profession, medicine is granted a high level of trust from patients.
Exams and regulatory processes are key to maintaining this trust and ensuring safe care.

It is with that mandate in mind, and to assist candidates in pursuing their route to licensure, that our teams introduced a remotely-proctored option for moving the Medical Council of Canada Qualifying Examination (MCCQE) Part I forward and made an earnest effort to shift the MCCQE Part II to a virtual format. Our objective was and remains to deliver assessments that are objective, fair, equitable and standardized. At the MCC, real-time innovation was driven by the need to deliver the MCCQE Part I outside closed test centres. The cancellation of exams in March of 2020 afforded the MCC an opportunity to shift to a new delivery model. Today, half of the candidates choose remote proctoring to take the MCCQE Part I, and research shows that the format has no impact on performance. However, embracing new technology comes with challenges, and we cannot minimize the courage and persistence of candidates. Already dealing with the stress of the pandemic, they endured many technical challenges during the first months of the remote proctoring modality that sometimes even required them to be rescheduled.

Contemplating virtualization in Objective Structured Clinical Examinations, now and tomorrow

After attempts to reschedule the MCCQE Part II were thwarted, there was and continues to be the need for objective and standardized assessment of clinical skills as part of the licensing process. While most technical challenges that occurred in the early stages of remote proctoring for the computer-based MCCQE Part I were resolved, we faced a different reality with the clinical exams, which aimed to replicate an actual patient encounter. And despite being able to adapt our clinical performance assessment content to a virtual delivery, with over 2,000 candidates registered or waiting to take the exam, we quickly identified major issues in terms of scalability and the extent of technical and staffing support required. Examiners and standardized patients who had been accustomed to the in-person setting also required support during this rapid modality switch. Following many technical challenges with the virtualization, the MCC ultimately ceased the delivery of the MCCQE Part II altogether in June 2021.

While there is undoubtedly a place for virtual delivery of clinical performance assessments, it is critical to manage participants’ expectations as there is a high possibility that some troubleshooting will be required during their exam. One other consideration for the use of virtual tools for high-stakes assessments is our mandate to assess candidates fairly whether they are trained in Canada or outside of Canada. While virtual exams can be offered without requiring the disruptions of travel for in-person exams, varying levels of experience with virtualization could impact a candidate’s performance when being tested.

Lessons learned

As physicians are placed in a position of trust and accountability, the value of objective and independent assessment in service to patient safety remains essential. The repercussions of removing high-stakes assessments and standardized national exams would be considerable.

Assessment drives learning and informs medical schools’ curriculum and educational activities. The mode of delivery of an assessment can also promote change in the educational environment.

The MCC will continue to evolve its assessments, recognizing that they must be consistent with testing of skills in a familiar care delivery setting, and that emerging competencies should include the skills required for virtual care delivery.  As we and our partner organizations continue the development of technology for objective assessments, it is clear that those must be guided by how health care is delivered, easily understood and readily available.

With respect to the future of MCC assessments, a multidimensional approach including independent assessment of a candidate’s knowledge and clinical skills for the purpose of licensure is critical. As we embrace new technologies, we must be prepared to face hurdles and to provide support to candidates, staff and participants through the changes. Considering the potential for the inclusion that virtual delivery of high-stakes assessments offers, we must continue to identify emerging technologies that will meet the requirements of assessments as well as the needs of candidates and stakeholders in the near and more distant future.

To watch Dr. Topps’ presentation and the question period that followed, please click here.