A chair for all seasons

PROFILE: Dr. Brent Kvern

Dr. Brent Kvern keeps everything running as Chair of MCC’s Central Examination Committee

Dr. Brent Kvern is a runner in all four seasons, even the punishing Manitoba winter. And persistence like that has served him well over his many years of involvement as chair of the Central Examination Committee (CEC). Before taking the helm of the CEC in 2014, Dr. Kvern was a member, then Chair, of the Medicine Test Committee. He said this experience, as well as his continuing medical practice at the Family Medicine Centre of St. Boniface Hospital in Winnipeg, have helped him gain insight into the requirements for today’s physicians.

Exam nerve centre

The CEC oversees and governs the Medical Council of Canada Qualifying Examination (MCCQE) Part I and MCCQE Part II and is a committee of committees. MCCQE Part I questions are developed by seven test committees. Performance-assessing objective structured clinical examination (OSCE) scenarios are developed by one committee for the MCCQE Part II. The chairs of these committees get together twice a year, in June and December, as the CEC. While the test committees design specific questions or scenarios, “the CEC sets the expectations and ensures the areas that need to be worked on are followed through,” said Dr. Kvern. (The MCC Evaluating Examination, now being phased out, and the National Assessment Collaboration Examination are governed by other oversight committees.)

The CEC has several critical roles in setting and administering the examinations, resulting in a packed agenda.  “The meetings are two-and-a-half to three days long,” said Dr. Kvern. “There’s a lot of stuff that goes on.”

First, the CEC “ensures the blueprint or roadmap for the areas we want to sample and represent on the examination is adhered to,” said Dr. Kvern. “We recommended the blueprint to the MCC Council, which ratified it.”

The CEC also ensures that the various test committees are aligned and is responsible for the administration of the exams.

“One of the benefits of having all the chairs of the test committees around the table is that they understand the issues surrounding the exams, what and why we’re testing, and make sure their committee members are all aligned too.

We want to make sure the exam is delivered consistently from site to site.”

Dr. Brent Kvern,
Chair, Central Examination Committee

But sometimes things go wrong despite everyone’s best efforts: computers used in the exam crash, a fire alarm goes off, or a candidate becomes ill and has to leave. Candidates are encouraged to submit incident reports in such cases, and these are reviewed by the CEC. “We determine what should be done so that the response is fair for the candidate.”

Fair and valid

“MCC is one of the world leaders in having a fair and valid test of the candidate’s ability,” said Dr. Kvern. It ensures this several ways.

The cut score at which candidates pass is decided through standard-setting exercises. The CEC authorizes the process used in these exercises, said Dr. Kvern, and sets the passing standard for the exam after discussing the results of the standard setting exercises.

The CEC also looks at the “performance” of exams, which means how well they discriminate between candidates who meet the standard and those who don’t. Dr. Kvern pointed out that if everyone answers a question right — or everyone gets it wrong — “it may indicate that there is something wrong with that question.”

“You want to make sure that all of your questions are contributing in some meaningful and valid way to assessing each candidate’s ability.”

Candidates may not realize that some questions in any exam are being piloted to ensure these questions perform well. Although “our committees are well-trained in the methodology of writing good-quality questions, sometimes things slip by.” If a pilot question on an exam is not performing properly, it is not counted toward the score, said Dr. Kvern. Pilot OSCE stations are not counted towards the final mark of the MCCQE part II.

Changing with the times

The CEC’s meetings were traditionally timed to follow sessions of the MCCQE Part I and the MCCQE Part II. With the changes planned for these exams in 2018, and the MCCQE Part I to be delivered many more times over the year, Dr. Kvern said the CEC will need to find a way to review the sessions continuously.

But such challenges will not stop Dr. Kvern from setting an example for work/life balance, finding time to go to his cottage and sail on the lake in summer, and keep on running through the winter.

For more like this:

Setting the standard