Medical Council of Canada

Multi-year grant recipient (2009-2013): Kevin Eva

Multi-year grant recipient (2009-2013): Kevin Eva

Multi-year grant recipient (2009-2013): Kevin Eva

Towards practice-based continuing education protocols:

Using test-enhanced learning, practice reflection, and commitment-to-change statements to enhance physician self-assessment and practice change

Investigator(s)

Eva, K.W.,
Armason, H.,
Wakefield, J.,
Kinzie S.,
Elmsie, T.

Abstract

Introduction:

It has become the norm, worldwide and inter-professionally, for maintenance of competence programs to place the onus on the individual practitioner to self-assess his/her strengths and weaknesses and take appropriate action to ensure that (s)he remains fit to practise. Over the past three decades, however, it has become increasingly clear that self-assessments aimed at providing summary judgments of one’s ability are not always trustworthy, thereby requiring the professional community to determine how and when to assess practitioners through the revision of external feedback regarding the limits of an individual practitioner’s knowledge and skill. How to deliver such feedback in a feasible, valid, and impactful manner is an open question although it goes without saying that it is preferable for this sort of feedback to be delivered in a proactive, explicit, and safe environment rather than waiting for difficulties with patient encounters to generate sufficient concern to trigger action to remediate identified gaps in performance. The purpose of the research outlined in this proposal is to move towards a better understanding of the relative (and potentially additive) contributions that these various sources of feedback provide with respect to learning and practice change, while simultaneously studying practitioners’ perceptions of their value.

Methods:

In this study we will assess the relative impact of two strategies that have been developed to overcome the challenge of maintaining competence: Test-enhanced learning and communities of practice, the latter of which commonly include formally prompted practice reflection. Test-enhanced learning is a burgeoning area of research in psychology, the phrase referring to pedagogical benefits of testing learners on material to be learned. Rather than treating an assessment as a simple reflection of what one has learned, this literature suggests that tests can be used deliberately to improve learning as the act of being tested on material appears to make that material more memorable to the individual than the act of studying the same material multiple times. Communities of learning approaches to continuing medical education have similarly been developed. Members of these communities are expected to support one another, motivate one another to continue learning and enable the provision of both implicit and explicit feedback to help clarify ways in which individual members’ performance could be improved. To study the effectiveness of these interventions, better understand physicians’ reactions to them, and identify factors that modulate their effectiveness, we propose to conduct a three-year, four- phase study within the context of the Problem Based Small Group Learning communities of practice organized by the Foundation for Medical Practice Education. Phase 1 involves a needs assessment, Phase 2 focuses on materials development and psychometric testing, Phase 3 involves an experimental design, and Phase 4 is a follow-up phase focused on assessing the extent to which practitioners implemented a change in their practice and their perceptions of the extent to which the learning interventions led to those changes.

Significance:

This study will help us better understand how continuing education protocols can be implemented with maximal effectiveness and how the benefits of assessment can be implemented in this context with minimal disadvantage. At a more basic level the results will improve our understanding of how to conceptualize professional self-regulation.

 Publication(s)

Study ongoing.