Competency based medical education continues to be embraced throughout the world.

In CBME, Entrustable Professional Activities (EPAs) are part of an accepted framework for teaching and assessing a candidate’s competence in a clinical context.  EPAs classify and describe key competencies that a candidate must successfully integrate in care delivery before being considered a qualified, full-service physician.  Once a candidate is considered entrustable, the learner is ready to perform the activity without direct supervision.

While adopting competency based medical education and EPAs, Program Directors and faculty may need to reassess their evaluation tools as well as their faculty-candidate communication.

According to The College of Family Physicians Canada, successful evaluations in competency based medical education are:

Valid (indicative of competence in the domain of tasks for the discipline in question)

Reliable (consistent in the measurement of and distinction between competence and non-competence)

Cost-effective (in time, money, and attention)

Acceptable (pertinent, rigorous, and fair to both candidates and evaluators)

Positive in their effect on learning (learning directed toward validated competence rather than merely passing an exam)

An electronic evaluation tool in a competency based medical education context should encompass these criteria to optimize the candidate’s progress toward competence.  From experience with our clients, we have distilled a list of principles to consider when designing an effective electronic evaluation tool.

Evaluations Should Facilitate Immediate Faculty-Student Communication

We often hear Program Directors often asking these questions:  How do we help candidates and mentors communicate with each other?  How do we improve feedback?  How can we optimize the learning process?

Residents are eager for feedback.   A mobile electronic solution with the capacity for immediate, on-the-spot evaluations facilitates clear and detailed feedback, resulting in an accelerated learning curve.

While a care-giving activity is in progress, a candidate can submit a self-evaluation through a mobile device which can immediately be received and reviewed by a preceptor.  Differences between the candidate’s self-evaluation and the preceptor’s own evaluation can prompt subsequent meaningful questions and discussions.

A Good Assessment Tool Not Only Documents, It Guides

Field notes and evaluation tools exist, in part, to ensure that preceptors and residents achieve a program’s desired outcomes.  Unfortunately, Program Directors often have to hope or assume that, over the course of a training period, a combination of comments, feedback, and observations will achieve all of the required learning components.

Effective assessments require that both evaluators and candidates understand the objectives and criteria of competence.  An efficient evaluation tool can guide them toward those objectives and inform them of precisely how to adjust their performance to meet them.  The tool also should guide both parties to understand what constitutes clear and detailed feedback.  Any EPA-specific field note and evaluation solution should complement the educational process, indicating what competency means and to what degree it needs to be demonstrated.  Features of the tool, such as number scales and narrative descriptors, can achieve all of these objectives.

EPA-Specific Evaluations Should Highlight Learning Gaps

The first part of the electronic evaluation process is data input.  The second part is formulating a clear picture of the data’s meaning.

A comprehensive picture of competence is like a completed puzzle composed of structured EPA pieces.  Within the electronic evaluation platform, a student should be able to see whether she has understood all the pieces of the puzzle, which pieces are missing and to what degree.  A software tool with granular data management of the puzzle pieces can facilitate the student’s proficiency in all these areas.  When a resident appraises his EPA field notes, he should be able to find and organize his data quickly, and visually identify any gaps.  He should also be able to answer the questions:  “What still needs to be accomplished to master this particular EPA?” and “Where am I stalled in my progress?”

Residents should be able to easily access their data, filter it, and comprehend which activities they may want to have observed and receive feedback on.  Program Directors can use the data to evaluate the larger picture of how the resident is progressing across all sites and all EPAs.  Preceptors can see which residents they are about to work with and determine which gaps they should fill during their time together.

The EPA Tool Takes Into Consideration the Unique Experiences of Students

In a week, one resident may do surgery for four and a half clinical shifts and family medicine for a half day.  Another resident may do two mornings of surgery, two of internal medicine, two of family practice, and one afternoon elective in addictions.

Students’ learning experiences are diverse and opportunistic across different sites.  Using a well-designed data management experience, residents can plan in advance which EPAs they are most likely to put into practice, and where they are most likely to perform them.

The EPA Tool is Designed to Maintain the Integrity of Your Data

Guidance within the tool is important to ensure activities are not chosen either randomly or inappropriately.  People will use the data to make critical decisions.  The accuracy of the data is of prime importance.

One solution to this is a text-based educational function within the application. If a resident is unsure what the Collaborator role (for instance) means, she can quickly, accurately, and effortlessly, access an explanation of the meaning, and make an informed submission.

The EPA Tool Facilitates Faculty Adoption through Frictionless Design

A common concern of preceptors about any new tool or process is its ease of implementation.  The EPA tool features intuitive navigation requiring a minimal learning curve.   Mastering the operation of the tool can be achieved in a very short time.