Rather than start with basic theories about assessment I thought I would discuss a relatively new approach, Entrustable Professional Activities (EPAs), about which I became quite excited when I first heard about it – at an AMEE conference in Lyon in 2012. One of the appeals was that they are Work- Based-Assessments (therefore closest to the top of the well-known Miller’s Pyramid) and try to address issues of competence. It seemed to me that they were eminently appropriate to general practice because they assess complex activities – MCQs are good for other things.
History of EPAs
Publications on EPAs go back to 2005 and presumably were being developed prior to that – in the Netherlands, then the US and Canada. Back in 2012 it appeared that they had been developed and used largely in the hospital context – beginning in obstetrics and paediatrics and some physician training. On my return from the conference I explored the literature and discovered that the RANZCP had already developed EPAs for psychiatry training and we met with one of the psychiatrists involved in their implementation. Whilst we were developing our model, a 2013 paper was published described the development of EPAs in a US family medicine residency and an AMEE workshop by Karen Schultz and Jane Griffiths reported on implementing them in Canadian GP training across a group of four universities. All these contexts are very different as are their implementations. Even their final number of EPAs varies greatly from sixteen to seventy two.
What are EPAs?
An EPA can be defined as a discrete unit of delegated work / professional task which is core to the particular discipline. EPAs are observable activities which can be judged. They are chosen on the basis of being important in the specialty and having implications for patient care. They relate to competencies but they involve global judgment rather than atomised checklists. So, “communication” is not an EPA because it is a specific skill/competency rather than a professional task. It is involved in many professional tasks. On the other hand “being able to correctly diagnose depression, assess the patient and initiate appropriate treatment” is a specific professional task, relevant to general practice, which requires several competencies and a collection of knowledge, attitudes and skills. The concept of trust is central to EPAs and the choice and wording of EPAs are important. This is a structured assessment by someone who knows the trainee in an ongoing way – in contrast to the snapshot of the Teaching Visit or MiniCEX.
The basis of the assessment / judgment
Competencies have always been a problem for GP training given the more complex skills required which can’t always be reduced to individual competencies – and some relevant competencies are difficult to measure. Just saying you are measuring a competency doesn’t make that the case. In a colloquial sense the EPA works at the intuitive level of “would you send your granny to this doctor” but it then attempts to articulate that intuitive judgement. It is analogous to clinical diagnostic judgments. The process relies on the expertise of supervisors and they can draw on various sources of data (observation, records, feedback etc). The involvement of supervisors is also why it is important not to overload them with irrelevant or superfluous EPAs.
The anchor for assessment relates to level of supervision and this fits in nicely with current RACGP standards. The question is “can this registrar be entrusted to perform this professional task at this particular level of supervision (defined)?” A specified number of EPAs can be assessed across training and can be blueprinted against curricula if wished. Given the breadth of GP skills, and the variation in training contexts, it does not make sense for them to be linked to milestones except perhaps for the most basic ones relevant to ubiquitous conditions or those with high significance for patient safety eg “the registrar can be trusted to manage the acute presentation of an unwell young child who presents with a fever.”
Why use EPAs
In hospital training it is more possible to observe all activities by trainees prior to entrustment. In general practice there is much unobserved interaction with patients from the outset. It is important to know if registrars are safe. The EPA process attempts to articulate what was unspoken, what supervisors already do and is intuitively appealing in the Australian supervisory situation. It happens cumulatively over time, not a one-off assessment. As an article by ten Cate notes, EPAs attempt to go beyond competencies and tick-boxes. General Practice is a discipline that requires integration of multiple competencies.
If you want to know what has been done in the Australian context…
Implementation has varied internationally. EPAs have been used as a basis for a new curriculum (a US example): they have been laboriously mapped to all competencies (US); they have been developed from and integrated with pre-existing “field notes” in Canada (Karen Schultz and Jane Griffiths kindly shared their experiences with us on Skype conversations. If you are going to AMEE they would be happy to chat). We chose a representative approach (we already have a curriculum) and decided specific EPAs for assessment should be chosen a) where the content and context suits the method; b) to represent the scope of practice; and c) with contribution from supervisors and educators. We came up with 11 EPAs which were intended to be part of an overall program of assessment – and I will mention Programmatic Assessment in a later post. They included guidelines for supervisors. Our process of consultation, workshops, implementation and evaluation was described at a 2014 GPET Research Workshop and presented at a 2015 AMEE conference https://amee.org/getattachment/conferences/AMEE-2015/AMEE-2015-App-Data/8G-Short-Communications.pdf
A summary can be seen on the poster presented at the 2014 GPET conference – see http://www.mededpurls.com/pdf/EPAposterAug14.pdf
If you want to read more about EPAs – There are a couple of readable articles by Olle ten Cate who introduced the concept
Trust, competence, and the supervisor’s role in postgraduate training. BMJ. 2006 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1592396/
Nuts and Bolts of Entrustable Professional Activities J Grad Med Educ.2013 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3613304/
The Australian psychiatry program can be found here for an interesting comparison. It is of course a longer training program. https://www.ranzcp.org/Pre-Fellowship/2012-Fellowship-Program/Assessment-overview/Entrustable-Professional-Activities.aspx
Maybe we will see these being implemented more broadly in the future.