Despite the date, this is not a Xmas newsletter (that particular seasonal art) or even a critique of Xmas newsletters – but season’s greetings to all anyway. This post is a look at written reports of a different kind – but often also summarising events in an overly positive way with the more average truths slipping through the cracks unremarked. Similarly, Xmas letters can become a bit impersonal and lose their impact. I guess it all depends on the purpose of the exercise.
A lot gets written (and presented) about how to give effective (verbal)feedback but there is much less space given to considering how to put this feedback into optimal written form that might serve a useful purpose. Only last week a teaching visitor suggested (after attending a workshop for Teaching Visitors) that she would appreciate some suggestions on writing a good report.
There’s not much point writing a literary masterpiece if nobody reads it and no point agonising over how to diplomatically state some unwelcome truths if it all disappears into the ether. There are several purposes and destinations of such reports. They may be primarily intended for the learner, or for the body that oversees training (as part of an in-training assessment framework or a way of satisfying accreditors). We write our Clinical Teaching visit reports assuming that the learner reads them and perhaps their supervisor skims them. The supervisor may note that we have observed the same things as they have during their observation sessions. We say such things as “would benefit from seeing more complex patients” and hope they do something about it. In the past I have known conscientious registrars to re-visit their reports prior to exams – but the report needs, therefore, to be worth re-visiting and contain content that informs the reader. A list of “strengths” and “weaknesses” may be vaguely helpful but it would be interesting to know what both learners and supervisors do with Likert type scores on multiple consultations skills.
It is possible of course that some organisations view the written report as legal evidence to justify later decision-making regarding competency, rather than a useful formative process. This 2011 report https://www.mja.com.au/journal/2011/195/7/review-prevocational-medical-trainee-assessment-new-south-wales noted that supervisors tended to routinely grade residents as at or above the expected level and that “As currently used by trainees and supervisors, the assessment forms may underreport trainee underperformance, do not discriminate strongly between different levels of performance of trainees …. and do not provide trainees with enough specific feedback to guide their professional development.” It questioned “What does this actually say about their developing competency? If a trainee does a core medical, surgical or emergency term in Term 1, performing “at expected level” indicates a lower level of performance than if the term was completed in Term 5. The phrases “at expected level” or “above expected level” do not indicate a specific level of competence”.
Although, as noted, there is little in the literature (compared to giving verbal feedback) there is an article on “Twelve tips for completing quality in-training evaluation reports” https://www.ncbi.nlm.nih.gov/pubmed/24986650 – although this is directed more at the end of term evaluations done by the ongoing supervisor rather than a one-off report done by a CT visitor. This article notes that the more recent literature emphasises the importance of qualitative assessments (as opposed to concern about the reliability of the assigned ratings) and the focus now is on improving the quality of the written, narrative comments. What you say is important. The article suggests completing the comments section prior to the ratings section in order to avoid the tendency to rate all components the same (eg all 4/5). It is important that the feedback form you use enables you to provide such feedback and also has meaningful anchors on the rating scale.
A study looking at the quality of written feedback noted consistent differences between trainee-trainer pairs in the nature of comments which suggested that feedback quality was determined not so much by the instrument as by the users. http://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-12-97
So how can our written feedback on observed consultations be most meaningful? I guess by this I mean that the learner hears what is said, is able to relate it to what they did and is able to recall it in order to improve future practice. The supervisor or educator should experience a document that enlightens them about the learner’s performance and progress in a way that informs future supervision / teaching / clinical experiences / remediation. In some instances it will contribute concisely to an overall program of evaluation. Here’s how this might be achieved.
- All the usual principles of effective feedback apply (non judgmental, based on observed behaviours etc) – see earlier posts.
- It is particularly important that it should be timely – if they don’t get the report for 6 weeks they will have forgotten the consultations. I know I have. The more complicated the process the more points there are at which this can fail. Unfortunately my Xmas newsletters are running late and are not very timely this year. Fortunately, in this context, this probably has no significant impact.
- The written report should reinforce what was said in person. No surprises.
- The written report speaks to the individual – it is not generic. It asks to be taken personally. (I am much more likely to read a Xmas newsletter that seems to know who I am). Filling in forms can be seen as “just a formality” but there is considerable engagement between learner and visitor during the GP Clinical Teaching Visit plus a relative lack of constraint because they aren’t the supervisor (with the attendant conflict of roles). However the conflict between feedback and assessment remains.
- Good feedback tries to be specific and behavioural such as “I like the way you listen to the patients at the start of every consultation – keep doing this.” or “remember to ensure that you advise the patient on what to expect and safety-net before the consult finishes” or “As we discussed, I noticed the patient had trouble understanding some of the technical terms you used eg hyponatraemia, globus and crepitations . I suggest practising the use of some lay terms when appropriate or providing explanation. Perhaps you could discuss this specifically with your supervisor in your next teaching session”. “
- However, feedback can also include some global encouraging comments such as “Dr Smith demonstrated many attributes of a good family doctor today” or “Dr Singh has a great manner with older patients.” (hopefully the specifics were discussed at the time).
- There is no need to cover everything in the narrative feedback – in fact a small number of concise points works best so stick to the most important. This will add weight to them and make them more memorable. (Just like long newsletters are unlikely to be read to the end). The message might be “If you work on anything in the next couple of months, it should be this….”
- It has been noted that effective feedback requires accurate self-assessment, reflection and insight on the part of the learner so it is a plus if the report can encourage this.
- The written report should suggest ways of improving, developing and exploring – with references and links that can be utilised at a future point in time. For instance, you might comment on their previous lack of experience and current lack of confidence in women’s health and suggest that the FPA course might be useful (and provide a link) or sitting in with Dr X in the practice who does a lot of women’s health.
- If ratings have been made at “below the expected level” then it would be useful to make specific comments about these areas and the expected improvements to be made. This requires being aware of the gap between performance and the appropriate standard which is the essence of feedback. (Perhaps this point can be left out of our Xmas newsletters!)
At the end of the process you will have conscientiously filled in the assessment form AND provided a couple of “take-home messages” that will be worth acting on and revisiting – for all concerned