Would you accept an evaluation without a final report?

A community artist came to one of my evaluation courses a few years ago. She had previously been asked by a local authority (the funder) to evaluate one of her art projects. Having been given no other guidance about what to report, she presented the local authority with a crocheted rabbit and the shortest of written notes. For her, the rabbit was more representative of what the art project had achieved than any wordy report.

Crocheted rabbits might not suit every occasion, but are evaluation reports always required? Before any of my current clients start to worry, this is a thought experiment rather than a request. Evaluation is all about constructive challenge, and I wanted to challenge the accepted notion that a written report must always be the ultimate end product of what we do.

Back to reports. Let’s consider at what point and in what ways an evaluation delivers value. In evaluation training, I tend to promote a rule of thumb that divides an evaluation’s value into three parts:

  • The process of conducting the evaluation: the benefits of learning and improving through actively engaging in the evaluation process: reflecting on performance, discussing the challenges that have been faced, and talking through ways of making improvements
  • The report: a recorded summary of the evaluation’s findings, and a document that allows a wider range of people to benefit from the learning
  • Acting on the findings: what a project, programme or organisation does in response to the evaluation – implementing recommendations, for example, or improving their own ability to self-reflect and improve in the future

In practice, the balance between these three will vary from evaluation to evaluation, but experience tells me that the first and third can be of much greater practical value than the second. If I’ve done my job, you should know the findings before the report is written – there shouldn’t be any surprises. And a final report risks creating an evaluation ‘full stop’: a sense that the whole process is over, when the hard work of making changes based on findings may have only just begun.

So what questions should we – evaluators, clients, and commissioners alike – ask ourselves when considering the value of that momentous final evaluation report to ensure that it’s worthwhile:

  • Would the effort be better spent elsewhere in the evaluation? A huge amount of time and effort can go into writing a good quality report – more than we often admit to. Is that the right use of precious evaluation resource? Where is the most impact to be had?
  • Who is going to read it? Reports need an audience. If this is for decision-makers, what information precisely do they need to see?
  • Can we publish it? A report assumes that an evaluation has something important to say to those not directly involved in the evaluation process. Charities can be nervous about publishing independent evaluations, yet the sector as a whole could benefit from the learning; we might consider the value for money of a report in terms of its cost per reader – the more readers, the better. So get it out there.

As evaluators, we should always be looking for new and different ways of communicating our findings that suit purpose and audience. Just as we are adopting new ways of working because of covid, so we can be more innovative about our reporting. Maybe not always a crocheted rabbit, but it’s a useful reminder that there are many ways of getting a message across.