As evaluators we are often asked – or suggest – preparing case studies as part of our evaluation methodologies. Our charity partners and their funders value the role of a case study to “tell the story” of a project or “bring the findings to life”. Most of us would acknowledge that quantitative data alone cannot always do justice to the impact of an intervention. Narrative can be a more engaging and compelling tool to communicate impact. As an evaluator with a strong qualitative bent, I have drafted many-a case study over the years and used case studies drafted by others (often practitioners) as part of my evaluation evidence.
There are lots of things that make a case study useful, or not. But today I’d like to share 4 questions that are top of my mind when I hear the words case study suggested as an evaluation tool.
- To celebrate or to learn?
A trap that I’ve seen many charity partners fall into is assuming that a case study used for evaluation (and therefore, learning) will automatically translate into a “good news story” for PR or social media. This is a dangerous assumption to make. There is nothing wrong with “good new stories” – in fact, can we have more please?! But it is important to be clear that drafting a case study for celebratory purposes is not the same as creating a case study for evaluation purposes. Some of the most useful case studies I have compiled have been when things haven’t gone well, things have gone wrong – this is when we can truly learn. When a practitioner provides me with a good news story as evaluation evidence I need to analyse it with care, aware of the inherent bias in the way the story has been told and presented. The sceptic in me (and some funders too) is aware that no journey is flawless – there are hurdles, obstacles and flaws in even the most impactful of services.
2. Which lens are we using?
In the world of charity evaluation, case studies are often assumed to be about a person and their journey through a service. However, sometimes taking a different lens can be more impactful. For example, is a peer group a more accurate way to tell the story of a service? Or is the story best told through the lens of an intervention, activity or project to put a spotlight on practice? Thinking about the audience and how the case study will be used can encourage us to think about alternative formats.
3. How will it be used?
Often case studies are drafted to work as “standalone” documents or snippets. However, I find it much more powerful when case studies are pooled to identify common themes, outcomes and challenges. This takes the case study from being a description of a service / activity and evolving it to be so much more powerful – a basis for insight and learning. Triangulated with other data sources – quant data, interview data, for example – the case study contributes to new depth and meaning.
4. Who owns the story?
If the case study is about a person, it is important that they “own” the narrative. Whether a case study is anonymised or not, I still like to ensure that person at the centre of the story has final “sign off” on the case study. This is important not just for accuracy but also for agency. It prevents the narrative being transactional – I have seen how sharing a story (and potentially having it published) can be empowering for people. Unfortunately, I have also seen how case studies have had the opposite effect where people feel “done to” by a research / evaluation process.
Our role as evaluators is to give people the best chance to tell their story in their own way – this might be based on a series of conversations or it might be through creative tasks. Often a combination of the two works best. Our role is to create spaces where we can truly listen, asking the right questions, in the right way.
This principal of ownership also applies to case studies which are about an organisation, project or intervention. We need to ask “who” owns the story. And often the most interesting case studies have more than one voice included.