Measuring outcomes and testing whether they can be attributed to your work is a key challenge in evaluation. For example how do you know that your service has made a difference to reoffending? How can services define and identify their own contribution or value alongside other services? How can you identify long term impact? How can you be sure that outcomes achieved would not have happened anyway? There is no easy answer to these questions but some of the things below may help.
Comparison group approaches
The strongest way to attribute impact is to compare outcomes for your service users to similar groups of people who do not receive the service (or receive something else). This enables us to estimate what is known as the "counterfactual". This guide introduces the different types of studies which take this approach and includes case studies of organisations that have used them. It will help you think about your own opportunities to conduct this type of research. The document is best suited to organisations that are already assessing their impact and want to explore more robust methods.
Theory of change
A theory of change is a description of how a service or project is intended to work. It is a widely used approach, increasingly required by funders ands should always be the first step in any evaluation process.
Working on a Theory of Change helps you to demonstrate outcomes because it guides you to the evidence you need to collect to assess whether a projects has made a difference. This guide introduces you to theories of change, provides examples and gives detailed advice on how to develop your own.
A key aspect of a Theory of Change is to have a good understanding of intermediate outcomes, which are the preliminary or short-term outcomes your service aims to achieve to support long term desistance from crime. To help you think about your own intermediate outcomes, the National Offender Management Service have outlined their understanding in their Commissioning Intentions and Needs and Evidence Tables documents.
Example theories of change
NPC have worked with a range of organisations to help them develop theories of change. The examples provided below cover a range of services from mentoring, to housing and family support. They also cover innovative practices that have been difficult to monitor and assess, especially when they are provided on a small scale. The HorseCourse theory of change case study is the most in-depth example of this, which includes a full literature review and a ‘Contribution Analysis’ of the data they have collected so far.
- Care Farm logic model
- Community Chaplaincy theory of change
- Criminon theory of change
- Employment project logic model
- Family therapy project logic model
- HorseCourse theory of change
- Housing project logic model
- Mentoring project logic model
- Music project logic model
- Prison Fellowship Sycamore Tree theory of change
- Rough Sleepers Assertive Outreach theory of change
- Sova theory of change
Ministry of Justice Data Lab
The Justice Data Lab has been established to help organisations working with offenders to access re-offending data for their service users alongside a matched control group.
Download our short guide to the Data Lab here; which includes background information and what you need to do to use it. This guide was updated in November 2015 to reflect changes made to the Justice Datalab service.
Download our frequently asked questions about the Data Lab here. The questions have been collated by Clinks with answers from the Ministry of Justice.
Clinks and NPC held a two part event on the 12th June covering first a practical overview of the JusticeDataLab and how it can be used; followed by a panel session discussing the value of the Justice Data Lab and what we've learnt one year on. Read the storify from the event
A new focus on measuring outcomes
This 2010 Clinks discussion paper highlights key issues affecting outcome measurement in the voluntary sector. It raises considerations for smaller organisations, and provides examples of organisations in the Criminal Justice System (CJS) that have been successful in monitoring their outcomes.