Evaluating whether programmes deliver value for money can be challenging, particularly when it is difficult to establish what would have happened without it. Break-even analysis can be a helpful tool to provide insights in such cases.

As outlined in the HM Treasury Green Book, a value for money (VfM) evaluation is intended to determine the value of the impacts of a particular programme, and whether those impacts align with or exceed the overall costs to society that are a result of the implementation of the programme. In traditional applied economics, VfM evaluations typically rely on undertaking a cost benefit analysis (CBA), with its associated BCR (benefit-to-cost-ratio) to provide clear indication of the societal benefits relative to the overall cost of the programme. A key requirement for a cost benefit analysis is a clear counterfactual. That is, what would have happened in the absence of the programme. As such, a CBA approach is only viable when there is an accurate understanding of the additionality and attribution of the observed impacts to the programme. So, what can we do when no counterfactual exists?

One option is to apply a break‑even economic analysis. This is the economic analysis approach closest to a CBA that can still be conducted alongside a theory‑based understanding of impact. A theory‑based impact evaluation builds a logical model (or theory of change) that maps how programme activities are expected to lead to outcomes and impacts, using evidence and assumptions to trace causal pathways when experimental counterfactuals are unavailable.

Determining the break-even point

A break-even analysis assesses the level of impact that would have been needed for the benefits, converted into monetary values, to outweigh the cost of programme. In simple terms, rather than comparing costs to the programme’s impacts, break-even analysis asks what level of benefit would need to have occurred for the programme to be worthwhile. This can also be referred to as the break-even point. To be tractable, a break-even analysis can only consider a limited number of monetised benefits and should focus on those that are most important to the programme’s aims.

Once the break-even point is estimated, other strands of the evaluation (e.g., stakeholder engagement, impact data points, etc.) can be used to determine how viable that break-even point is. That is, whether the required level of impact for the programme to break-even is feasible.

For the break-even analysis it is possible that additional data will need to be collected to ensure that the programme costs are estimated comprehensively (e.g., including any time spent to set up / administer the programme). It is also possible that some additional research and/or application of existing research will be necessary to assess the value of certain key impacts (for which there isn’t an existing market price).

Dealing with Uncertainty

Uncertainty is unavoidable when evaluating programmes (even more so without a counterfactual). Rather than treating this uncertainty as a reason to suspend VfM evaluation, this can be explicitly incorporated into the break-even analysis.

By testing alternative assumptions, for example shorter versus longer persistence of impacts or higher programme costs, it is possible to identify the conditions under which the programme would fail to break even. This can be done in the form of stress-testing the assumptions and presenting different sensitivity range scenarios.

Nonetheless, in the presence of data limitations and lack of a counterfactual, caution should be applied in drawing firm VfM conclusions from a break-even analysis.

Insights from the Staying Close Pilot Evaluation

Staying Close is a programme developed by the Department for Education (DfE) that provides enhanced support for young people leaving residential care and other forms of placement as they transition to independence. The Centre for Homelessness Impact, Verian and Simetrica-Jacobs conducted a pilot evaluation of the programme.

For the Staying Close pilot evaluation, a clear counterfactual was not available, as such a theory-based impact evaluation was undertaken. Rather than treating the lack of counterfactual as a barrier to the VfM assessment, break-even analysis was used to frame the economic analysis differently: how many young people need to be impacted for the programme to represent value for money?

The break-even analysis focused on two key outcomes that could be linked to participant wellbeing – improved social connectedness and more suitable accommodation. Sensitivity analysis was undertaken to explore how the break-even point varied as the cost estimate increased, the time period varied and combinations of those outcomes were simultaneously achieved.

Overall, the break-even analysis showed that it was possible, under certain scenarios, for the generated benefits from the programme to at least meet the costs. But it highlighted that the outcomes would need to persist over time for the level of impact required to break-even to be reasonable.  The break-even analysis also provided recommendations for future data collection and evaluations.

What Break-Even Analysis Adds to Decision-Making

There are many benefits to undertaking break-even analysis as part of a theory-based evaluation:

  • Economic analysis remains possible without counterfactuals; break‑even analysis can be used to assess value for money when a traditional cost‑benefit analysis isn’t feasible.
  • Break‑even analysis centres on meaningful, key programme outcomes, ensuring the analysis prioritises the most important results.
  • Break‑even analysis can be integrated into various VfM frameworks (e.g., the NAO 4Es or Value for Investment framework).
  • Early break‑even analysis embeds learning into programme design, guiding data collection and evaluation to strengthen VfM assessments over time.

Overall, under the right conditions, break-even analysis can be a suitable tool to evaluate value for money of a programme.

You can read the full Stayibng Close Pilot Evaluation here.

Want to learn more about applying economic analysis approaches in value for money evaluations? Book an introduction with our Chief Economist, Ed Dallas, today.