Value for Money Methodologies: Assessing Value for Money (VfM) using Outcome Harvesting

Value for Money (VfM) analysis enables organizations to assess whether their investment is worthwhile and identify ways to further maximise it. There are several Value for Money methodologies that can be used for these assessments. In this blog, we explore how we use and have adapted Outcome Harvesting (OH), a participatory and flexible evaluation methodology that helps organizations to capture and analyze the outcomes of their interventions.

 

What is VfM?

The UK Foreign and Commonwealth and Development Office (FCDO) – previously the Department for International Development (DFID) – defines Value for Money (VfM) as “maximising the impact of each pound spent to improve poor people’s lives” (DFID) It involves assessing the value that is generated by an intervention in relation to the cost of implementing it.

One of the most common frameworks used for VfM analysis is the 4Es approach, which focuses on demonstrating how a programme or organisation addresses the 4Es: Economy, Efficiency, Effectiveness, and Equity (and in some cases Cost-Effectiveness). To be VfM a programme has to deliver an optimum balance of the 4Es. To do so, programmes usually identify a series of VfM indicators or criteria to track in order to demonstrate how the Es have been addressed, using, in some cases, some monetary valuation techniques such as Cost-Benefit Analysis (CBA) or Social Returns of Investment (SROI). While these methods have their merits, they can also be limiting as they tend to focus on preconceived indicators and may not capture the full scope of an intervention’s impact and value.

Linked to this, it is worth bearing in mind that one of the key initial challenges with VfM is that it can reflect a simplistic and linear view of how change happens and can assume that positive changes desired in international development can be planned, controlled, and measured easily. Over time VfM approaches have been adapted by international development organizations to reflect better their values and objectives, including measuring VfM through a social impact lens. This means that the impact of an intervention is evaluated not only in terms of financial costs and benefits but also in terms of social benefits/costs.

Learning and Change (L&C) has been striving to consider and propose to its clients an alternative approach that offers a participatory and holistic method to capture unexpected and unanticipated outcomes, promoting a more comprehensive evaluation of the investment.

 

What is Outcome Harvesting?

With this in mind, L&C has been testing the use and adaptation of the Outcome Harvesting (OH) methodology to assess VfM. OH is a more participatory and adaptable approach to evaluation. It involves working collaboratively with stakeholders to identify and document the changes resulting from an intervention. This approach focuses on capturing both the qualitative and quantitative changes resulting from an intervention, rather than relying solely on pre-established indicators. The approach is particularly useful when the outcomes of an intervention are unpredictable and difficult to measure using traditional approaches.

OH is a monitoring and evaluation methodology used to identify, describe, verify, and analyse the changes brought about through a development intervention. It is designed to collect evidence of change, and then work backwards to assess the contribution to that change. In the context of outcome harvesting, an outcome is defined as “a change in the behaviour, relationships, actions, activities, policies, or practices of an individual, group, community, organisation, or institution” (Outcome Harvesting).

At the heart of the OH methodology is a very unique definition of outcome: Who has an actor influenced to change what, and when and where was it changed? What is the observable, verifiable change that can be seen in the individual, group, community, organization, or institution? What is being done differently that is significant? Based on the change, the harvester works backward to determine the contribution: How did the actor contribute to this change? Concretely, what did she, he, or they do that influenced the change?

Outcome harvesting consists of six clear steps: (i) design the harvest; (ii) Review documentation and draft outcomes; (iii) Engage with informants; (iv) Substantiation; (v) Analyse and interpret; (vi) Support the use of findings. The steps are not always distinct, and feedback from one or more steps can cause a return to earlier steps. Source: https://outcomeharvesting.net/outcome-harvesting-one-pager/

 

What do OH and VfM have in common?

Our idea to use OH to assess VfM originated from the fact that the two have many elements in common.

Firstly, OH, centers on identifying and harvesting outcomes, this means capturing what difference a programme has made by looking at what actors are doing differently. In VfM analysis this is what we call value, i.e. the difference that a programme is making in the life of the people it is targeting.

Secondly, in OH we analyse the significance of each outcome to help contextualise why it may be considered important or relevant. The significance also helps the VfM analysis as it enables to define the extent of the value, for example, whether it can be considered high, moderate, or low given the context and the programme characteristics.

Finally, OH also collects information on the contribution that has enabled an outcome to be achieved. These are usually activities or interventions implemented that helped the change to take place. While this is not usually costed, in VfM analysis we can see the contribution as the information that leads us to the investment, allowing us to relate the outcome (i.e. the value) to the contribution (i.e. the investment).

 

Assessing the VfM of the Humanitarian Innovation Fund (HIF) using OH

These common areas enabled us to test the use of OH to undertake VfM analysis. One of these pilots was done as part of the evaluation led by The Research People of the Humanitarian Innovation Fund (HIF) managed by Elhra. The evaluation is expected to be published on the Elhra website in the coming weeks.

The VfM assessment within this evaluation was designed to build on the data and evidence collected through the OH methodology to answer the following questions:

● What is the value of the HIF? In other words, what is changing as a result of the HIF’s work, and what is not changing as much?

● What is the value of the HIF in relation to the resources invested? In other words, which areas are worth the investment?

● What are the recommendations that emerge based on VfM considerations? In other words, what does the HIF need to do differently in the future?

The value analysis was built on the findings of the outcome harvesting exercises. The value generated was deducted by looking at the difference that the programme made according to different stakeholders, including the innovators supported by the HIF, the end-users, and other humanitarian actors engaged in the innovation ecosystem. The value of the different focus areas of the HIF was scored according to a predefined set of criteria, aligned with the HIF’s understanding of VfM and its Theory of Change.

At the same time, the financial analysis focussed on clustering the financial expenditure against the focus areas of work used for the analysis. The aim of this was to have an overview of the investment that had gone into the HIF and particularly how the resources had been allocated across the different areas of work.

 

Concluding remarks

Using OH to assess VfM offers several benefits, including a clear understanding of the value generated and its extent, identification of unintended consequences, a less siloed approach than traditional VfM approaches (4Es), and the ability to generate learning on cross-cutting areas.

OH can capture unexpected or unanticipated outcomes, promoting a more comprehensive evaluation of the investment. Its participatory approach involving stakeholders can provide a more holistic view of the investment and generate insights into areas that may have been overlooked in traditional evaluations, which can inform strategic decision-making and promote learning and change within the organization.

To learn more about how Learning and Change can help your Organization create a tailored VfM framework through the outcome harvesting approach visit us here: www.learningandchange.org.

 

Learn more about our work on VfM here.

 

Get in touch to improve your impact and make a difference in the lives of the people you work with.