The importance of evaluating effectiveness to inform decisions

Evaluation approaches, tools and methodologies are nothing new to the world of Business, but their relevance has never been more important. This is especially prevalent in an environment of reduced funding yet higher expectations from both politicians and the public of more impact and value for society.

The rationale behind evaluating effectiveness goes beyond just creating a “funding rationale for your project or service”, it is about answering those key questions within your organisation to help you make informed decisions.

What changes as a result of our activities?

Where do we have the biggest impact?

How do we allocate our scarce resources?

Although, hardly an exhaustive list of questions, it encapsulates what many organisations are constantly trying to decide … “how to most effectively allocate resources to achieve the biggest impact for our stakeholders”.

In seeking answers, there are a plethora of various tools and techniques that seek to inform your decisions through calculations, ratios and other mathematical formula’s. So to get your head round where to start and what may be of value to your organisation, I have outlined three of the methodologies that we value.

1. Cost-Benefit Analysis (CBA)

As one of the most recognisable methodologies, CBA is an approach commonly used to determine whether a project or investment decision is viable, as well as, utilised to compare the relative cost and benefits between various projects.

CBA is often favoured because it can often be done from your desk; it explicitly measures the relative cost and benefits; and it provides a figure for organisations to compare between options.

A few of the advantages:

• Can help organisations boil down a “go” / “no go” decision to a single measurable unit
• It considers benefits RELATIVE to cost
• It is comparable across different options

Some of the limitations:

• Difficulties in accurately assessing financial value for associated costs and benefits
• Providing accurate cost, namely, delivering a project, often has costs far beyond just a technology build

For an example in applying this methodology check out this guide and case study by the World Health Organisation.

2. Cost Effectiveness Analysis (CEA)

A common methodology used in healthcare, CEA can be used to help organisations directly contrast the relative impact of different projects against a specific outcomes or objective, such as, improved quality of life. CEA can help inform decisions on either reallocating resources or making the most of available resources to achieve a set outcome.

In essence, the methodology is about comparing relative costs, and can provide a clear way of assessing different options based on cost.

A few of the advantages:

• Help inform organisations through being able to directly compare options
• Focus on cost of delivering an outcome is extremely valuable in the current environment of reduced resources and funding
• No need to identify a monetary unit for effectiveness i.e. use of different measurement units, such as, Quality Adjusted Life Year (QALY) – a measure of disease burden including quality and quantity of life lived

Some of the limitations:

• Many different approaches to calculation i.e. different interpretations in placing a value on life
• Capturing the full scope of costs involved, such as, ‘in-kind’ contribution through time given up
• Different choices of measurement units can significantly impact the interpretation of the ratio

If you like to see a lot of detail behind the methodology and approach, take a look at the World Health Organisation Guide.

3. Social Return on Investment (SROI)

The most recently developed of the evaluation methodologies, Social Return on Investment is a stakeholder (i.e. all individuals and / or groups involved) driven approach that aims to identify, measure and account for value (i.e. the change for an individual or group) created by undertaking project.

The methodology encourages a new way of thinking about impact by using financial proxies, such as, a market value of a change, to monetise the social outcomes of a project. Most importantly, this approach is about getting out there and talking to people about what changes and the associated perceived value for those impacted, including the recipients of project, as well as the broader community.

A few of the advantages:

• Get the opportunity to listen to those who are directly impacted
• About more than just cash value
• About engaging and communicating the change to help inform decision-making

Some of the limitations:

• Not designed to compare across projects or interventions where different social outcomes are being achieved, due to different measurements of value
• It is a judgment driven process, meaning that often it is only as valuable as the rationale and assumptions that are set in the calculation
• Complexity associated with calculating the influence of other factors i.e. understanding what might have happened without the project (‘deadweight’) or the role of other activities happening outside the project (‘attribution’)

To learn more about the nuances of SROI, take a look at Social Value International.

In digesting the value of these approaches and whether they are even practical for your organisation, think first about the decisions you are trying to make, as well as what support you may need to help undertake the assessment.

One option to reflect on is to create your own evaluation toolkit, which is bringing together a group of approaches that can be used in different situations. Feel free to get in touch should you wish to discuss what this might look like for you.If your are interested in implementing any of these evaluation techniques in your organisation, get in touch and we can set up an informal coffee to talk through your needs. If you would rather learn a bit more about Spark Strategy then download our latest whitepaper on partnerships in the NFP space.