Impact measurement. Theory of change. Impact evaluation. Program logic. Outcomes measurement. These terms are bandied around a lot in the for-purpose sector but can be confusing and even intimidating for people. In this article, we demystify measuring your impact and talk through how to embed an impact measurement framework. While measuring your impact may initially appear daunting, the key is to get stuck in and continually look for ways to enhance your approach. So, happy reading and good luck!
Impact measurement is how your organisation evidence and demonstrates the results of your work. It’s about going beyond tracking what you do (your activities) to monitoring and articulating the so what? (your outcomes and impact).
Another way to think of impact measurement is:
(a) “if we do X, then Y happens”; and
(b) how do we track and demonstrate the Y?
A social impact measurement example here is an organisation that provides information and education about healthy eating (this is their activities, or the ‘X’). They are seeking to increase awareness about the importance of diet and support behaviour change so people eat more fruit and vegetables (these are the outcomes they seek, or the ‘Y’).
They can track their activities fairly easily by monitoring things like:
- the number of healthy eating information resources they develop and provide
- the number and location of healthy eating education sessions they run
- the number of people who download their materials or attend their education sessions.
When it comes to evidencing the outcomes they achieve or contribute to, they could:
- provide people who attended the healthy eating classes with pre-and post-session surveys to see if there was a change in knowledge and attitudes
- contact education participants a few months after the session to see if their fruit and vegetable intake had increased
- look at the municipality, state, federal or other public data relating to rates of obesity, and average fruit and vegetable intake.
We wanted to get in early with a word of warning about the term ‘measurement’. Some people associate measurement with numbers. This is extremely limiting, especially given the complexity of social, economic, cultural and environmental change. In our opinion, effective and meaningful social impact measurement needs to include quantitative data and qualitative information. The latter includes people with lived experience sharing what’s changed for them, on-the-ground insights from frontline workers, and themes and trends observed by thought leaders.
At its heart, impact measurement helps your organisation understand whether it is driving positive change. This is vitally important for all for-purpose organisations. By better understanding your impact, you can make decisions about which activities to start, enhance, stop or change.
Other key benefits of impact measurement include:
|Enhanced decision-making on what to do and what not to do, based on what’s proven to be impactful. This can be strategic, operational and tactical.||Demonstrate to service users, communities, partners and stakeholders that you are committed to positive change and playing your part in making it a reality.|
|Inspire and motivate your team by articulating the positive change you are contributing to and evidencing your organisation’s successes and achievements.||Better engage with your existing funders and customers by clearly showing how their investment in your activities, products or services is driving positive outcomes.|
|Demonstrate your impact to attract new employees, volunteers and board directors to your organisation.||Attract new sources of funding by demonstrating and evidencing that your organisation is effective and impactful.|
|Shows what’s effective and what’s not (so) effective in your organisation’s activities. This can include stopping legacy activities that aren’t actually impactful.||Engage new and existing partners who are aligned to the impact you’re seeking.|
|Use social impact measurement to embed a culture that focuses on results and the so what? rather than activities and looking busy.||Strengthen your reputation as impactful, honest and effective through taking an evidence-based, transparent approach.|
At Spark, our approach to impact measurement includes two key elements: a theory of change and an impact measurement framework. We’ve explained both of these below.
Theory of change
A theory of change maps the logic underpinning your impact measurement. It defines the problems and challenges you exist to solve (the context), how you go about solving these challenges (your activities) and the resulting medium and long-term change (your outcomes and impact). While different people use different terms within their theory of change, it’s generally structured as context > activities > outcomes > impact. Some organisations also include ‘inputs’ between the context and activities sections and ‘outputs’ between the activities and outcomes sections.
We’ve provided additional detail on the theory of change and its components later in this article. If you’d like to deep dive into the theory of change, you can also download our free whitepaper about the theory of change here.
An impact measurement framework builds on your theory of change by identifying the data you will collect to understand, evaluate and demonstrate your impact. It consists of measures that your organisation will use to evaluate the tangible change that arises from your activities. Instead of an impact measurement framework, some people also refer to this as an evaluation framework.
There are a lot of myths and misconceptions about measuring your impact, which can hinder and hamper impact measurement efforts. We’ve summarised some key fallacies and why they’re not accurate in the table below.
|The misconceptions||The reality|
|You need to be a researcher or academic to do impact measurement well or at all.||With some training, dedicated time and support, it’s highly likely that there’s an employee in your organisation who can lead your impact measurement efforts.|
|Impact measurement is impossible at worst and hard at best.||While impact measurement does have its technical elements and can be complex, it’s also very doable. Getting stuck in, breaking it down into bite-sized chunks, and looking for ongoing opportunities to improve will all help you to measure your impact.|
|Social impact measurement needs to demonstrate economic or financial impact.||Understanding whether your initiative saves costs and/or is a good return on investment can be useful but it’s not the whole picture. Focusing only on the finances is limiting given the complexity of social, economic, cultural and environmental change. Bringing in qualitative information alongside quantitative data can be useful in moving beyond a purely monetary approach, while also adding credibility to your impact measurement effort.|
|You need a large team to develop and roll out an impact measurement framework.||While your theory of change and impact measurement framework ideally considers your organisation holistically, it doesn’t require a large team to implement. Having clearly defined roles and responsibilities is key here.|
|Measuring your impact takes too much time.||Impact measurement can take as much or as little time as you are able to give. You might start with identifying and collecting one outcome-level indicator, decide to take a really comprehensive approach or land somewhere in-between.|
We are often asked about the relationship between strategy and impact measurement. In our opinion, it’s a bit of a chicken and egg situation. Let us explain…
As per the cliché structure follows strategy, starting with strategy is generally a good move. This is about setting a clear purpose, direction and priorities; this helps you make decisions about where to focus your organisation’s time, efforts, funding and other resources.
From a strategy, an organisation conducts tasks and activities which lead to positive change. The organisation can then measure these positive impacts and changes.
Our clients are for-purpose organisations that exist to make an impact. A theory of change articulates their positive impact and unpacks the logic of how they create it. Social impact measurement then takes this a step further, identifying the data that will help evidence this logic. This makes the theory of change and impact measurement frameworks useful tools in informing, testing and implementing your strategy.
A theory of change and impact measurement framework can help explore and answer questions such as:
- What positive change are we trying to create? How does this look like 3, 5 and 10 years from now?
- Which activities, programs, services, and offerings are required to deliver on our impact? Which of these create the most change? What should we prioritise? What should we stop doing? Is there something we need to start doing to achieve our impact aspirations?
- What are the enablers, foundations and assumptions that connect our activities to our medium-term outcomes to our long-term impact? Can we influence these elements, either directly or indirectly? If so, which ones are the priority? – What does the data we collect tell us? What’s working and what’s working not?
This means that effective impact measurement can guide and inform your organisational priorities. That’s why impact measurement is vital to strategy. It’s the logic behind your strategy and can help set the focus of your organisation.
Moreover, be mindful that sometimes things don’t work in a linear direction where A follows B. Impact measurement is rarely straightforward. Bringing together different elements, ideas and tools can be really useful. In combination, these points fuel our belief that the stronger the relationship between your impact measurement and your strategic planning, the stronger your organisation becomes.
It may sound obvious but communicating your impact means (more) people know about your impact. Without dedicated and effective communications, it’s likely that only a handful of people will know about your contribution to positive change.
Strong internal and external communication is vital to great social impact measurement. It’s about articulating and sharing data, stories and other information about your outcomes and impact. Whether it’s a compelling impact measurement report, an entertaining video, a pithy social media post or a colourful infographic, reaching the right people at the right time in an engaging way is vital in the information-dense world we live in.
There are some important considerations to think about when it comes to communicating your outcomes and impact:
1) Identify key stakeholders
Firstly, your organisation needs to identify the stakeholders who should know about your impact. Find out who they are and what they care about. This could be employees; service users; community members; government agencies; volunteers; members; partners; donors or someone else entirely. Keep in mind that you need to tailor communications with each stakeholder group to ensure relevance and engagement. For example, government employees might have different agendas and communication preferences than your service users.
2) Understand the channels
Once you’ve identified the key stakeholders, consider which sources and channels they use to access information and engage with others. Ask yourself: how do they get their information? Is it through events or social media? Maybe its traditional media (newspapers, radio, magazines) or word-of-mouth through their networks? Next, think about channels your organisation is already using to engage with stakeholders such as your website, eNewsletter, internal meetings and notice boards, etc.
3) Consider the format and content
If you’ve prioritised stakeholders and channels, it’s a good time to think about the format and content of the information you’re sharing. This is about the length of the information, what information is shared, and how it is presented e.g., videos, infographics, short statements, long-form articles, audio recordings etc.
There are a few other things to remember regarding impact communications:
- Use accessible and easy-to-understand language in your impact measurement messaging wherever possible.
- Consider translating your written communications into accessible formats such as plain language or Easy English, to make them easier to read and understand for people with low English literacy. This may include members of CALD communities and people with communication difficulties.
- Consider how you’ll build in celebration and positivity into your impact measurement communications. It can be a great way to acknowledge and celebrate the hard work of your staff, board and volunteers.
- Actively seek feedback on your impact measurement communications to help refine your approach and increase effectiveness over time.
A theory of change is a logical model that articulates how your organisation creates a positive impact. As mentioned earlier in this article, outlines the context and need at hand; how your organisation responds to this context; and the resulting medium-term outcomes and long-term impact. A theory of change is an incredibly effective tool for understanding,
exploring and representing your work and impact measurement.
When facilitating a discussion on the theory of change, it can be helpful to start the conversation with the positive impact you want to create, then work backwards to the other components. The reasoning here is that impact is your ultimate goal towards which all the other components are directed.
While the core elements of a theory of change are context, activities, outcomes and impact, it can also include inputs and outputs. We have provided definitions of all six components of a theory of change below:
Below are some common pitfalls and strategies to avoid when developing a theory of change.
|Confusing short and medium-term outcomes with the long-term social impact your organisation creates.||Consider outcomes by asking what you want to have achieved in 1 year, 3 years or even 5 years. Turn your mind to impact by looking ahead 10 or 20 years, or even additional decades out.|
|Listing too many outcomes in your theory of change.||Keeping your theory of change simple and straightforward can help increase engagement and effectiveness. If you can, focus on the most important outcomes. Consider limiting yourself to six priority outcome statements.|
|Establishing too much of a negative context.||Try to take a balanced approach to the context, considering positives and strengths as well as barriers, challenges and problems.|
|Not unpacking the causal links (and associated assumptions and evidence) between the layers of your theory of change.||As well as mapping context, activities, outcomes and impact, it’s essential to look at which activities link to which outcomes. As well as doing this based on anecdotal information, take a look at what other evidence is out there to back up these causal links.|
As mentioned above, an impact measurement framework summarises the data you will collect to evidence your theory of change. An impact measurement framework takes the logic of your theory of change and looks for ways to evaluate, back up and (ideally!) prove it is happening on the ground. There are many tools and methods you can use within the framework. The information below provides a general summary rather than diving into the specifics.
Ultimately, an impact measurement framework is a tool to determine whether your activities lead to your desired outcomes. To understand this, you need to consider your activities, the outcomes you seek, whether those outcomes are changing for the better, and whether your activities are linked to those better outcomes.
As we mentioned earlier, the key with impact measurement is to go beyond measuring what you do (your activities) to measuring the so what? (your outcomes and impact). As you probably know all too well, it’s generally far easier for an organisation to identify and track activity measures as opposed to outcome measures.
To move beyond activity-level indicators to effective impact measurement, we consider the following elements and questions when developing an impact measurement framework:
|Element||Questions to ask yourself|
|Activity or outcome||What activity or outcome from our theory of change are we tracking?|
|Indicator||What information will we keep an eye on to know how we’re tracking against this activity or outcome?|
|Information type||Is this indicator quantitative or qualitative?|
|New or existing||Do we already track this indicator or is this a new indicator that we will start tracking?|
|Method||How will we collect this information (e.g. through a survey, our financial reports etc.)? Who are we collecting the information from (e.g. beneficiaries/service users, staff etc.)?|
|Person responsible||Who is responsible for collecting this information?|
|Frequency||How often will we collect this information (e.g. weekly, monthly, quarterly, annually)?|
We provide more detail on how to develop an impact measurement framework in our free whitepaper here but, in summary, we propose the following 10 steps:
Detailed below are a few key risks and considerations for organisations to keep in mind when developing an impact measurement framework:
|Going too hard too fast||While it’s great to be aspirational and try to track as much as possible, going from zero to 100 in your impact measurement can be overwhelming and add capacity pressure. It can be easier to start with a few measures across a few critical outcomes and build from that over time.|
|Too much of one type of data|| |
Some of us, individuals or organisations, lean towards the quantitative (numbers, data) while others tilt towards the qualitative (stories, words). We believe that both have an important role to play so if you feel your framework is.
skewed significantly one way, take a step back and consider how you could bring in something from the other camp.
Lack of clear roles,
responsibilities and accountabilities
|You ideally want to embed impact measurement across your whole organisation. Although, there’s a risk that without a clearly assigned team and individual responsibilities, it may not be actioned. The key here is ensuring that employees have received training and internal communications. In that way, they understand the value of impact measurement and their role within it. In addition, people need time and space (and sometimes money too) to undertake impact measurement.|
|Assuming or claiming attribution rather than contribution||Attribution is when you say this (often my) work led to that outcome. Contribution is when you recognise that you played a part in driving positive change, along with others. Generally, given social and environmental change is complex, it’s about demonstrating and recognising contribution rather than expecting or asserting attribution. We also have a blog on this very topic which you can access here.|
Your organisation has developed a theory of change, incorporated a social impact measurement framework and is communicating this to the outside world – job done, right? Unfortunately not! In order to make sure it is relevant and effective, it’s worthwhile reviewing and refreshing your approach at regular intervals.
Here are some prompts that your impact measurement may not be working as effectively as it could, or requires a review:
- If you’re not clear on your own impact and whether your work is effective.
- There’s been a change in the strategic direction or strategic priorities of your organisation.
- You’re consistently not successful in securing funding from government, philanthropy, corporate or other donors.
- It’s unclear to your stakeholders which outcomes and impact you play a part in.
- Your peers, partners, funders and competitors are using and benefitting from newer impact measurement approaches and technology solutions.
- A year or two has gone by since you last reviewed your theory of change and framework.
Keep in mind that this does not necessarily mean a full-scale review. It could be just a few tweaks to enhance your theory of change framework. In addition, it’s important to note that major disruptions such as global financial crises or pandemics like COVID-19 have significantly changed how we live and work, and what we prioritise. At times like this, it may be necessary to reassess your approach to social impact measurement to see if it is aligned with community needs and your organisation’s strategic direction.
Bringing it to life: An impact measurement example
We’ve worked with a number of not-for-profit and for-purpose organisations on their theory of change and impact measurement frameworks. It includes Youth Projects who focus on young people and are committed to breaking the cycle of disadvantage. Their theory of change and impact reports are publicly available. See below for a visual version of their theory of change, and you can find out more and download their impact report here.
Impact measurement is increasingly becoming an essential part of how for-purpose organisations operate. In conclusion, measuring your impact is about showcasing your organisation’s ability to drive positive change. It is by using powerful stories and credible evidence. Developing a theory of change and associated measurement framework is a way to unpack, evidence and demonstrate your impact and guide strategic decisions.
References and resources
- Nail your theory of change by taking a look at a short Spark blog or our more detailed theory of change handbook.
- Our impact measurement whitepaper provides a practical 10 step checklist, case studies and templates to help you evidence, evaluate and demonstrate your impact.
- The Centre for Social Impact provides a detailed report on Impact measurement with frameworks that are easy to understand: see here.
- SIMNA is a knowledge-sharing network for social impact measurement with lots of helpful resources.
- SOPACT provides a handy theory of change cheat sheet.
Comments and feedback
We’d love to hear more about your experience, questions and ideas when it comes to the theory of change and impact measurement. What’s worked well for you in measuring your impact? What hasn’t? Do you have questions or suggestions for us about impact measurement? Do you have any predictions for the future of impact measurement? If so, you can get in touch with us by phone, email or in-person at our Melbourne and Sydney offices. You can find our contact information here.