Warning: call_user_func() expects parameter 1 to be a valid callback, class 'ET_Builder_Module_Helper_Style_Processor' does not have a method 'process_extended_icon' in /home/customer/www/esoprinciples.asia/public_html/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 20048
IMPES
Principle Three
Measure immediate, intermediate, and long-term outcomes
IMPES
Principle three
measure immediate, intermediate, and long-term outcomes
As well as immediate outputs, it is important to define what the long-term success of your work looks like and how it could be measured.
‘Outcomes’ are the knowledge gained and results achieved from your work (often measurable) and may not materialise for months or even years. They should be aligned with your organisational vision, mission, and Theory of Change.
The measurement of ‘outputs’ often means monitoring the more immediate effects of your programming and overall work, for example the number of entrepreneurs that graduate from your programs. However, on top of considering these immediate numbers, also examine what longer-term results your work may have and how these can be measured. This is how ESOs can move from measuring activities and outputs to understanding the true long-term impact they can have on the communities, country, and broader ecosystems they are operating within.
Intermediate outcomes (typically 6-12 months, however this can change for different organisations) are important for entrepreneurs, ESOs, and funders alike. Earlier results can keep entrepreneurs engaged and motivated throughout the program as they start to see the positive changes the program has had. Funders often need to see demonstrated results of funding fairly quickly.
Key stakeholders (including funders) feel more confident in an ESO when they take a long-term outlook towards outcomes and impact. A strong ESO has a long-term vision and a plan in place to measure the implementation and progress towards that vision.
As well as immediate outputs, it is important to define what the long-term success of your work looks like and how it could be measured.
‘Outcomes’ are the knowledge gained and results achieved from your work (often measurable) and may not materialise for months or even years. They should be aligned with your organisational vision, mission, and Theory of Change.
The measurement of ‘outputs’ often means monitoring the more immediate effects of your programming and overall work, for example the number of entrepreneurs that graduate from your programs. However, on top of considering these immediate numbers, also examine what longer-term results your work may have and how these can be measured. This is how ESOs can move from measuring activities and outputs to understanding the true long-term impact they can have on the communities, country, and broader ecosystems they are operating within.
Intermediate outcomes (typically 6-12 months, however this can change for different organisations) are important for entrepreneurs, ESOs, and funders alike. Earlier results can keep entrepreneurs engaged and motivated throughout the program as they start to see the positive changes the program has had. Funders often need to see demonstrated results of funding fairly quickly.
Key stakeholders (including funders) feel more confident in an ESO when they take a long-term outlook towards outcomes and impact. A strong ESO has a long-term vision and a plan in place to measure the implementation and progress towards that vision.
3.1 Start with a Theory of Change and a Results Framework
If you do not know where you are heading, then you cannot track your progress towards long-term outcomes and goals. Prioritise developing a clear program or project Theory of Change as soon as possible, and refer back to this as often as possible within your daily activities and programs.
Developing a Results Framework will help to clearly plan and track progress towards your Theory of Change. This can take different forms: you can choose a Logical Framework, a Results Chain, or other commonly used tools for outcomes and program management.
3.2 Be specific and set clear indicators against your outcomes
Your intended outcomes should not be theoretical or so vague that they cannot be broken down and clearly measured. Against each outcome, ensure there is a set of clear ‘success indicators’: specific activities, examples or evidence that indicate whether that outcome has been achieved and that set specific targets. For example, if an intended long-term outcome is ‘business growth’, define what ‘growth’ means to your ESO, and identify a set of specific indicators which demonstrate this.
3.3 Be realistic about your outcomes and your ability to effectively measure them
Your ESO may have a big vision and goals, however when it comes to identifying program outcomes, make sure they are realistically achievable and that you have the capacity to measure them effectively. Consider the context that you and your entrepreneurs are operating within and what can realistically be achieved within that context.
For example, if you are operating within an emerging and largely informal economy with a relatively small percentage of enterprises formally registered, consider how realistic it is to aim to formalise a large percentage of the businesses you support. What other environmental factors play a role in business registration, and what are realistic targets that you can achieve or contribute to?
3.4 Strengthen alumni networks for long-term engagement
The long-term impact of an ESO can be determined by the impact that their alumni entrepreneurs make in the wider economy or communities in the months and years after they graduate. A strong alumni network leads to ongoing engagement with those entrepreneurs and is a chance to collect richer data to measure long-term outcomes and impact.
Remember that businesses start and fail. Entrepreneurs may go on to start new, or even multiple, enterprises. Tracking the progress of entrepreneurs as individuals over a longer period (such as several years) will give you a deeper insight into their journey.
3.5 Use mixed methods for outcome measurement
When it comes to measuring short-term, intermediate, and long-term outcomes and wider impact, there is no ‘one-size-fits-all’. Both qualitative and quantitative measurements are important and effective for analysing, measuring, and communicating your data. Quantitative measurements should tell the wider story of your work by using data from a larger number of entrepreneurs to demonstrate trends and progress towards success indicators. Qualitative measurements, such as storytelling, interviews, focus groups, and case studies, can provide deeper insights into the numbers and thus result in greater learning.
Tools & Resources
- Example Theory of Change for Accelerators
- Frontier Incubators: Building a Theory of Change
- 10 Steps to Building a Theory of Change
- Understanding Impact & Building a MEL Framework
- Building Your Measurement Framework: NPC’s 4 Pillar Approach
- ANDE: How do we measure social impact?
- Frontier Incubators: Facilitating a Theory of Change Workshop Guide
- Frontier Incubators: Impact Measurement 101
- Frontier Incubators: Revisiting your Theory of Change
- ISIP: Crash Course on Theory of Change
- Good Finance: The Outcomes Matrix
- Workshop Template: Running a Theory of Change Workshop with your team
- Defining your Outcomes: IMP’s 5 Dimensions of Impact
- Scaling Frontier Innovation: an overview of the Dimensions of Impact in relation to ESOs
- Defining Outcomes & Indicators
- GALI: Measuring the Value for Money of Acceleration
- Example Accelerator Metrics – calculating ‘Value for Money’
- Using Mixed Methods: NPC – Qualitative Data Collection Methods
- SSIR: A Playbook for Designing Social Impact Measurement
- J-PAL: A Practical Guide to Measuring Measuring Women’s and Girls’ Empowerment in Impact Evaluations
- More than Measurement: A Practitioners Journey to Impact Management
- For ESOs working with agriculture enterprises: Smallholder Performance Measurement: Common Indicators and Metrics
- ANDE: Measuring Value Created by Impact Incubators & Accelerators
- The World Bank: Handbook on Impact Evaluation
The principles in practice
Bringing your team on board and around an aligned Theory of Change and the impact you want to create together will help build a strong team of passionate people committed to achieving this change.
Print Me
Sample Program Logic Model
Expand
Involve your team in the process
Involve your team when developing a Theory of Change. They will have valuable insights and suggestions from different angles that you may not be able to see yourself. It can also strengthen your team culture by creating buy-in and a deeper understanding of what you are doing and, most importantly, why. Bringing your team on board and around an aligned Theory of Change and the impact you want to create together will help build a strong team of passionate people committed to achieving this change.
How can outcomes be measured?
A Program Logic Model or Results Chain can demonstrate how inputs (activities) can lead to outputs, which in turn can result in both immediate and longer-term outcomes (results). These outcomes should then be broken down into specific indicators of success per outcome so that you can clearly define what it means to achieve the intended results.
Sample Program Logic Model
Note: Logic models and Theories of Change all tend to use different terms (short, intermediate, long, impact, etc.) to define the lengths of time they cover, so just choose what makes most sense for your organisation. The length of time for intermediate and long-term outcomes may also change depending on your organisation and programming.
THE PRINCIPLES
A set of living, open source Guiding Principles for ESO Impact Measurement, led by a Community of Practice, and developed with input from key stakeholders.
Explore by Principle or start with
THE PRINCIPLES
A set of living, open source Guiding Principles for ESO Impact Measurement, led by a Community of Practice, and developed with input from key stakeholders.
Explore by Principle, or start with
1: Understand what success looks like for entrepreneurs
2: Measure the Health of your ESO
3: Measure immediate, intermediate and long-term outcomes
4. Understand and align with the goals of key stakeholders
5. Invest in Monitoring, Evaluation, & Learning (MEL)
6: Practise data collection methods that are accessible for diverse entrepreneurs
7: Validate what you measure
