Using self-reported data for impact measurement: How to use stakeholder surveys to improve impact performance
Standardised stakeholder surveys allow enterprises to measure performance across five impact dimensions. This report provides piloted survey questions and implementation guidance to help organisations collect self-reported data, manage impact risks, and improve social and environmental outcomes based on direct feedback from customers and employees.
Please login or join for free to read more.
OVERVIEW
Foreword by Omidyar Network
Jessica Kiessel of Omidyar Network highlights that while the potential for enterprises to measure social and environmental impact is widely understood, variation in measurement methodologies remains high. This report introduces standardised stakeholder surveys as a solution to collect rapid data across the five dimensions of impact identified by the Impact Management Project (IMP).
Executive summary
The IMP identified data categories necessary to understand performance against each impact dimension. In partnership with 60 Decibels, Keystone Accountability, and Social Value International, the project designed repeatable customer and employee surveys. Pilot testing with portfolio companies of Acumen and Bridges Fund Management revealed that surveys are essential for the ‘What’ dimension and provide valuable first impressions of impact across other dimensions. However, full understanding of ‘How Much’ and ‘Contribution’ often requires additional measurement approaches.
Introduction
The central purpose of impact management is to reduce negative impact and improve positive outcomes. Any enterprise seeking to achieve this must start by understanding its current effects on people and the planet. While many organisations use third-party proxies, the greatest accuracy is achieved by gathering new data directly from stakeholders. This ‘self-reported data’ is particularly valuable at the beginning of an impact measurement journey.
Designing a survey along the five dimensions of impact
To understand any impact, data is required across five dimensions: What, Who, How Much, Contribution, and Risk. The IMP reached global consensus that impact can be deconstructed into these areas. By collecting data for each dimension, enterprises and investors can assess impact in a consistent and comparable manner. The framework identifies 15 categories of data to enable this analysis, including outcome level, stakeholder characteristics, scale, and counterfactuals.
The survey questions
Two surveys were developed: one for customers and one for employees. The customer survey aims to understand the outcomes the product contributes to, the demographics of the underserved stakeholders, the degree of change (depth), the duration of the impact, and the enterprise’s contribution relative to what would have happened anyway. Questions were piloted using various mechanisms, including phone calls and online surveys, to ensure they were both specific and agnostic of sector or outcome.
Examples and guidance for using the customer survey questions
Guidance is provided for each dimension based on pilot lessons:
What: Open-text questions about sought improvements help organisations test their theory of change. For example, a healthcare enterprise found that improved fitness was the most common sought outcome (45%).
Who: Demographic data allows for segmentation to ensure the most underserved customers are reached. Pilot results from The Hub Pharmacy showed a 62% female customer base, informing their marketing strategy.
How Much: This dimension covers depth, duration, and rate. Depth questions perform well for giving a first impression of impact. Duration data is harder to attribute to the enterprise alone as more time passes.
Contribution: Questions about alternatives help determine if a company is addressing a market gap. Pilots showed that 52% of customers for one company could not easily find a good alternative.
Risk: Nine risk types guide assessment, including execution and endurance risk. Surveys assess these by asking if anything ‘bad’ happened or if customers intend to purchase again.
Guidance for using the employee survey questions
Employee surveys focus on quality employment drivers. While some areas like safety and income are best assessed through non-self-reported data, others like workplace relationships and skills development are well-suited to surveys. The report recommends using the Gallup 12-Question Quality Employment Survey to capture positive outcomes in these areas.
Conclusion
Feedback suggests that the standardised questions are highly actionable for management teams. Surveys should be executed iteratively, as they are a reliable method for uncovering both intended and unintended outcomes. For most feedback surveys, a representative sample of 200-300 respondents is considered ample for significance. Finally, stakeholder engagement should be viewed as an ongoing process rather than a one-time exercise.
Appendix 1: Using both self-reported and non-self-reported data
The report provides practical examples of combining data types. A gym effect table matches self-reported ‘improved fitness’ results with objective data like active member percentages (78%) and average weekly visits (1.3). Similarly, a weight-loss example combines self-reported quality-of-life rating improvements with objective BMI point reductions to provide a full understanding of impact.