Impact Measurement & Management: One size does not fit all

Impact performance has increasingly become a powerful brand differentiator and a meaningful new dimension of overall performance for all types of investing. Intentional strategies around how to measure impact have become central to a successful impact-oriented approach to investing. Indeed, measuring and reporting impact is one of the most important ways for any mission-driven organization to hold itself accountable. Achieving increased impact requires a measurement approach that is aligned with the investment strategy and integrated into the investment process. As noted in the Global Impact Investing Network’s (GIIN) recent report, The State of Impact Measurement and Management Practice, “the old focus on building buy-in for Impact Measurement and Management (IMM), has evolved into a new focus on integrating IMM into all investment processes. Almost universally, investors report that IMM leads to real impact benefits.”

The Autodesk Foundation’s approach to Impact Measurement & Management

At the Autodesk Foundation, we strive to be intentional and proactive managing the impact we want to see in the world. We believe that the success of our grants and impact investments depends on our ability to use impact evidence to inform our investments and how we partner with our portfolio organizations. However, assessing the impact of the organizations in our portfolio and how we support their performance is complex. Particularly because, like many foundations, we support a diverse portfolio of grants and investments across multiple sectors and stages.

Our IMM approach focuses on creating shared value and actionable insights to help strengthen our understanding and produce sector specific learning. Our three main IMM goals are:

• Establish Accountability: 
  Staying focused on well-defined impact goals and outcomes enable both us and our portfolio organizations to remain accountable.
• Inform Decision Making: 
  Using impact data to make better decisions and look for opportunities where impact data and associated insights can help our portfolio organizations clarify their strategy or improve how they serve their stakeholders.
• Generate Evidence of Success: 
  Validating with evidence that a product, service, initiative, or organization is making a measurable social and environmental impact. This is an opportunity to expand reach and involve more funders, which is especially important given our focus on supporting early-stage organizations.

Utilizing unique impact evaluation approaches 

There are many different frameworks and methodologies to evaluate impact. This makes it critical for us to have complete clarity regarding what kind of impact assessment we need and why. This past year we used a combination of approaches — including theory of change, self-reported key performance indicators (KPIs), third-party auditing, predictive impact modeling, and lean data — to collect and track metrics like greenhouse gas emissions, work placements, product innovations, and individuals reached through our portfolio organizations. But the key to a successful evaluation is knowing when to employ a particular method and to what end.

In our effort to create a culture of accountability and shared value through the Autodesk Foundation’s IMM practices, we work collaboratively with our portfolio organizations to align on a measurement approach that works for both of us — a set of measurable goals linked to the organization’s mission as well as ours. We first evaluate whether a portfolio organization’s existing reporting methods will work, and if not, we discuss what kind of support the Autodesk Foundation can provide. Below are a few examples of the different impact evaluation approaches the Autodesk Foundation has used and how they support our IMM goals:

Establishing accountability: Theory of change driven KPIs 

The Autodesk Foundation’s Work & Prosperity portfolio includes investments in systems and solutions that help at-risk workers prosper in the era of automation in service of a more equitable future. In 2019, we worked with Deloitte’s Monitor Institute to map the workforce ecosystem and developed a framework for identifying the main opportunities for philanthropic investment that later translated into our theory of change. Theory of change is a methodology for planning, participation, and evaluation that is used to promote social change by defining long-term goals and then mapping backward to identify the necessary preconditions to achieve those goals.

Using the theory of change, we identified the specific impact metrics that are relevant to our investment thesis and portfolio organizations. Most of the impact metrics identified were business KPI’s that our portfolio organizations were already tracking and were able to report to us. The table below illustrates a sample of the identified metrics.

Collecting and aggregating these metrics helped us build an accountability mechanism and glean some insights. For example, now we can distinguish between organizations whose models offer fundamentally different approaches to scaling impact — low touch, highly scalable solutions that can reach millions; or high touch, low scale solutions that can deeply transform the lives of thousands.

Informing decisions: Learning from communities through lean data

The Autodesk Foundation’s Health & Resilience portfolio includes investments in technology-based solutions that improve resilience in low-resource communities most vulnerable to climate change, focusing on the built environment, agriculture, water, sanitation, and energy access.
 
A part of the decision making process is understanding the impact of our investments on the lives of people. As a foundation, we are a step removed from the actual customer and community, so our goal was to understand what was happening in the field. The Lean Data approach, popularized by Acumen, leverages the power of low-cost technology and lean experimentation principles to collect high quality data at a fraction of the time and cost of other methods. It is a quick and accessible way to hear directly from customers that provide value to the portfolio organization by identifying where their product or solution is doing very well and where it is lacking. Using the Lean Data method created shared value for us, as well as the organizations in this portfolio, giving them the information they needed to make better strategic as well as product and service improvements. For example, one organization learned how their innovations best meet the needs of enterprising farmers, and where improvements are needed.

Generating evidence of success: External third-party audits

The Autodesk Foundation’s Energy & Materials portfolio includes investments in early-stage technologies that have the potential to drastically reduce greenhouse gas emissions to accelerate a low-carbon future. The standardized metric related to low carbon technology that most organizations track is the amount of greenhouse gas emissions (GHG) reduced. Developed by the GIIN’s Iris+ — an impact measurement system developed by the GIIN — this metric tracks the amount of greenhouse gases emitted through the organization’s operations during the reporting period, reported in Number of Metric Tons of CO₂ Equivalent, or “CO₂e”.

However, it is not always easy to accurately measure reduced GHG emissions and rely solely on self-reported data. It is also unfair to put the burden of this measurement solely on the investee. This was an opportunity to get creative and develop shared value for us and our portfolio companies by bringing in third-party experts to review and audit the existing CO2e reporting and calculation methods of individual portfolio companies. The project included reviewing the methodology, assessing its robustness and consistency across the portfolio, structure and documentation, and sources used for assumptions. The partnership added rigor, provided validation, and helped build our investees’ IMM capacity for this work. It is an example of how proving the measurable environmental impact of a product, service, initiative, or organization can help spread the word and spark further interest in these initiatives.

Challenges and opportunities

The challenges we faced throughout the process of developing the Autodesk Foundation’s impact measurement framework include:

• Collecting outcome related impact metrics, especially if solutions are upstream in the value chain. For example, organizations that are working on solutions in career support and navigation often don’t have enough information regarding the outcome of their services i.e., job placements, retention, income gains etc. They are limited to reporting beyond the numbers of learners or workers that availed their service.
• Connecting impact metrics to decision making. There is a need to dive deeper into understanding how the portfolio organization is using this data to inform their strategic decision making as well as how we are going to use this data.
• Continuing to refine the process of aggregating diverse data sets. Bringing nuance to reported metrics is where we often find the most interesting insights, but we also aim to aggregate metrics across the portfolio. With the diversity of the portfolio, relevant metrics vary, as do timelines to impact along with organizational capacity. We continue to refine the right balance between standardization and salience.

While we have learned that there are no easy workarounds, we continue to work with our portfolio organizations to understand how we can support outcome related data collection practices. The three questions that we continue to ask ourselves as we determine the appropriate frameworks, methodologies, and valuation approaches are:

• How can we create shared value for all stakeholders? Will the result of the impact reporting be useful to both the portfolio organization and to us?
• What lessons and actionable insight are we generating from this work? What will we learn about the sectors in which we work from the impact metrics we collect?
• How can we prioritize forward looking impact reporting through our approach to IMM? Will this impact reporting be predictive of future impact? Read more read more about future-looking metrics for emissions here.

We continue to explore ways to incorporate a systemic view of our impact into our IMM practices, using a variety of measurement tools has enabled us to cover many different impact categories, geographies, and sectors. Our goal is to get closer to impact accountability, measuring what really matters and allocating resources to maximize benefits to people and the planet. This is an evolving and complex area of work, and the Autodesk Foundation looks forward to sharing more about its approach as it evolves. We value hearing the perspectives and insights of the broader impact community and invite you to join the conversation and share your expertise by reaching out to us.