What's this thing for?
Differentiating Frameworks, Metrics, Standards and Norms
I’ve noticed that the term ‘‘framework” gets applied willy-nilly to anything that includes more than three variables, even by impact practitioners. Once someone says “we’ve got a framework”, they tend to get confused and defensive when you tell them it doesn’t do what they think it does. The SDGs—as valuable as they are!—are not a measurement framework. The SDG Impact Standards are not a reporting framework.
I’ve been known to get annoyed about this, but then I realised that I’d accidentally blundered into a useful insight. I’m deeply nerdy, and love a good interoperable framework. This is not representative of the professional population at large—which is for the best, really. How would anything get done if we were all gathered around whiteboards debating the impact of x angels dancing on the heads of y pins?
I think that’s helping rather than hindering the field. Some have argued that rather than ‘dumbing things down’, we should be asking analysts to raise their game. While I’d love for that to happen, I’m not waiting around. If we want IMM to be broadly utilised, it needs to be accessible to all.
The Impact Stack
I recently had the pleasure of collaborating with impact communication superstar John Treadgold on a practitioner report called The Impact Stack - A toolkit for impact measurement and management, in which we attempted to demystify the existing IMM toolset by helping people understand how and when to use different tools.
In order to do that, we needed to walk readers through a bit of history of how it came to be. Why is the GIIN or IMP’s name on so many things? Simply because in the early days this was all one conversation, with a lot of shuttling back and forth to other fields to find out what they were doing, and whether we could borrow their homework.
It’s not particularly constructive to say “that doesn’t do what you think it does”, I needed to be able to definitively say what it does do. Thus, I had to write a short paper of my own just to sort out the differences and allocate the tools in use to specific categories, on which this post is based.
What’s a framework?
Frameworks are a coherent way of structuring metrics and key performance indicators around the strategy, goals, and objectives of the investors.
A good framework helps an investor understand the relationship between the different performance metrics as well as the metrics themselves. This ensures that metrics are consistently collected and reported in the context of objectives, segmented in ways that generate specific, actionable insights.
There are many different frameworks available, and most investors use a combination of systems and frameworks to build their impact practice. This is because different frameworks are designed for different purposes and users, and often to answer different questions.
As with any tool, it’s a matter of understanding what kind of tool matches your needs – a screwdriver is not inherently better or worse than a hammer, they simply play different roles.
In broad strokes,
Normative frameworks establish shared norms and values. The SDGs represent shared norms about what problems are worth addressing, and how much change is required, while the IMP framework offers standardized guidelines for impact management.
Dimensional frameworks (e.g. 5 Dimensions) allow investors to describe the multiple characteristics of impact in ways that are structured, specific and comparable.
Categorical frameworks (e.g. ABC framework, Impact Classes, IRIS+ thematic taxonomy) allow investors and other stakeholders to group like things together so they can be compared.
Management and Disclosure Standards (e.g. Operating Principles for Impact Management, SDG Impact Standards) provide principles and guidance on how impact considerations should be put into practice within management and reporting.
Standard metrics (e.g. IRIS Catalog of Metrics) provide a common language and framework for impact measurement and reporting, and enables comparability across different investments and contexts. They can be used singularly or in defined sets such as the IRIS+ Core Metric Sets developed by stakeholders through the Navigating Impact Project.
Feel free to stop here with no hard feelings if you’re time poor and satisfied with the summary above. The remainder of this post is my manifesto synthesis of that work, in more depth than the format of The Impact Stack allowed.
Normative frameworks: Shared norms and language
The 17 UN Sustainable Development Goals (SDG) and their associated targets provide a vision and a normative framework for sustainable development that is based on the principles of universality, integration, and leaving no one behind.
A normative framework* refers to a set of guidelines or principles that prescribe desired behaviour, values, or outcomes in a given context. In the case of the SDGs, the framework provides a normative vision and agenda for sustainable development that is globally agreed upon and widely recognized.
The SDGs provide a common language and framework for measuring and monitoring progress towards sustainable development, and for identifying gaps and challenges that need to be addressed. The same framework can be used by countries, companies, investors and civil society to align their policies and actions with the global sustainable development agenda.
Aligning to the SDGs can help investors by mitigating risks, attracting capital, creating value, and meeting stakeholder expectations. By incorporating the SDGs into their investment strategies, investors can contribute to the achievement of the global sustainable development agenda while also pursuing their financial goals.
*my former colleagues in the Melbourne Uni Accounting Department debated this category, countering that ALL frameworks are normative by virtue of the fact that they make truth claims about which boxes things go in. I guess that makes this normative-normative?
Normative framework examples
Sustainable Development Goals (SDGs)
Year: 2015
Goal: developed to balance the needs and realities of rich and poor countries in the promotion of basic needs, economic development, and environmental sustainability (together, this is sustainable development).
Used by: many, despite being focused on country-level results.
Known for: 17 SDGs with global endorsement; high-level shared goals, supported by indicators of progress, and targets
Impact Management Project
This is a bit of a trick question, as this not really a framework, but is regularly cited as though it is one: “we use IMP”
Year: 2017
Goal: global consensus on how we measure, improve and disclose our positive and negative impacts (now called impact management)
Used by: Impact Practitioners
Targets and thresholds
An important, but often overlooked, normative feature of IMM is the impact threshold. As in any field, this is a performance thresholds that represents the minimum or desired level of performance (target), and provides a clear and objective way to evaluate whether performance is meeting expectations.
An impact threshold is a minimum level of impact that must be achieved in order to be considered meaningful or relevant. Without targets thresholds, investors may not contextualise the ambition and impact performance of their investments, which can lead to misalignment with impact goals and wasted resources.
Impact thresholds can also help to promote accountability and transparency in impact reporting. By setting clear standards for impact measurement, organizations can demonstrate their commitment to impact management and provide stakeholders with reliable and comparable data on their impact.
Targets and thresholds examples
Science Based Targets initiative (SBTi)
Year: 2015
Goal: to help companies to set emission reduction targets in line with climate science and the Paris Agreement
Used by: companies and organizations that are committed to taking ambitious climate action.
Known for: separate sector-specific methodologies, frameworks and requirements for different industries
UNRISD Sustainable Development Performance Indicators (SDPI)
Year: 2020
Goal: assess impacts or performance against norms and thresholds that indicate a target consistent with the notion of sustainable development
Used by: Businesses and other entities
Known for: integration of sustainability thresholds and norms (often ignored or neglected within IMM and reporting).
Dimensional frameworks: Understanding attributes
Dimensional frameworks are used to group together different coexisting aspects of a single thing, and apply measurement scales to those aspects.
In the same way that a person can simultaneously be tall, dark-haired and wearing pants, impact is multidimensional, and no single metric can provide sufficient information to understand or compare the impact of an investment.
Instead, impact investors have settled on a model of impact that includes five dimensions: what, who, how much, risk, and contribution.
The five dimensions are matched with 15 data categories that can be populated to give a more complete picture of the impact. For instance "how much?" includes understanding:
the scale of change (how many are affected)
the depth of change (the degree of change from a baseline), and
the duration of the change (short-term to long-term)
Categorical frameworks: Putting things in boxes
Categorical frameworks (also called taxonomies) are used to group similar things together into categories, so that they can be clustered and compared. These frameworks are often structured in hierarchies so that they can be used in more specific or more general ways depending on the question or conversation required.
For instance, Clean Energy, Energy Access, and Energy Efficiency are all impact themes related to Energy and SDG 7. They may complement one another in some cases, but they are different goals that require slightly different performance information to know whether they are successful. However, they have more in common with each other than they do to each other than with investments related to water and sanitation.
While related, the different themes also imply investing in different models of delivering impact. Clean energy is likely to involve generation, transmission, distribution, and storage of energy from renewable sources. Investors interested in energy efficiency, on the other hand, will look at energy-efficient materials, products, and equipment.
Alongside existing financial categories like asset classes, categorical frameworks are often used in impact investing, to group and talk about impact goals (ABC), investor contribution (impact classes), investment themes and strategic goals (IRIS+ Thematic Taxonomy, shown below).
Categorical framework examples
ABC impact classes
Year: 2017
Goal: to categorise goals and enable comparison across investments of different intent, context and investor contribution
Used by: private impact investors
Known for: being impact’s answer to asset classes
IRIS+ Thematic Taxonomy
Year: 2019
Goal: bevelop harmonized definitions of common terms and impact investing themes to enable communicating and comparing impact performance.
Used by: impact investors
Known for: harmonised impact categories, themes, and strategic goals, supported by core metric sets
EU taxonomy for sustainable activities
Year: 2020
Goal: create an EU-wide classification system with appropriate definitions for which economic activities can be considered environmentally sustainable
Used by: companies, investors and policymakers
Known for: first binding sustainable taxonomy
Global Impact Taxonomy (eg. Global Climate Landscape)
Year: 2022
Goal: open source classification framework for the global impact economy
Used by: governments, institutions, businesses, and investors
Known for: common structure and language for identifying, tracking and making sense of the breadth and depth of innovation; unaligned to existing industry classifications
Management and disclosure standards: Doing things well
A management and disclosure standard is a set of guidelines or rules that provide a framework for investors to manage and disclose information related to their operations, activities, and performance. These standards are often developed by international organizations, industry groups, or regulatory bodies, and they are intended to promote transparency, accountability, and ethical behavior.
Management and disclosure standards are more similar to ISO standards than they are to a tax code - they are about how you do the work rather than being external reporting or performance standards that dictate exactly what has to be reported. The Operating Principles for Impact Management and the SDG Impact Standards are the primary management and disclosure standards used by impact investors to demonstrate their credibility, promote best practice, and identify areas for improvement. Investors may also apply voluntary standards related to financial reporting, environmental sustainability, social responsibility, and governance practices.
Management and disclosure framework examples
Operating Principles for Impact Management
Year: 2019
Goal: to ensure that impact considerations are purposefully integrated throughout the investment life cycle
Used by: though built with development banks and development finance institutions in mind, also used by other investors
Known for: calling for independent verification and impact at exit
SDG Impact Standards
Year: 2020
Goal: addresses the management practice gap by providing decision-making framework to provide guidance on the IMM practices organizations should adopt
Used by: investors and enterprises
Known for: the first standards to be consistent across enterprises and investors
Measurement frameworks: Metrics and meaning
Metrics are an Important Part of a Measurement Framework
A standard metric is a widely accepted and commonly used measure that is used to evaluate performance or progress in a specific area. Most industries have them, such as the use of return on investment (ROI) in finance. In the impact space we have settled on particular measures that can be used to understand and assess the social and environmental impact of an investment, such as tonnes CO2-e emissions produced, megalitres of potable water delivered, or quality full-time jobs created.
Standard metrics provide a common language and framework for impact measurement and reporting, and enable comparability across different investments and contexts. Standard metrics help us select, monitor, and report the social and environmental impact of their investments in a more rigorous, transparent, and comparable way. This can enhance the effectiveness and credibility of impact investing as a tool for promoting positive social and environmental outcomes.
Indicator is a more general term than metric, but still a defined (and hopefully standardised) measure related to some aspect of performance, either of the organisation itself, its products/services, or its context. Indicators are a measure (metric, set of metrics or other evidence) that provides evidence of the degree to which a certain condition exists or certain results have been achieved. Sometimes the terms metrics and indicators are used interchangeably, but indicator should imply a clear link to what the metric is intended to measure in the real world.
Metrics are better in sets
I tell my students that if you were to have a combination altimeter/speedometer, you’d have a single number that could tell you that you were in grave danger, and not enough information to know what to correct (as illustrated by XKCD in very simple terms in the frame below).
Because it’s worth repeating: impact is multi-dimensional and no single metric can provide sufficient information to understand or compare the impact of an investment.
I’m a big fan of the IRIS+ Core metric sets, which I see as an important tool for standardizing impact measurement and reporting in the impact investing industry, and for promoting transparency, comparability, and accountability. In large part—not to discount the important and tireless works of my colleagues—this is because they are a reputable off-the-shelf solution that keeps people from doing dumb things like making up their own bespoke metrics (please don’t do this, or if you do, use them alongside standard metrics!)
IRIS+ Core metric sets offer a standardized set of metrics that can be used to measure and report the social and environmental performance of an investment. They also provide a common language and framework for impact measurement and reporting across different investment types, sectors, and geographies.
I’m obliged to say that published metric sets are a starting point rather than an end point—please customise and supplement them with additional indicators and measures to reflect the unique characteristics and goals of a particular investment. They should be seen as part of a broader approach to impact measurement that includes other elements, such as theories of change, impact pathways, data collection methods, and analysis techniques because they do not constitute a measurement framework in and of themselves. Rather, they can be used as a starting point or a reference for developing a more comprehensive and context-specific approach to impact measurement. The metrics can be customized and supplemented with additional indicators and measures to reflect the unique characteristics and goals of an investment.
IRIS Catalog of Metrics
Year: 2009
Goal: standardisation of social, environmental, and financial metrics in order to provide a common language that enables comparison and communication across sectors
Used by: impact investors. also used by others.
Known for: incorporated and built on existing sector-specific efforts; aligned with 50+ existing standards
IRIS+ Core Metric Sets
Year: 2019
Goal: increase data clarity and comparability to enable impact investors make informed decisions using a common language to describe goals, strategies and expectations; support the comparison and communication of impacts using core metrics, standard data categories and templates.
Used by: impact investors and enterprises
Known for: linked 5 dimensions to standard metrics; IMM templates; linked to evidence and based on best practices across the industry.
From metrics to measurement frameworks
While standard metrics are important for ensuring comparability and transparency in impact measurement, a measurement framework is necessary for developing a comprehensive and tailored approach to impact measurement that reflects the unique characteristics and goals of an investment. Metrics are what gets measured, while the framework is the overall approach to using metrics to evaluate performance in alignment with best evidence and in fulfillment of an investment strategy (one hopes!).
A measurement framework is a structured approach to selecting, organizing, and using metrics to evaluate performance or progress. It provides a systematic way to measure and evaluate performance, and it helps to ensure that the metrics used are relevant, reliable, and consistent. A measurement framework can help investors to identify the most relevant and meaningful indicators and measures, and to collect and analyze data in a way that is robust, consistent, and credible.
Closing thoughts
That’s a lot of content at once, and doesn’t even cover my favourite parts. If you want to know my thoughts on the wonders that can be worked by a Theory of Change, see this video, or contemplate the diagram below.
