Measuring impact like the Gates – redux

How do you measure impact?

Do the words evaluation and measurement strike fear into the heart of your staff and board members? Are they never mentioned, or if mentioned, always put onto the back burner?

What if instead of talking about evaluation and measurement,  your conversations on that topic were always framed as ones about learning?

Clearly cost and difficulty are factors in avoiding evaluation and measurement. But down deep, who wants to be judged, which is what evaluation feels like. It seems to be something done to us , rather than with us, like being graded on a test in school or having a performance evaluation at work.

But what if evaluation and measurement couldn’t be separated from a commitment to learning? A commitment that always seeks to do better for the people or community you serve? That puts measurement to active use to improve the quality of your programming?

You might want to take a look at  A Guide to Actionable Measurement, released in 2010 by the Bill and Melinda Gates Foundation.

Jodi Nelson, director of Strategy, Measurement, and Evaluation at the Gates Foundation, was interviewed in 2012 in Stanford Social Innovation Review about the report and the Foundation’s approach to evaluation. In that interview she said:

Inspired by the co-chairs’ preference that the foundation measure strategically and to inform action—think of the question “What would you do differently if you have that data?”—actionable measurement privileges purpose over methodology or evaluation design. Purpose is about how you intend to use the information you gain from measurement to do something differently or effect some change.

Their formula for “actionable measurement” is just brilliant in my book:

“Planned collection, analysis and synthesis of data

“+ time devoted to development of reflection and insight

“+ willingness and ability to change

“= Informed Decisions & Actions”

With the growing drumbeat to rate and rank nonprofit outcomes, it is refreshing to hear such an important funder talk about reflection, insight, adaptation, learning.

I’m also impressed that this foundation that has more money than any of my clients could ever dream of makes a point that they are judicious about what they measure as they can’t (and shouldn’t) try to measure everything.

Frameworks to guide measurement

In the Guide, The Gates Foundation outlined three frameworks to guide its approach to measurement:

1. 1. “Measurement should inform specific decisions and/or actions.

 

1. 1. “We do not measure everything, but we do strive to measure what matters most.

 

1. “The data we gather help us learn and adapt our initiatives and strategies.”

They also note that the initiatives they fund are one of many factors that lead to large community change. They call these  “strategy-level”  outcomes and impact. It is refreshing to hear such an important funder note that they are part of a system of change and not  the sole driver:

That should come as a relief to many smaller organizations whose funders are asking them for impossible measures. “Measurement at the strategy level will involve joint data collection and analysis.” Who’s funding that where you live?

My advice: download the guide, read and discuss it with your Board and staff. It may give you many ideas for how to approach the idea of evaluation and measurement in your organization.

Postscript:

It’s been a few years. Here’s the updated Evaluation Policy of the Bill and Melinda Gates Foundation

A few other reads:

Hope, dignity and quality of life are also valuable outcomes, even when measured in hours.

Are you benefiting from the extraordinary knowledge in your community?

Still waiting for your performance review? Why not take charge.