Ask a TechnoServe Expert: Kate Scaife Diaz on Impact and Measurement

"Ask a TechnoServe Expert" is a series where our staff members, who work on a range of important global development issues, answer your questions. In this edition, Director of Impact Kate Scaife Diaz answered your questions about impact and measurement.

As an avid proponent of data-driven decision-making, Kate Scaife Diaz advises TechnoServe on best practices in data collection, analysis, reflection, and learning. She holds a Master’s of Law and Diplomacy from the Fletcher School at Tufts University and a Bachelor’s degree from the University of Chicago.

Editor’s note: The following questions were sourced from our audience on Facebook, Instagram, and LinkedIn.

What does impact mean at TechnoServe?

At the fundamental level, TechnoServe views impact as the monetary value added to our clients’ incomes. However, we know that impact means much more than that and represents different things for different people. Sustainability is a key component of impact – we want to make sure increased earnings endure long after a project closes. We are also careful to consider ways our impact is sustained through changes in the market system. We recognize additional income is not the end goal, but rather a tool that enables people to transform their lives. With additional income, the women and men we work with are able to send their children to school, receive better healthcare, and advocate for their community. These are the impacts we strive to achieve.

It is also important to mention the non-quantified impact of our work. The enterprising people we work with often experience higher self-esteem and an increased sense of agency and empowerment. When they earn additional income and are more included in a market, that success carries over to other aspects of their lives as well.

What development indicators does TechnoServe measure?

TechnoServe is committed to measuring four main indicators each year across our portfolio. Those indicators are financial benefits (the additional income that TechnoServe has helped our clients achieve), number of beneficiaries (the men and women for whom we have evidence of added income), the percentage of beneficiaries who are women, and finance mobilized (a measure of the additional financial investment we have helped generate). At the end of each project, we perform a calculation based on the total financial benefits compared to the total cost of the project to get the return on investment (ROI). TechnoServe’s projects achieved an average ROI of 3.3, or $3.30 in income gains for every $1 spent. For more information, see the “impact” section of our 2018 Annual Impact Report.

In addition to the four main indicators, each TechnoServe project has additional project-specific indicators, e.g., more indicators around women’s inclusion, behavior change, etc.

Additional income is not the end goal, but rather a tool that enables people to transform their lives.

How can development organizations best compare cost-effectiveness across a wide spectrum of projects and regions?

We need to be able to compare projects that are aiming to achieve similar things, and we need to measure them in similar ways. For example, it is useful to have at least one indicator that all projects are aiming to achieve, such as additional revenue for beneficiaries. A project can have its own measure of impact, but in order to compare, we need to standardize the measurement approaches. In addition, advanced efforts should take into consideration the differing economies each country has, and apply some sort of price equivalency like the purchasing power parity (PPP) index. One example of this is The Economist’s Big Mac index, which uses the cost of a Big Mac to measure PPP between countries.

Finally, organizations need to be mission-driven and prioritize what is most cost-effective that also aligns with their mission. The idea is not to drive all donor dollars into one sector, such as healthcare or education, but rather to use the cost-effectiveness measure to identify and improve on what works in your specific field.

Sustainability is a key component of impact – we want to make sure increased earnings endure long after a project closes.

How is TechnoServe using data to improve development solutions?

Committing to measuring our impact has helped us home in on our impact. We use data at a corporate level and at a project level. The big picture portfolio aggregation helps us know where we should be investing long term. The project level data collection has the ability to rapidly change our programs.

At the corporate level, we use data to determine which sectors are creating the most impact. For example, our work in coffee has been highly impactful across our portfolio. In East Africa and Latin America, we know we can generate significant value for smallholder coffee farmers.

Organizations need to be mission-driven and prioritize what is most cost-effective that also aligns with their mission.

At the project level, we use data on a smaller scale, but the impact when aggregated is incredible. Every day, our staff is collecting information about who is attending our trainings, what they’re learning, when they’re participating, etc. We learn what is effective and what isn’t, and use that information to make changes as necessary. For example, if we see we have a fall off in attendance in one area, we combine it with our local knowledge and realize we need a new strategy for the next month to better support the farmers we work with. In doing so, we are better able to generate higher impact for farmers and for the project as a whole.

Learn more about our commitment to measuring lasting impact.