Real change leads to real results

Randomized Controlled Trials and Quasi-Experimental Impact Evaluations

Randomized controlled trials, or RCTs, are often referred to as the gold standard in impact evaluation. In an RCT, evaluators randomly assign a set of community members to receive project assistance (known as the treatment group) or to receive no additional assistance (known as the control group). The evaluators then capture and statistically analyze data across the treatment and control groups to quantify the project’s impact. In projects that use quasi-experimental design, evaluators use different techniques to approximate a control group or counterfactual when true randomization is unfeasible.

We enthusiastically support any opportunity to evaluate our projects to this degree of rigor, while acknowledging the difficulty of running RCTs in development interventions. As we continue to seek opportunities to implement RCTs and quasi-experimental impact evaluations, we also encourage projects to implement creative and resourceful ways of measuring impact where measurement resources are limited.

Entrepreneurship

• STRYDE (2020, 3ie and IPA) – This RCT measured the effect of entrepreneurial and personal effectiveness training on Tanzanian youth’s incomes 2 years after the training. The results found significant increase in earnings for women, but not for men, with a break even point of 15 months. The research also showed significant impacts for both men and women across psychosocial wellbeing, market-relevant skills, intimate partner violence, and a host of other outcomes. 
• Impulsa Tu Empresa (2020, Miami University) – This mixed-method analysis found 13.2 percent in sales relative to the pre-acceleration period and found that access to resources is insufficient on its own, and secondary in importance to building entrepreneurial skill when promoting business acceleration.
• Business Women Connect (2018, World Bank) – This RCT measures the effect of business training and increased access to a mobile savings platform on Tanzanian women micro entrepreneurs’ ability to save. While the study found demonstrable impact on the wellbeing of entrepreneurs, it found no improvement in profit. 
• Women Mean Business (2017, IPA) – This RCT found that training and mentorship programs aimed to help SMEs develop business expansion strategies in Uganda resulted in an average increase in monthly revenues by 22 percent and take home income from business by 17 percent for women-led business 3 years after the intervention. Increased share of  take-home earnings indicate greater financial security for households of businesswomen. (draft report)

Agriculture

• Contract Farming R&D (2016, IDInsight) – The pilot program which was launched to assist farmers adopt improved agricultural technology had some very positive impacts. Assistance to integrate farmers into contract farming increased the proportion of farmers planting reliable hybrid maize by 40 percent. 
• Coffee Initiative II (2013, Laterite) –  Participation in the agronomy training program resulted in an increase in yields of up to 75.5 percent for farmers in 2011. Estimates at the cooperative level also show positive changes, with one cooperative experiencing an increase in yield as high as 148.1 percent after one year of training.
• Coffee Initiative II (2012, JPAL/IPA) – This RCT found that training in good agronomic practices resulted in farmers being more knowledgeable about best practices. Trained farmers were significantly more likely to have adopted good agronomic practices on their coffee plantations. 
• Coffee Initiative II (2016, TripleLine) – This ex-post evaluation found evidence that farmers continue to apply improved agronomic practices five years after the end of TechnoServe’s intervention. It found evidence of continued investment in coffee washing stations and sustained price premiums for coffee. (Read a summary of the findings here)
• Mejoramiento Agrícola Sostenible (MAS) (2017, ANED) – Examined an agronomic training program in coffee and bean value chains in Honduras using a quasi-experimental control group. It found that trained farmers had statistically significant improvements in farming practice that led to a 48% improvement in coffee yields over the control group.

Forthcoming Research

• Uganda Coffee Project (2023, IFPRI) – Measures the effect of training on smallholder coffee farmer yields. RCT
• HERZ Foundation Ethiopia Coffee Study – This examines the impact of agronomy training, particularly the adoption of stumping practices, on farmer income. It assesses data quality or self-reported survey data compared to direct measurement approaches. 
• MAS 2.0 (2023, IFPRI) – Measures the impact of a coffee market intervention on smallholder farmers’ revenues. RCT.
• Impulsa Tu Empresa 3.0 (Stanford) – This RCT evaluates the impact of TechnoServe’s mixed businesses acceleration approach as well as cost-effective digital delivery alternatives. 
• IKEA Study in India (IPA) – This RCT test assesses whether adopting a “regenerative business” lens into business acceleration can improve both business and environmental outcomes.
• Ex-post study on sustainable impacts in Latin America – This mixed method study revisits businesses that graduated TechnoServe’s business accelerator program in Latin America between 2016 and 2020 to better understand the long-term impact of business acceleration on business growth and job creation.  
• Women’s Economic Empowerment in Agriculture Index: pro-WEAI-MI (IFPRI) – This research initiative (https://www.cgdev.org/blog/mindful-saving-tanzania-strategies-women-use) will develop and validate measures of women’s empowerment for use by projects engaged in marketing agricultural products using collective-based organizations, and gathers data from the Walmart Central America Smallholder Market Access project.
• MOCCA (Centro Internacional de Agricultura Tropical – CIAT) – This research partnership encompasses the project’s evaluative cycle (baseline, midterm and endline; 2.184 producers in coffee and cocoa value chains across 6 countries) and additional elements of the learning agenda (e.g. an RCT to examine the most effective way of gathering profitability data from coffee farmers).