Program Evaluation in Development without Randomized Assignment to Treatment

This dissertation is a study of how to conduct informative evaluation of development programs under adverse but realistic conditions. Chapter 2 is a study of program evaluation methodology when program uptake is low and individuals are heterogeneous. Low participation, a common feature of complex development interventions, makes the task of impact evaluation more difficult because estimates are less precise. In addition, the program impacts identified by different designs will vary when individuals are heterogeneous in ways that affect program participation and outcomes, further complicating choices made in research design. To put these issues in context, I use the example of index insurance, an innovative financial tool characterized by low participation rates. I focus on the choice between a research design based on randomized eligibility and a randomized encouragement design. Randomized encouragement designs offer a stronger incentive for program participation to a randomly chosen subpopulation, and then use the incentive as an instrumental variable in econometric impact evaluation. In general, the effect estimated using a randomized encouragement design will be biased relative to the true impact of a program on participants. However, this bias may be offset by greater precision, resulting in an estimator with a relatively low Mean Squared Error. In addition, greater unobserved heterogeneity among the study population will not necessarily increase this bias. These conclusions depend on the nature of the program and outcomes being studied, and ought to be considered carefully by researchers weighing alternative research designs.;Chapter 3 is an evaluation of year one of the Rural Business Development (RBD) program for small rice farmers in Leon, Nicaragua. The RBD program is administered by the Millennium Challenge Corporation, and is designed to deliver agricultural extension advice and affordable credit in the form of inputs to farm households. In this essay I use inverse propensity score weighting combined with linear regression to estimate the average impact of the program on rice yields and revenues. In conducting statistical inference, it also accounts for the fact that agricultural outcomes are likely correlated over space in a small area such as the one studied here. The results suggest that the program had no impact on average, likely because of a severe drought during the 2008--2009 rice growing season, but that poorer households may have benefited more than their wealthier counterparts. This does not account for program costs, which when factored in would likely make the overall net benefit of the program negative. There may very well be long term benefits from exploiting extension advice and better access to credit created by the RBD program, and it appears to have shielded poorer farmers somewhat from the impact of the drought. But the results highlight the danger of introducing programs aimed at raising productivity and incomes in areas subject to unanticipated shocks. Incorporating risk management techniques or insurance against systemic risk into extension programs may improve welfare and encourage broader participation in agricultural productivity programs. Chapter 4 concludes.