Steeping the organization's tea: Examining the relationship between evaluation use, organizational context, and evaluator characteristics

This study explored the ways in which the context of an organization affects the design of an evaluation and its use. It also examined evaluator characteristics and their relationship to the choice of evaluation design. Theories of organizational learning and knowledge transfer were utilized to develop the study's conceptual model. A review of research on evaluation use in the non-profit sector was conducted along with a review of theories of organizational learning and knowledge transfer. The study surveyed 393 evaluators who were members of the American Evaluation Association via a web-based survey. Respondents answered a series of 47 mostly closed-ended questions about how they would design an evaluation for two organizations described in the survey.;Findings indicated that evaluators design evaluations in distinct ways based on whether or not an organization is ready for learning. In particular, evaluators recommended high levels of process evaluation for an organization that was not ready for learning and high levels of outcome evaluation for an organization that was ready for learning. Evaluators also reported that process evaluation would be more useful for an organization not ready for learning as compared to an organization that was ready for learning. Also, the study found that the type of evaluation design chosen is based on evaluators' individual characteristics. Evaluator characteristics such as older age, working in the for-profit sector, and working with social service agencies predicted recommending high levels of process evaluation. Findings also indicated that evaluators who were more experienced or who conducted social program evaluations were accurate in discerning an organization's readiness for learning. The findings provide evidence that characteristics of the evaluator, characteristics of the evaluation, and qualities of the organization, in conjunction with one another, are predictors of evaluation use. The study builds on literature that has attempted to understand the ways in which organizational context impacts evaluation. The study contributes to the understanding of factors that predict and enhance the use of evaluation.