Demystifying impact evaluation: an impact evaluation framework

Abstract

As the global financial, economic, social, environmental, political, technological and health crises deepen and become more complex, funders are increasingly eliciting for programs/research that demonstrate impact. A lot of evaluations often lack the methodological robustness to inform further action by failing to demonstrate the context mechanism and outcome pathways. The landscape is changing. The value of programs/interventions and research is increasingly coming under scrutiny. Impact evaluation is the process of determining to what extent observed changes in the outcome are attributable to the intervention. Figures alone cannot explain why things are that way, and stories alone cannot demonstrate who or how many people benefited and to what extent. Additional methodological tools, such as participatory methods, theories of change, and human centred designs citizen science and the engagement of all key stakeholders, including those previously known as beneficiaries is fundamental. This facilitates a better understanding of the problems while unraveling potential solutions, bearing in mind that any health system intervention can have positive, negative, intended, unintended, direct and indirect consequences. Transdisciplinary, multi and inter-disciplinary approaches and mixed methods therefore become indispensable. To that end we propose an impact evaluation framework with seven central tenets namely; Theory of change (TOC) or program theory, Stakeholder engagement including beneficiaries, Use of mixed method indicators, Baseline of outcome of interest, Midline assessment of outcome of interest, Endline assessment of outcome of interest and Validation/Co-creation.