Overview

This course provides an introduction to evaluating public health programs at scale. This course focuses on evaluating public health programs and policies in low- and middle-income countries, however, core skills of designing and carrying out an evaluation are applicable to any public health programs and policies. The course will equip you with skills to: 1. Critique an evaluation of an international health program, identifying its strengths and possible weaknesses and how they could be addressed. 2. Develop a technically-sound evaluation plan for a reproductive, maternal, newborn, child health (RMNCAH) and nutrition program being implemented at scale in a low- or middle-income country, including evaluation design, key indicators, measurement methods, analysis, and communication of results. 3. Guide program managers and donors through a process of agreeing on priority evaluation activities included in an evaluation plan for a specific RMNCAH and nutrition program. 4. Make informed decisions about whether they want to pursue further learning and/or a professional role as an evaluator of large-scale programs. The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

Syllabus

Module 1: Getting Started in the Course

Program evaluations are important in global health for many reasons. Some of the most important reasons are to: 1) service as a "reality check" on subjective expectations; 2) to guide continued improvement of the program; and 3) to justify continued funding. In this course, you will learn how to design and carry out high-quality program evaluations that answer the priority questions of program managers. Although much of the content is relevant to all program evaluations, we will focus on the evaluation of programs being delivered at scale by governments and their partners in low and middle income countries. This module will introduce you to program evaluation and our focus on effectiveness evaluations of large-scale reproductive, maternal, newborn, and child health and nutrition (RMNCH&N) programs.

Module 2: Designing the Evaluation - Part 1

This module will walk you through the steps to designing an evaluation including identifying relevant stakeholders, developing an impact model, and selecting appropriate data sources and indicators.

Module 3: Designing the evaluation - Part 2

This module will help you choose appropriate evaluation designs after defining priority evaluation questions.

Module 4: Methods and tools to answer priority evaluation questions

This module will guide you through how to critically appraise a program's targets for coverage and impact using the Lives Saved Tool, assess program implementation strength and quality of care, measure coverage, and estimate program impact.

Module 5: Implementing an effectiveness evaluation of a large-scale program

This module will outline steps to developing measurement and analysis plans, integrating equity in your evaluations, and how to promote the uptake and use of evaluation results.

Module 6: Course review and exploring your potential as an external evaluator of large-scale programs

This module will review all course concepts using a real-world example - the evaluation of integrated community case management of childhood illness (iCCM) in Ethiopia. The course concludes with a reading on helpful skills for evaluators and resources for further learning about evaluation.