Monitoring and Evaluation
Developing the Results Framework
Balance realism with ambition in the PDO and the targets. The PDO should focus on outcomes; committing to outputs (for example, delivering a certain number of hours of training) is probably not ambitious enough. Projects can balance ambitious objectives by setting realistic targets, particularly for hard-to-employ populations or when working in complex country contexts. The project targets should be set after assessing the baseline levels of economic activity. Even if the project is well-targeted to those who are most in need of skills training, most poor youth will be doing something at baseline to earn money.
Define employment broadly to account for the types of work that women do. AGI pilots defined employment as engagement in any income-generating actitivies. This definition was able to aptly capture all types of economic activities, whether they be formal or informal. AGI projects also measured all types of productive activities that women do, including paid and unpaid labor as well as activities that received in-kind payments.
Ensuring that all stakeholders understand and buy into the results framework is critical for project success. Definitions and targets need to be understood and adopted by all project stakeholders (including trainers) so that everyone understands the standards that the project is being held to. For example, do unpaid internships count as “employed”? What if the participant has only been employed for one week? What about a trainee who didn’t complete the training, but still found employment? Will the project results include those who are self-employed? Do in-kind earnings count as earnings?
Be realistic about the time horizon for measuring outcomes. Projects should measure outcomes at both the baseline and endline to assess the project success. AGI projects measured endline outcomes three to six months after the classroom training ended to allow time for job placement. In some cases projects also verified long-term employment outcomes more than a year later. Behavioral outcomes can take even longer to manifest than economic outcomes. AGI pilots were less successful in capturing behavior change in quantitative surveys over short-time periods, particularly for outcomes such as fertility, which may need to be measured over longer intervals.
The Results Framework begins with the project development objective (PDO), which describes the effect that the project’s outputs will have on the beneficiaries in terms of changed behavior or improved performance. The PDO sets the standard for the project’s success (see Further Reading). A good PDO should focus on outcomes rather than outputs. All the AGI projects had a development objective of improving labor market outcomes such as employment or earnings, rather than outputs such as “employability” or training completion rates. The table, below, provides more details on the results framework strategy.
| What is a Results Framework? | Resource |
| A results framework depicts (in a graphic display, matrix, or summary) the different levels or chains of results expected from the intervention. The framework shows the longer term objectives (often referred to as “outcomes” or “impact”) and the intermediate outcomes and outputs that lead to those desired longer term objectives and clarifies the expected time horizon to see changes in those outcomes.
The World Bank uses results frameworks in its projects, and other agencies use similar conceptual tools, including: logical frameworks, logic models, theories of change, results chains, and outcome mapping. |
 Liberia EPAG Results Framework |
Results frameworks should include clear definitions and conceptual measures of what counts as “employed” and set realistic targets. Each AGI handled these issues differently, but a key success ingredient was that the project teams developed a definition of employment that was adopted by the training providers.
Coeducational projects should always track and use sex-disaggregated indicators in the results framework to understand who is benefitting (and who is not) and make program design adjustments, as needed. At a minimum, projects should always disaggregate by age and sex; in most contexts it is also appropriate to disaggregate by ethnicity. Disaggregated data can help implementers know who the project is or is not reaching. Projects can then adjust the selection criteria and recruitment processes to focus on those who can be better served by the project or may adjust the design to improve outcomes among certain subgroups.
