When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages

When is the rigorous impact evaluation of development projects a luxury, and when a necessity? This Paper studies one high-profile case: the Millennium Villages Project (MVP), an experimental and intensive package intervention to spark sustained lo...

Full description

Bibliographic Details
Main Authors: Clemens, Michael A., Demombynes, Gabriel
Format: Policy Research Working Paper
Language:English
Published: 2012
Subjects:
HIV
IRD
Online Access:http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20101115114009
http://hdl.handle.net/10986/3959
id okr-10986-3959
recordtype oai_dc
repository_type Digital Repository
institution_category Foreign Institution
institution Digital Repositories
building World Bank Open Knowledge Repository
collection World Bank
language English
topic AGRICULTURAL DEVELOPMENT
AGRICULTURAL PRODUCTION
AGRICULTURAL TECHNOLOGY
BASELINE SURVEYS
BULLETIN
CENSUSES
CHILD HEALTH
CHILD MORTALITY
CHILD MORTALITY RATE
CHILD MORTALITY RATES
CHILD SURVIVAL
CHRONIC MALNUTRITION
CLINICS
COMMUNITIES
COMMUNITY DEVELOPMENT
COMMUNITY HEALTH
CORRELATIONS
CORRUPTION
COUNTERFACTUAL
DATA COLLECTION
DEVELOPING COUNTRIES
DEVELOPMENT INTERVENTIONS
DEVELOPMENT PLANS
DEVELOPMENT RESEARCH
DRINKING WATER
ECONOMIC DEVELOPMENT
ECONOMIC GROWTH
ECONOMIC POLICIES
EPIDEMIC
EVALUATION METHODS
EVALUATION RESULTS
EXPERIMENTS
FIELD EXPERIMENTS
GENDER
GENDER EQUALITY
GLOBAL DEVELOPMENT
GLOBAL POVERTY
HEALTH INTERVENTIONS
HIV
HOSPITAL
HOUSEHOLD INCOME
HOUSEHOLD SURVEYS
HOUSEHOLDS
HUMAN CAPITAL
IMPACT EVALUATION
IMPORTANT POLICY
INCOME
INFORMATION CAMPAIGNS
INTEGRATED DEVELOPMENT
INTEGRATED RURAL DEVELOPMENT
INTERNATIONAL COMMUNITY
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
INTERVENTION
INTERVENTIONS
IRD
LEARNING
LIVE BIRTHS
LIVELIHOOD
LIVING CONDITIONS
LIVING STANDARDS
LOCAL ECONOMY
LOCAL GOVERNMENTS
LOCAL INFRASTRUCTURE
MALARIA
MATERNAL MORTALITY
MEASLES
MEASLES IMMUNIZATION
MILLENNIUM DEVELOPMENT GOAL
MILLENNIUM DEVELOPMENT GOALS
MODERNIZATION
MONITORING TEAMS
NATIONAL GOVERNMENT
NATIONAL LEVEL
NATIONAL PLANS
NATIONAL POLICY
NATIONAL POPULATION
NATURAL RESOURCE
NEWBORNS
NUMBER OF HOUSEHOLDS
NURSE
NUTRITION
ORAL REHYDRATION SOLUTION
PER CAPITA INCOME
PILOT PROJECTS
PLAN OF ACTION
POLICY PROCESS
POLICY RESEARCH
POLICY RESEARCH WORKING PAPER
POPULATION DATA
POPULATION ESTIMATES
POVERTY REDUCTION
PRACTITIONERS
PREGNANT WOMEN
PRIMARY EDUCATION
PRIMARY SCHOOL
PROGRAM EFFECTS
PROGRAMS
PROGRESS
PROJECT EVALUATION
PUBLIC HEALTH
PUBLIC SERVICE
PURCHASING POWER
PURCHASING POWER PARITY
QUANTITATIVE INDICATORS
REGIONAL POPULATION
REMITTANCE
RESEARCH DESIGN
RESEARCH WORKING PAPERS
RESEARCHERS
RESETTLEMENT
RESOURCE CONSTRAINTS
RURAL AREA
RURAL AREAS
RURAL POPULATION
RURAL POVERTY
SAMPLE SIZE
SANITATION
SANITATION FACILITIES
SCARCE RESOURCES
SCHOOL ATTENDANCE
SCIENCES
SCIENTIFIC INFORMATION
SERVICE DELIVERY
SERVICE DELIVERY SYSTEMS
SKILLED BIRTH ATTENDANCE
SKILLED PERSONNEL
SPILLOVER
SURVEY DATA
SURVIVAL RATE
SUSTAINABLE GROWTH
TRANSPARENCY
TRANSPORTATION
VALIDITY
VILLAGES
spellingShingle AGRICULTURAL DEVELOPMENT
AGRICULTURAL PRODUCTION
AGRICULTURAL TECHNOLOGY
BASELINE SURVEYS
BULLETIN
CENSUSES
CHILD HEALTH
CHILD MORTALITY
CHILD MORTALITY RATE
CHILD MORTALITY RATES
CHILD SURVIVAL
CHRONIC MALNUTRITION
CLINICS
COMMUNITIES
COMMUNITY DEVELOPMENT
COMMUNITY HEALTH
CORRELATIONS
CORRUPTION
COUNTERFACTUAL
DATA COLLECTION
DEVELOPING COUNTRIES
DEVELOPMENT INTERVENTIONS
DEVELOPMENT PLANS
DEVELOPMENT RESEARCH
DRINKING WATER
ECONOMIC DEVELOPMENT
ECONOMIC GROWTH
ECONOMIC POLICIES
EPIDEMIC
EVALUATION METHODS
EVALUATION RESULTS
EXPERIMENTS
FIELD EXPERIMENTS
GENDER
GENDER EQUALITY
GLOBAL DEVELOPMENT
GLOBAL POVERTY
HEALTH INTERVENTIONS
HIV
HOSPITAL
HOUSEHOLD INCOME
HOUSEHOLD SURVEYS
HOUSEHOLDS
HUMAN CAPITAL
IMPACT EVALUATION
IMPORTANT POLICY
INCOME
INFORMATION CAMPAIGNS
INTEGRATED DEVELOPMENT
INTEGRATED RURAL DEVELOPMENT
INTERNATIONAL COMMUNITY
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
INTERVENTION
INTERVENTIONS
IRD
LEARNING
LIVE BIRTHS
LIVELIHOOD
LIVING CONDITIONS
LIVING STANDARDS
LOCAL ECONOMY
LOCAL GOVERNMENTS
LOCAL INFRASTRUCTURE
MALARIA
MATERNAL MORTALITY
MEASLES
MEASLES IMMUNIZATION
MILLENNIUM DEVELOPMENT GOAL
MILLENNIUM DEVELOPMENT GOALS
MODERNIZATION
MONITORING TEAMS
NATIONAL GOVERNMENT
NATIONAL LEVEL
NATIONAL PLANS
NATIONAL POLICY
NATIONAL POPULATION
NATURAL RESOURCE
NEWBORNS
NUMBER OF HOUSEHOLDS
NURSE
NUTRITION
ORAL REHYDRATION SOLUTION
PER CAPITA INCOME
PILOT PROJECTS
PLAN OF ACTION
POLICY PROCESS
POLICY RESEARCH
POLICY RESEARCH WORKING PAPER
POPULATION DATA
POPULATION ESTIMATES
POVERTY REDUCTION
PRACTITIONERS
PREGNANT WOMEN
PRIMARY EDUCATION
PRIMARY SCHOOL
PROGRAM EFFECTS
PROGRAMS
PROGRESS
PROJECT EVALUATION
PUBLIC HEALTH
PUBLIC SERVICE
PURCHASING POWER
PURCHASING POWER PARITY
QUANTITATIVE INDICATORS
REGIONAL POPULATION
REMITTANCE
RESEARCH DESIGN
RESEARCH WORKING PAPERS
RESEARCHERS
RESETTLEMENT
RESOURCE CONSTRAINTS
RURAL AREA
RURAL AREAS
RURAL POPULATION
RURAL POVERTY
SAMPLE SIZE
SANITATION
SANITATION FACILITIES
SCARCE RESOURCES
SCHOOL ATTENDANCE
SCIENCES
SCIENTIFIC INFORMATION
SERVICE DELIVERY
SERVICE DELIVERY SYSTEMS
SKILLED BIRTH ATTENDANCE
SKILLED PERSONNEL
SPILLOVER
SURVEY DATA
SURVIVAL RATE
SUSTAINABLE GROWTH
TRANSPARENCY
TRANSPORTATION
VALIDITY
VILLAGES
Clemens, Michael A.
Demombynes, Gabriel
When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
geographic_facet Africa
West Africa
Sub-Saharan Africa
East Africa
Nigeria
Ghana
Kenya
relation Policy Research working paper ; no. WPS 5477
description When is the rigorous impact evaluation of development projects a luxury, and when a necessity? This Paper studies one high-profile case: the Millennium Villages Project (MVP), an experimental and intensive package intervention to spark sustained local economic development in rural Africa. it illustrates the benefits of rigorous impact evaluation in this setting by showing that estimates of the project's effects depend heavily on the evaluation method. Comparing trends at the MVP intervention sites in Kenya, Ghana, and Nigeria to trends in the surrounding areas yields much more modest estimates of the project's effects than the before-versus-after comparisons published thus far by the MVP. Neither approach constitutes a rigorous impact evaluation of the MVP, which is impossible to perform due to weaknesses in the evaluation design of the project's initial phase. These weaknesses include the subjective choice of intervention sites, the subjective choice of comparison sites, the lack of baseline data on comparison sites, the small sample size, and the short time horizon. The authors describe how the next wave of the intervention could be designed to allow proper evaluation of the MVP's impact at little additional cost.
format Publications & Research :: Policy Research Working Paper
author Clemens, Michael A.
Demombynes, Gabriel
author_facet Clemens, Michael A.
Demombynes, Gabriel
author_sort Clemens, Michael A.
title When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
title_short When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
title_full When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
title_fullStr When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
title_full_unstemmed When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
title_sort when does rigorous impact evaluation make a difference? the case of the millennium villages
publishDate 2012
url http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20101115114009
http://hdl.handle.net/10986/3959
_version_ 1764389220418322432
spelling okr-10986-39592021-04-23T14:02:14Z When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages Clemens, Michael A. Demombynes, Gabriel AGRICULTURAL DEVELOPMENT AGRICULTURAL PRODUCTION AGRICULTURAL TECHNOLOGY BASELINE SURVEYS BULLETIN CENSUSES CHILD HEALTH CHILD MORTALITY CHILD MORTALITY RATE CHILD MORTALITY RATES CHILD SURVIVAL CHRONIC MALNUTRITION CLINICS COMMUNITIES COMMUNITY DEVELOPMENT COMMUNITY HEALTH CORRELATIONS CORRUPTION COUNTERFACTUAL DATA COLLECTION DEVELOPING COUNTRIES DEVELOPMENT INTERVENTIONS DEVELOPMENT PLANS DEVELOPMENT RESEARCH DRINKING WATER ECONOMIC DEVELOPMENT ECONOMIC GROWTH ECONOMIC POLICIES EPIDEMIC EVALUATION METHODS EVALUATION RESULTS EXPERIMENTS FIELD EXPERIMENTS GENDER GENDER EQUALITY GLOBAL DEVELOPMENT GLOBAL POVERTY HEALTH INTERVENTIONS HIV HOSPITAL HOUSEHOLD INCOME HOUSEHOLD SURVEYS HOUSEHOLDS HUMAN CAPITAL IMPACT EVALUATION IMPORTANT POLICY INCOME INFORMATION CAMPAIGNS INTEGRATED DEVELOPMENT INTEGRATED RURAL DEVELOPMENT INTERNATIONAL COMMUNITY INTERNATIONAL JOURNAL OF EPIDEMIOLOGY INTERVENTION INTERVENTIONS IRD LEARNING LIVE BIRTHS LIVELIHOOD LIVING CONDITIONS LIVING STANDARDS LOCAL ECONOMY LOCAL GOVERNMENTS LOCAL INFRASTRUCTURE MALARIA MATERNAL MORTALITY MEASLES MEASLES IMMUNIZATION MILLENNIUM DEVELOPMENT GOAL MILLENNIUM DEVELOPMENT GOALS MODERNIZATION MONITORING TEAMS NATIONAL GOVERNMENT NATIONAL LEVEL NATIONAL PLANS NATIONAL POLICY NATIONAL POPULATION NATURAL RESOURCE NEWBORNS NUMBER OF HOUSEHOLDS NURSE NUTRITION ORAL REHYDRATION SOLUTION PER CAPITA INCOME PILOT PROJECTS PLAN OF ACTION POLICY PROCESS POLICY RESEARCH POLICY RESEARCH WORKING PAPER POPULATION DATA POPULATION ESTIMATES POVERTY REDUCTION PRACTITIONERS PREGNANT WOMEN PRIMARY EDUCATION PRIMARY SCHOOL PROGRAM EFFECTS PROGRAMS PROGRESS PROJECT EVALUATION PUBLIC HEALTH PUBLIC SERVICE PURCHASING POWER PURCHASING POWER PARITY QUANTITATIVE INDICATORS REGIONAL POPULATION REMITTANCE RESEARCH DESIGN RESEARCH WORKING PAPERS RESEARCHERS RESETTLEMENT RESOURCE CONSTRAINTS RURAL AREA RURAL AREAS RURAL POPULATION RURAL POVERTY SAMPLE SIZE SANITATION SANITATION FACILITIES SCARCE RESOURCES SCHOOL ATTENDANCE SCIENCES SCIENTIFIC INFORMATION SERVICE DELIVERY SERVICE DELIVERY SYSTEMS SKILLED BIRTH ATTENDANCE SKILLED PERSONNEL SPILLOVER SURVEY DATA SURVIVAL RATE SUSTAINABLE GROWTH TRANSPARENCY TRANSPORTATION VALIDITY VILLAGES When is the rigorous impact evaluation of development projects a luxury, and when a necessity? This Paper studies one high-profile case: the Millennium Villages Project (MVP), an experimental and intensive package intervention to spark sustained local economic development in rural Africa. it illustrates the benefits of rigorous impact evaluation in this setting by showing that estimates of the project's effects depend heavily on the evaluation method. Comparing trends at the MVP intervention sites in Kenya, Ghana, and Nigeria to trends in the surrounding areas yields much more modest estimates of the project's effects than the before-versus-after comparisons published thus far by the MVP. Neither approach constitutes a rigorous impact evaluation of the MVP, which is impossible to perform due to weaknesses in the evaluation design of the project's initial phase. These weaknesses include the subjective choice of intervention sites, the subjective choice of comparison sites, the lack of baseline data on comparison sites, the small sample size, and the short time horizon. The authors describe how the next wave of the intervention could be designed to allow proper evaluation of the MVP's impact at little additional cost. 2012-03-19T18:42:51Z 2012-03-19T18:42:51Z 2010-11-01 http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20101115114009 http://hdl.handle.net/10986/3959 English Policy Research working paper ; no. WPS 5477 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo/ World Bank Publications & Research :: Policy Research Working Paper Africa West Africa Sub-Saharan Africa East Africa Nigeria Ghana Kenya