When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages
When is the rigorous impact evaluation of development projects a luxury, and when a necessity? This Paper studies one high-profile case: the Millennium Villages Project (MVP), an experimental and intensive package intervention to spark sustained lo...
| Main Authors: | , |
|---|---|
| Format: | Policy Research Working Paper |
| Language: | English |
| Published: |
2012
|
| Subjects: | |
| Online Access: | http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20101115114009 http://hdl.handle.net/10986/3959 |
| id |
okr-10986-3959 |
|---|---|
| recordtype |
oai_dc |
| repository_type |
Digital Repository |
| institution_category |
Foreign Institution |
| institution |
Digital Repositories |
| building |
World Bank Open Knowledge Repository |
| collection |
World Bank |
| language |
English |
| topic |
AGRICULTURAL DEVELOPMENT AGRICULTURAL PRODUCTION AGRICULTURAL TECHNOLOGY BASELINE SURVEYS BULLETIN CENSUSES CHILD HEALTH CHILD MORTALITY CHILD MORTALITY RATE CHILD MORTALITY RATES CHILD SURVIVAL CHRONIC MALNUTRITION CLINICS COMMUNITIES COMMUNITY DEVELOPMENT COMMUNITY HEALTH CORRELATIONS CORRUPTION COUNTERFACTUAL DATA COLLECTION DEVELOPING COUNTRIES DEVELOPMENT INTERVENTIONS DEVELOPMENT PLANS DEVELOPMENT RESEARCH DRINKING WATER ECONOMIC DEVELOPMENT ECONOMIC GROWTH ECONOMIC POLICIES EPIDEMIC EVALUATION METHODS EVALUATION RESULTS EXPERIMENTS FIELD EXPERIMENTS GENDER GENDER EQUALITY GLOBAL DEVELOPMENT GLOBAL POVERTY HEALTH INTERVENTIONS HIV HOSPITAL HOUSEHOLD INCOME HOUSEHOLD SURVEYS HOUSEHOLDS HUMAN CAPITAL IMPACT EVALUATION IMPORTANT POLICY INCOME INFORMATION CAMPAIGNS INTEGRATED DEVELOPMENT INTEGRATED RURAL DEVELOPMENT INTERNATIONAL COMMUNITY INTERNATIONAL JOURNAL OF EPIDEMIOLOGY INTERVENTION INTERVENTIONS IRD LEARNING LIVE BIRTHS LIVELIHOOD LIVING CONDITIONS LIVING STANDARDS LOCAL ECONOMY LOCAL GOVERNMENTS LOCAL INFRASTRUCTURE MALARIA MATERNAL MORTALITY MEASLES MEASLES IMMUNIZATION MILLENNIUM DEVELOPMENT GOAL MILLENNIUM DEVELOPMENT GOALS MODERNIZATION MONITORING TEAMS NATIONAL GOVERNMENT NATIONAL LEVEL NATIONAL PLANS NATIONAL POLICY NATIONAL POPULATION NATURAL RESOURCE NEWBORNS NUMBER OF HOUSEHOLDS NURSE NUTRITION ORAL REHYDRATION SOLUTION PER CAPITA INCOME PILOT PROJECTS PLAN OF ACTION POLICY PROCESS POLICY RESEARCH POLICY RESEARCH WORKING PAPER POPULATION DATA POPULATION ESTIMATES POVERTY REDUCTION PRACTITIONERS PREGNANT WOMEN PRIMARY EDUCATION PRIMARY SCHOOL PROGRAM EFFECTS PROGRAMS PROGRESS PROJECT EVALUATION PUBLIC HEALTH PUBLIC SERVICE PURCHASING POWER PURCHASING POWER PARITY QUANTITATIVE INDICATORS REGIONAL POPULATION REMITTANCE RESEARCH DESIGN RESEARCH WORKING PAPERS RESEARCHERS RESETTLEMENT RESOURCE CONSTRAINTS RURAL AREA RURAL AREAS RURAL POPULATION RURAL POVERTY SAMPLE SIZE SANITATION SANITATION FACILITIES SCARCE RESOURCES SCHOOL ATTENDANCE SCIENCES SCIENTIFIC INFORMATION SERVICE DELIVERY SERVICE DELIVERY SYSTEMS SKILLED BIRTH ATTENDANCE SKILLED PERSONNEL SPILLOVER SURVEY DATA SURVIVAL RATE SUSTAINABLE GROWTH TRANSPARENCY TRANSPORTATION VALIDITY VILLAGES |
| spellingShingle |
AGRICULTURAL DEVELOPMENT AGRICULTURAL PRODUCTION AGRICULTURAL TECHNOLOGY BASELINE SURVEYS BULLETIN CENSUSES CHILD HEALTH CHILD MORTALITY CHILD MORTALITY RATE CHILD MORTALITY RATES CHILD SURVIVAL CHRONIC MALNUTRITION CLINICS COMMUNITIES COMMUNITY DEVELOPMENT COMMUNITY HEALTH CORRELATIONS CORRUPTION COUNTERFACTUAL DATA COLLECTION DEVELOPING COUNTRIES DEVELOPMENT INTERVENTIONS DEVELOPMENT PLANS DEVELOPMENT RESEARCH DRINKING WATER ECONOMIC DEVELOPMENT ECONOMIC GROWTH ECONOMIC POLICIES EPIDEMIC EVALUATION METHODS EVALUATION RESULTS EXPERIMENTS FIELD EXPERIMENTS GENDER GENDER EQUALITY GLOBAL DEVELOPMENT GLOBAL POVERTY HEALTH INTERVENTIONS HIV HOSPITAL HOUSEHOLD INCOME HOUSEHOLD SURVEYS HOUSEHOLDS HUMAN CAPITAL IMPACT EVALUATION IMPORTANT POLICY INCOME INFORMATION CAMPAIGNS INTEGRATED DEVELOPMENT INTEGRATED RURAL DEVELOPMENT INTERNATIONAL COMMUNITY INTERNATIONAL JOURNAL OF EPIDEMIOLOGY INTERVENTION INTERVENTIONS IRD LEARNING LIVE BIRTHS LIVELIHOOD LIVING CONDITIONS LIVING STANDARDS LOCAL ECONOMY LOCAL GOVERNMENTS LOCAL INFRASTRUCTURE MALARIA MATERNAL MORTALITY MEASLES MEASLES IMMUNIZATION MILLENNIUM DEVELOPMENT GOAL MILLENNIUM DEVELOPMENT GOALS MODERNIZATION MONITORING TEAMS NATIONAL GOVERNMENT NATIONAL LEVEL NATIONAL PLANS NATIONAL POLICY NATIONAL POPULATION NATURAL RESOURCE NEWBORNS NUMBER OF HOUSEHOLDS NURSE NUTRITION ORAL REHYDRATION SOLUTION PER CAPITA INCOME PILOT PROJECTS PLAN OF ACTION POLICY PROCESS POLICY RESEARCH POLICY RESEARCH WORKING PAPER POPULATION DATA POPULATION ESTIMATES POVERTY REDUCTION PRACTITIONERS PREGNANT WOMEN PRIMARY EDUCATION PRIMARY SCHOOL PROGRAM EFFECTS PROGRAMS PROGRESS PROJECT EVALUATION PUBLIC HEALTH PUBLIC SERVICE PURCHASING POWER PURCHASING POWER PARITY QUANTITATIVE INDICATORS REGIONAL POPULATION REMITTANCE RESEARCH DESIGN RESEARCH WORKING PAPERS RESEARCHERS RESETTLEMENT RESOURCE CONSTRAINTS RURAL AREA RURAL AREAS RURAL POPULATION RURAL POVERTY SAMPLE SIZE SANITATION SANITATION FACILITIES SCARCE RESOURCES SCHOOL ATTENDANCE SCIENCES SCIENTIFIC INFORMATION SERVICE DELIVERY SERVICE DELIVERY SYSTEMS SKILLED BIRTH ATTENDANCE SKILLED PERSONNEL SPILLOVER SURVEY DATA SURVIVAL RATE SUSTAINABLE GROWTH TRANSPARENCY TRANSPORTATION VALIDITY VILLAGES Clemens, Michael A. Demombynes, Gabriel When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages |
| geographic_facet |
Africa West Africa Sub-Saharan Africa East Africa Nigeria Ghana Kenya |
| relation |
Policy Research working paper ; no. WPS 5477 |
| description |
When is the rigorous impact evaluation
of development projects a luxury, and when a necessity? This
Paper studies one high-profile case: the Millennium Villages
Project (MVP), an experimental and intensive package
intervention to spark sustained local economic development
in rural Africa. it illustrates the benefits of rigorous
impact evaluation in this setting by showing that estimates
of the project's effects depend heavily on the
evaluation method. Comparing trends at the MVP intervention
sites in Kenya, Ghana, and Nigeria to trends in the
surrounding areas yields much more modest estimates of the
project's effects than the before-versus-after
comparisons published thus far by the MVP. Neither approach
constitutes a rigorous impact evaluation of the MVP, which
is impossible to perform due to weaknesses in the evaluation
design of the project's initial phase. These weaknesses
include the subjective choice of intervention sites, the
subjective choice of comparison sites, the lack of baseline
data on comparison sites, the small sample size, and the
short time horizon. The authors describe how the next wave
of the intervention could be designed to allow proper
evaluation of the MVP's impact at little additional cost. |
| format |
Publications & Research :: Policy Research Working Paper |
| author |
Clemens, Michael A. Demombynes, Gabriel |
| author_facet |
Clemens, Michael A. Demombynes, Gabriel |
| author_sort |
Clemens, Michael A. |
| title |
When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages |
| title_short |
When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages |
| title_full |
When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages |
| title_fullStr |
When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages |
| title_full_unstemmed |
When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages |
| title_sort |
when does rigorous impact evaluation make a difference? the case of the millennium villages |
| publishDate |
2012 |
| url |
http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20101115114009 http://hdl.handle.net/10986/3959 |
| _version_ |
1764389220418322432 |
| spelling |
okr-10986-39592021-04-23T14:02:14Z When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages Clemens, Michael A. Demombynes, Gabriel AGRICULTURAL DEVELOPMENT AGRICULTURAL PRODUCTION AGRICULTURAL TECHNOLOGY BASELINE SURVEYS BULLETIN CENSUSES CHILD HEALTH CHILD MORTALITY CHILD MORTALITY RATE CHILD MORTALITY RATES CHILD SURVIVAL CHRONIC MALNUTRITION CLINICS COMMUNITIES COMMUNITY DEVELOPMENT COMMUNITY HEALTH CORRELATIONS CORRUPTION COUNTERFACTUAL DATA COLLECTION DEVELOPING COUNTRIES DEVELOPMENT INTERVENTIONS DEVELOPMENT PLANS DEVELOPMENT RESEARCH DRINKING WATER ECONOMIC DEVELOPMENT ECONOMIC GROWTH ECONOMIC POLICIES EPIDEMIC EVALUATION METHODS EVALUATION RESULTS EXPERIMENTS FIELD EXPERIMENTS GENDER GENDER EQUALITY GLOBAL DEVELOPMENT GLOBAL POVERTY HEALTH INTERVENTIONS HIV HOSPITAL HOUSEHOLD INCOME HOUSEHOLD SURVEYS HOUSEHOLDS HUMAN CAPITAL IMPACT EVALUATION IMPORTANT POLICY INCOME INFORMATION CAMPAIGNS INTEGRATED DEVELOPMENT INTEGRATED RURAL DEVELOPMENT INTERNATIONAL COMMUNITY INTERNATIONAL JOURNAL OF EPIDEMIOLOGY INTERVENTION INTERVENTIONS IRD LEARNING LIVE BIRTHS LIVELIHOOD LIVING CONDITIONS LIVING STANDARDS LOCAL ECONOMY LOCAL GOVERNMENTS LOCAL INFRASTRUCTURE MALARIA MATERNAL MORTALITY MEASLES MEASLES IMMUNIZATION MILLENNIUM DEVELOPMENT GOAL MILLENNIUM DEVELOPMENT GOALS MODERNIZATION MONITORING TEAMS NATIONAL GOVERNMENT NATIONAL LEVEL NATIONAL PLANS NATIONAL POLICY NATIONAL POPULATION NATURAL RESOURCE NEWBORNS NUMBER OF HOUSEHOLDS NURSE NUTRITION ORAL REHYDRATION SOLUTION PER CAPITA INCOME PILOT PROJECTS PLAN OF ACTION POLICY PROCESS POLICY RESEARCH POLICY RESEARCH WORKING PAPER POPULATION DATA POPULATION ESTIMATES POVERTY REDUCTION PRACTITIONERS PREGNANT WOMEN PRIMARY EDUCATION PRIMARY SCHOOL PROGRAM EFFECTS PROGRAMS PROGRESS PROJECT EVALUATION PUBLIC HEALTH PUBLIC SERVICE PURCHASING POWER PURCHASING POWER PARITY QUANTITATIVE INDICATORS REGIONAL POPULATION REMITTANCE RESEARCH DESIGN RESEARCH WORKING PAPERS RESEARCHERS RESETTLEMENT RESOURCE CONSTRAINTS RURAL AREA RURAL AREAS RURAL POPULATION RURAL POVERTY SAMPLE SIZE SANITATION SANITATION FACILITIES SCARCE RESOURCES SCHOOL ATTENDANCE SCIENCES SCIENTIFIC INFORMATION SERVICE DELIVERY SERVICE DELIVERY SYSTEMS SKILLED BIRTH ATTENDANCE SKILLED PERSONNEL SPILLOVER SURVEY DATA SURVIVAL RATE SUSTAINABLE GROWTH TRANSPARENCY TRANSPORTATION VALIDITY VILLAGES When is the rigorous impact evaluation of development projects a luxury, and when a necessity? This Paper studies one high-profile case: the Millennium Villages Project (MVP), an experimental and intensive package intervention to spark sustained local economic development in rural Africa. it illustrates the benefits of rigorous impact evaluation in this setting by showing that estimates of the project's effects depend heavily on the evaluation method. Comparing trends at the MVP intervention sites in Kenya, Ghana, and Nigeria to trends in the surrounding areas yields much more modest estimates of the project's effects than the before-versus-after comparisons published thus far by the MVP. Neither approach constitutes a rigorous impact evaluation of the MVP, which is impossible to perform due to weaknesses in the evaluation design of the project's initial phase. These weaknesses include the subjective choice of intervention sites, the subjective choice of comparison sites, the lack of baseline data on comparison sites, the small sample size, and the short time horizon. The authors describe how the next wave of the intervention could be designed to allow proper evaluation of the MVP's impact at little additional cost. 2012-03-19T18:42:51Z 2012-03-19T18:42:51Z 2010-11-01 http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20101115114009 http://hdl.handle.net/10986/3959 English Policy Research working paper ; no. WPS 5477 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo/ World Bank Publications & Research :: Policy Research Working Paper Africa West Africa Sub-Saharan Africa East Africa Nigeria Ghana Kenya |