Evaluation of Development Programs : Randomized Controlled Trials or Regressions?
Can project evaluation methods be used to evaluate programs: complex interventions involving multiple activities? A program evaluation cannot be based simply on separate evaluations of its components if interactions between the activities are impor...
| Main Authors: | , |
|---|---|
| Format: | Policy Research Working Paper |
| Language: | English en_US |
| Published: |
World Bank, Washington, DC
2013
|
| Subjects: | |
| Online Access: | http://documents.worldbank.org/curated/en/2013/09/18192682/evaluation-development-programs-randomized-controlled-trials-or-regressions-evaluation-development-programs-randomized-controlled-trials-or-regressions http://hdl.handle.net/10986/16038 |
| id |
okr-10986-16038 |
|---|---|
| recordtype |
oai_dc |
| spelling |
okr-10986-160382021-04-23T14:03:27Z Evaluation of Development Programs : Randomized Controlled Trials or Regressions? Elbers, Chris Gunning, Jan Willem BENEFICIARIES CALCULATION CAPACITY BUILDING CONTROL GROUPS CORRELATIONS COVARIANCE ESTIMATORS EVALUATION METHODS EX POST EVALUATION EXPERIMENTAL DATA EXPERIMENTAL METHODS EXPERIMENTS HEALTH OUTCOMES IMPACT EVALUATION INSTRUMENTAL VARIABLES INTERVENTION LEARNING LIVING STANDARDS MATRIX NGO NUTRITIONAL STATUS POLICY CHANGES PRECISION PROBABILITY PROGRAM EFFECTS PROGRAM EVALUATION PROGRAM IMPLEMENTATION PROGRAM INTERVENTIONS PROGRAMS PROJECT EVALUATION RANDOMIZATION RANDOMIZED CONTROLLED TRIALS REGRESSION ANALYSIS RESEARCH WORKING PAPERS SAMPLING ERRORS STANDARD ERRORS STATISTICAL ANALYSIS TARGETING TECHNIQUES TREATMENT EFFECTS VALIDITY treatment heterogeneity sector-wide programs aid effectiveness program evaluation policy evaluation Can project evaluation methods be used to evaluate programs: complex interventions involving multiple activities? A program evaluation cannot be based simply on separate evaluations of its components if interactions between the activities are important. In this paper a measure is proposed, the total program effect (TPE), which is an extension of the average treatment effect on the treated (ATET). It explicitly takes into account that in the real world (with heterogeneous treatment effects) individual treatment effects and program assignment are often correlated. The TPE can also deal with the common situation in which such a correlation is the result of decisions on (intended) program participation not being taken centrally. In this context RCTs are less suitable even for the simplest interventions. The TPE can be estimated by applying regression techniques to observational data from a representative sample from the targeted population. The approach is illustrated with an evaluation of a health insurance program in Vietnam. 2013-10-02T21:25:40Z 2013-10-02T21:25:40Z 2013-09 http://documents.worldbank.org/curated/en/2013/09/18192682/evaluation-development-programs-randomized-controlled-trials-or-regressions-evaluation-development-programs-randomized-controlled-trials-or-regressions http://hdl.handle.net/10986/16038 English en_US Policy Research Working Paper;No. 6587 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo/ World Bank World Bank, Washington, DC Publications & Research :: Policy Research Working Paper Publications & Research East Asia and Pacific Vietnam |
| repository_type |
Digital Repository |
| institution_category |
Foreign Institution |
| institution |
Digital Repositories |
| building |
World Bank Open Knowledge Repository |
| collection |
World Bank |
| language |
English en_US |
| topic |
BENEFICIARIES CALCULATION CAPACITY BUILDING CONTROL GROUPS CORRELATIONS COVARIANCE ESTIMATORS EVALUATION METHODS EX POST EVALUATION EXPERIMENTAL DATA EXPERIMENTAL METHODS EXPERIMENTS HEALTH OUTCOMES IMPACT EVALUATION INSTRUMENTAL VARIABLES INTERVENTION LEARNING LIVING STANDARDS MATRIX NGO NUTRITIONAL STATUS POLICY CHANGES PRECISION PROBABILITY PROGRAM EFFECTS PROGRAM EVALUATION PROGRAM IMPLEMENTATION PROGRAM INTERVENTIONS PROGRAMS PROJECT EVALUATION RANDOMIZATION RANDOMIZED CONTROLLED TRIALS REGRESSION ANALYSIS RESEARCH WORKING PAPERS SAMPLING ERRORS STANDARD ERRORS STATISTICAL ANALYSIS TARGETING TECHNIQUES TREATMENT EFFECTS VALIDITY treatment heterogeneity sector-wide programs aid effectiveness program evaluation policy evaluation |
| spellingShingle |
BENEFICIARIES CALCULATION CAPACITY BUILDING CONTROL GROUPS CORRELATIONS COVARIANCE ESTIMATORS EVALUATION METHODS EX POST EVALUATION EXPERIMENTAL DATA EXPERIMENTAL METHODS EXPERIMENTS HEALTH OUTCOMES IMPACT EVALUATION INSTRUMENTAL VARIABLES INTERVENTION LEARNING LIVING STANDARDS MATRIX NGO NUTRITIONAL STATUS POLICY CHANGES PRECISION PROBABILITY PROGRAM EFFECTS PROGRAM EVALUATION PROGRAM IMPLEMENTATION PROGRAM INTERVENTIONS PROGRAMS PROJECT EVALUATION RANDOMIZATION RANDOMIZED CONTROLLED TRIALS REGRESSION ANALYSIS RESEARCH WORKING PAPERS SAMPLING ERRORS STANDARD ERRORS STATISTICAL ANALYSIS TARGETING TECHNIQUES TREATMENT EFFECTS VALIDITY treatment heterogeneity sector-wide programs aid effectiveness program evaluation policy evaluation Elbers, Chris Gunning, Jan Willem Evaluation of Development Programs : Randomized Controlled Trials or Regressions? |
| geographic_facet |
East Asia and Pacific Vietnam |
| relation |
Policy Research Working Paper;No. 6587 |
| description |
Can project evaluation methods be used
to evaluate programs: complex interventions involving
multiple activities? A program evaluation cannot be based
simply on separate evaluations of its components if
interactions between the activities are important. In this
paper a measure is proposed, the total program effect (TPE),
which is an extension of the average treatment effect on the
treated (ATET). It explicitly takes into account that in the
real world (with heterogeneous treatment effects) individual
treatment effects and program assignment are often
correlated. The TPE can also deal with the common situation
in which such a correlation is the result of decisions on
(intended) program participation not being taken centrally.
In this context RCTs are less suitable even for the simplest
interventions. The TPE can be estimated by applying
regression techniques to observational data from a
representative sample from the targeted population. The
approach is illustrated with an evaluation of a health
insurance program in Vietnam. |
| format |
Publications & Research :: Policy Research Working Paper |
| author |
Elbers, Chris Gunning, Jan Willem |
| author_facet |
Elbers, Chris Gunning, Jan Willem |
| author_sort |
Elbers, Chris |
| title |
Evaluation of Development Programs : Randomized Controlled Trials or Regressions? |
| title_short |
Evaluation of Development Programs : Randomized Controlled Trials or Regressions? |
| title_full |
Evaluation of Development Programs : Randomized Controlled Trials or Regressions? |
| title_fullStr |
Evaluation of Development Programs : Randomized Controlled Trials or Regressions? |
| title_full_unstemmed |
Evaluation of Development Programs : Randomized Controlled Trials or Regressions? |
| title_sort |
evaluation of development programs : randomized controlled trials or regressions? |
| publisher |
World Bank, Washington, DC |
| publishDate |
2013 |
| url |
http://documents.worldbank.org/curated/en/2013/09/18192682/evaluation-development-programs-randomized-controlled-trials-or-regressions-evaluation-development-programs-randomized-controlled-trials-or-regressions http://hdl.handle.net/10986/16038 |
| _version_ |
1764432050996117504 |