Can We Trust Shoestring Evaluations?

Many more impact evaluations could be done, and at lower unit cost, if evaluators could avoid the need for baseline data using objective socio-economic surveys and rely instead on retrospective subjective questions on how outcomes have changed, asked post-intervention. But would the results be relia...

Full description

Bibliographic Details
Main Author:	Ravallion, Martin
Format:	Journal Article
Language:	en_US
Published:	Oxford University Press on behalf of the World Bank 2016
Subjects:	antipoverty program household income household surveys impact evaluation living standards longitudinal data poverty measures poverty reduction program evaluation selection bias targeting welfare levels
Online Access:	http://hdl.handle.net/10986/24191

id	okr-10986-24191
recordtype	oai_dc
spelling	okr-10986-241912021-04-23T14:04:20Z Can We Trust Shoestring Evaluations? Ravallion, Martin antipoverty program household income household surveys impact evaluation living standards longitudinal data poverty measures poverty reduction program evaluation selection bias targeting welfare levels Many more impact evaluations could be done, and at lower unit cost, if evaluators could avoid the need for baseline data using objective socio-economic surveys and rely instead on retrospective subjective questions on how outcomes have changed, asked post-intervention. But would the results be reliable? This paper tests a rapid-appraisal, “shoestring” method using subjective recall for welfare changes. The recall data were collected at the end of a full-scale evaluation of a large World Bank supported poor-area development program in China. Qualitative recalls on how living standards have changed are found to provide only weak and biased signals of the changes in consumption as measured from contemporaneous surveys. Importantly, the shoestring method was unable to correct for the selective placement of the program favoring poor villages. The results of this case study are not encouraging for future applications of the shoestring method, although similar tests are needed in other settings. 2016-05-03T20:59:49Z 2016-05-03T20:59:49Z 2014-10-02 Journal Article World Bank Economic Review 1564-698X http://hdl.handle.net/10986/24191 en_US CC BY-NC-ND 3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo World Bank Oxford University Press on behalf of the World Bank Publications & Research :: Journal Article Publications & Research
repository_type	Digital Repository
institution_category	Foreign Institution
institution	Digital Repositories
building	World Bank Open Knowledge Repository
collection	World Bank
language	en_US
topic	antipoverty program household income household surveys impact evaluation living standards longitudinal data poverty measures poverty reduction program evaluation selection bias targeting welfare levels
spellingShingle	antipoverty program household income household surveys impact evaluation living standards longitudinal data poverty measures poverty reduction program evaluation selection bias targeting welfare levels Ravallion, Martin Can We Trust Shoestring Evaluations?
description	Many more impact evaluations could be done, and at lower unit cost, if evaluators could avoid the need for baseline data using objective socio-economic surveys and rely instead on retrospective subjective questions on how outcomes have changed, asked post-intervention. But would the results be reliable? This paper tests a rapid-appraisal, “shoestring” method using subjective recall for welfare changes. The recall data were collected at the end of a full-scale evaluation of a large World Bank supported poor-area development program in China. Qualitative recalls on how living standards have changed are found to provide only weak and biased signals of the changes in consumption as measured from contemporaneous surveys. Importantly, the shoestring method was unable to correct for the selective placement of the program favoring poor villages. The results of this case study are not encouraging for future applications of the shoestring method, although similar tests are needed in other settings.
format	Journal Article
author	Ravallion, Martin
author_facet	Ravallion, Martin
author_sort	Ravallion, Martin
title	Can We Trust Shoestring Evaluations?
title_short	Can We Trust Shoestring Evaluations?
title_full	Can We Trust Shoestring Evaluations?
title_fullStr	Can We Trust Shoestring Evaluations?
title_full_unstemmed	Can We Trust Shoestring Evaluations?
title_sort	can we trust shoestring evaluations?
publisher	Oxford University Press on behalf of the World Bank
publishDate	2016
url	http://hdl.handle.net/10986/24191
_version_	1764455891801735168

Can We Trust Shoestring Evaluations?

Similar Items