Can We Trust Shoestring Evaluations?
Many more impact evaluations could be done, and at lower unit cost, if evaluators could avoid the need for baseline data using objective socio-economic surveys and rely instead on retrospective subjective questions on how outcomes have changed, ask...
| Main Author: | |
|---|---|
| Format: | Policy Research Working Paper |
| Language: | English |
| Published: |
2012
|
| Subjects: | |
| Online Access: | http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20120305094659 http://hdl.handle.net/10986/3269 |
| Summary: | Many more impact evaluations could be
done, and at lower unit cost, if evaluators could avoid the
need for baseline data using objective socio-economic
surveys and rely instead on retrospective subjective
questions on how outcomes have changed, asked
post-intervention. But would the results be reliable? This
paper tests a rapid-appraisal, "shoestring,"
method using subjective recall for welfare changes. The
recall data were collected at the end of a full-scale
evaluation of a large poor-area development program in
China. Qualitative recalls of how living standards have
changed are found to provide only weak and biased signals of
the changes in consumption as measured from contemporaneous
surveys. Importantly, the shoestring method was unable to
correct for the selective placement of the program favoring
poor villages. The results of this case study are not
encouraging for future applications of the shoestring
method, although similar tests are needed in other settings. |
|---|