Measuring the Measurement Error : Method to Qualitatively Validate Survey Data

Empirical social science relies heavily on self-reported data, but subjects may misreport behaviors, especially sensitive ones such as crime or drug abuse. If a treatment influences survey misreporting, it biases causal estimates. We develop a validation technique that uses intensive qualitative wor...

Full description

Bibliographic Details
Main Authors:	Blattman, Christopher, Jamison, Julian, Koroknay-Palicz, Tricia, Rodrigues, Katherine, Sheridan, Margaret
Format:	Journal Article
Language:	en_US
Published:	Elsevier 2016
Subjects:	measurement error survey data validation field experiments crime drugs risky behavior
Online Access:	http://hdl.handle.net/10986/24259

id	okr-10986-24259
recordtype	oai_dc
spelling	okr-10986-242592021-05-26T09:05:19Z Measuring the Measurement Error : Method to Qualitatively Validate Survey Data Blattman, Christopher Jamison, Julian Koroknay-Palicz, Tricia Rodrigues, Katherine Sheridan, Margaret measurement error survey data validation field experiments crime drugs risky behavior Empirical social science relies heavily on self-reported data, but subjects may misreport behaviors, especially sensitive ones such as crime or drug abuse. If a treatment influences survey misreporting, it biases causal estimates. We develop a validation technique that uses intensive qualitative work to assess survey misreporting and pilot it in a field experiment where subjects were assigned to receive cash, therapy, both, or neither. According to survey responses, both treatments reduced crime and other sensitive behaviors. Local researchers spent several days with a random subsample of subjects after surveys, building trust and obtaining verbal confirmation of four sensitive behaviors and two expenditures. In this instance, validation showed survey underreporting of most sensitive behaviors was low and uncorrelated with treatment, while expenditures were under reported in the survey across all arms, but especially in the control group. We use these data to develop measurement error bounds on treatment effects estimated from surveys. 2016-05-09T21:05:41Z 2016-05-09T21:05:41Z 2016-05 Journal Article Journal of Development Economics 0304-3878 http://hdl.handle.net/10986/24259 en_US CC BY-NC-ND 3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo World Bank Elsevier Publications & Research :: Journal Article Publications & Research
repository_type	Digital Repository
institution_category	Foreign Institution
institution	Digital Repositories
building	World Bank Open Knowledge Repository
collection	World Bank
language	en_US
topic	measurement error survey data validation field experiments crime drugs risky behavior
spellingShingle	measurement error survey data validation field experiments crime drugs risky behavior Blattman, Christopher Jamison, Julian Koroknay-Palicz, Tricia Rodrigues, Katherine Sheridan, Margaret Measuring the Measurement Error : Method to Qualitatively Validate Survey Data
description	Empirical social science relies heavily on self-reported data, but subjects may misreport behaviors, especially sensitive ones such as crime or drug abuse. If a treatment influences survey misreporting, it biases causal estimates. We develop a validation technique that uses intensive qualitative work to assess survey misreporting and pilot it in a field experiment where subjects were assigned to receive cash, therapy, both, or neither. According to survey responses, both treatments reduced crime and other sensitive behaviors. Local researchers spent several days with a random subsample of subjects after surveys, building trust and obtaining verbal confirmation of four sensitive behaviors and two expenditures. In this instance, validation showed survey underreporting of most sensitive behaviors was low and uncorrelated with treatment, while expenditures were under reported in the survey across all arms, but especially in the control group. We use these data to develop measurement error bounds on treatment effects estimated from surveys.
format	Journal Article
author	Blattman, Christopher Jamison, Julian Koroknay-Palicz, Tricia Rodrigues, Katherine Sheridan, Margaret
author_facet	Blattman, Christopher Jamison, Julian Koroknay-Palicz, Tricia Rodrigues, Katherine Sheridan, Margaret
author_sort	Blattman, Christopher
title	Measuring the Measurement Error : Method to Qualitatively Validate Survey Data
title_short	Measuring the Measurement Error : Method to Qualitatively Validate Survey Data
title_full	Measuring the Measurement Error : Method to Qualitatively Validate Survey Data
title_fullStr	Measuring the Measurement Error : Method to Qualitatively Validate Survey Data
title_full_unstemmed	Measuring the Measurement Error : Method to Qualitatively Validate Survey Data
title_sort	measuring the measurement error : method to qualitatively validate survey data
publisher	Elsevier
publishDate	2016
url	http://hdl.handle.net/10986/24259
_version_	1764456259580329984

Measuring the Measurement Error : Method to Qualitatively Validate Survey Data

Similar Items