Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries
Several developing countries are currently implementing phone surveys in response to immediate data needs to monitor the socioeconomic impact of COVID-19. However, phone surveys are often subject to coverage and non-response bias that can compromis...
| Main Authors: | , , |
|---|---|
| Format: | Working Paper |
| Language: | English |
| Published: |
World Bank, Washington, DC
2021
|
| Subjects: | |
| Online Access: | http://documents.worldbank.org/curated/en/859261622035611710/Reducing-Bias-in-Phone-Survey-Samples-Effectiveness-of-Reweighting-Techniques-Using-Face-to-Face-Surveys-as-Frames-in-Four-African-Countries http://hdl.handle.net/10986/35637 |
| id |
okr-10986-35637 |
|---|---|
| recordtype |
oai_dc |
| spelling |
okr-10986-356372021-06-03T05:10:45Z Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries Ambel, Alemayehu McGee, Kevin Tsegay, Asmelash CORONAVIRUS COVID-19 PANDEMIC IMPACT SURVEY METHODOLOGY NONRESPONSE BIAS COVERAGE BIAS WEIGHTING METHODS SAMPLE REPRESENTATIVENESS Several developing countries are currently implementing phone surveys in response to immediate data needs to monitor the socioeconomic impact of COVID-19. However, phone surveys are often subject to coverage and non-response bias that can compromise the representativeness of the sample and the external validity of the estimates obtained from the survey. Using data from high-frequency phone surveys in Ethiopia, Malawi, Nigeria, and Uganda, this study investigates the magnitude and source of biases present in these four surveys and explores the effectiveness of techniques applied to reduce bias. Varying levels of coverage and non-response bias are found in all four countries. The successfully contacted samples in these four countries were biased toward wealthier households with higher living standards. Left unaddressed, this bias would result in biased estimates from the interviewed sample that do not fully reflect the situation of poorer households in the country. However, phone survey biases can be substantially reduced by applying survey weight adjustments using information from the representative survey from which the sample is drawn. Applying these methods to the four surveys resulted in a substantial reduction in bias, although the bias was not fully eradicated. This highlights one of the potential advantages of drawing phone survey samples from existing face-to-face, representative surveys over random digit dialing or using lists from telecom providers where such adjustment methods can be more limited. 2021-06-02T13:10:41Z 2021-06-02T13:10:41Z 2021-05 Working Paper http://documents.worldbank.org/curated/en/859261622035611710/Reducing-Bias-in-Phone-Survey-Samples-Effectiveness-of-Reweighting-Techniques-Using-Face-to-Face-Surveys-as-Frames-in-Four-African-Countries http://hdl.handle.net/10986/35637 English Policy Research Working Paper;No. 9676 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo World Bank World Bank, Washington, DC Publications & Research Publications & Research :: Policy Research Working Paper Africa Africa Eastern and Southern (AFE) Africa Western and Central (AFW) Sub-Saharan Africa Ethiopia Malawi Nigeria Uganda |
| repository_type |
Digital Repository |
| institution_category |
Foreign Institution |
| institution |
Digital Repositories |
| building |
World Bank Open Knowledge Repository |
| collection |
World Bank |
| language |
English |
| topic |
CORONAVIRUS COVID-19 PANDEMIC IMPACT SURVEY METHODOLOGY NONRESPONSE BIAS COVERAGE BIAS WEIGHTING METHODS SAMPLE REPRESENTATIVENESS |
| spellingShingle |
CORONAVIRUS COVID-19 PANDEMIC IMPACT SURVEY METHODOLOGY NONRESPONSE BIAS COVERAGE BIAS WEIGHTING METHODS SAMPLE REPRESENTATIVENESS Ambel, Alemayehu McGee, Kevin Tsegay, Asmelash Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries |
| geographic_facet |
Africa Africa Eastern and Southern (AFE) Africa Western and Central (AFW) Sub-Saharan Africa Ethiopia Malawi Nigeria Uganda |
| relation |
Policy Research Working Paper;No. 9676 |
| description |
Several developing countries are
currently implementing phone surveys in response to
immediate data needs to monitor the socioeconomic impact of
COVID-19. However, phone surveys are often subject to
coverage and non-response bias that can compromise the
representativeness of the sample and the external validity
of the estimates obtained from the survey. Using data from
high-frequency phone surveys in Ethiopia, Malawi, Nigeria,
and Uganda, this study investigates the magnitude and source
of biases present in these four surveys and explores the
effectiveness of techniques applied to reduce bias. Varying
levels of coverage and non-response bias are found in all
four countries. The successfully contacted samples in these
four countries were biased toward wealthier households with
higher living standards. Left unaddressed, this bias would
result in biased estimates from the interviewed sample that
do not fully reflect the situation of poorer households in
the country. However, phone survey biases can be
substantially reduced by applying survey weight adjustments
using information from the representative survey from which
the sample is drawn. Applying these methods to the four
surveys resulted in a substantial reduction in bias,
although the bias was not fully eradicated. This highlights
one of the potential advantages of drawing phone survey
samples from existing face-to-face, representative surveys
over random digit dialing or using lists from telecom
providers where such adjustment methods can be more limited. |
| format |
Working Paper |
| author |
Ambel, Alemayehu McGee, Kevin Tsegay, Asmelash |
| author_facet |
Ambel, Alemayehu McGee, Kevin Tsegay, Asmelash |
| author_sort |
Ambel, Alemayehu |
| title |
Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries |
| title_short |
Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries |
| title_full |
Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries |
| title_fullStr |
Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries |
| title_full_unstemmed |
Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries |
| title_sort |
reducing bias in phone survey samples : effectiveness of reweighting techniques using face-to-face surveys as frames in four african countries |
| publisher |
World Bank, Washington, DC |
| publishDate |
2021 |
| url |
http://documents.worldbank.org/curated/en/859261622035611710/Reducing-Bias-in-Phone-Survey-Samples-Effectiveness-of-Reweighting-Techniques-Using-Face-to-Face-Surveys-as-Frames-in-Four-African-Countries http://hdl.handle.net/10986/35637 |
| _version_ |
1764483470144307200 |