In Pursuit of Balance : Randomization in Practice in Development Field Experiments
Randomized experiments are increasingly used in development economics, with researchers now facing the question of not just whether to randomize, but how to do so. Pure random assignment guarantees that the treatment and control groups will have id...
| Main Authors: | , |
|---|---|
| Format: | Policy Research Working Paper |
| Language: | English |
| Published: |
World Bank, Washington, DC
2012
|
| Subjects: | |
| Online Access: | http://documents.worldbank.org/curated/en/2008/10/9938284/pursuit-balance-randomization-practice-development-field-experiments http://hdl.handle.net/10986/6910 |
| Summary: | Randomized experiments are increasingly
used in development economics, with researchers now facing
the question of not just whether to randomize, but how to do
so. Pure random assignment guarantees that the treatment and
control groups will have identical characteristics on
average, but in any particular random allocation, the two
groups will differ along some dimensions. Methods used to
pursue greater balance include stratification, pair-wise
matching, and re-randomization. This paper presents new
evidence on the randomization methods used in existing
randomized experiments, and carries out simulations in order
to provide guidance for researchers. Three main results
emerge. First, many researchers are not controlling for the
method of randomization in their analysis. The authors show
this leads to tests with incorrect size, and can result in
lower power than if a pure random draw was used. Second,
they find that in samples of 300 or more, the different
randomization methods perform similarly in terms of
achieving balance on many future outcomes of interest.
However, for very persistent outcome variables and in
smaller sample sizes, pair-wise matching and stratification
perform best. Third, the analysis suggests that on balance
the re-randomization methods common in practice are less
desirable than other methods, such as matching. |
|---|