Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. Ho...
| Main Authors: | , , , , |
|---|---|
| Format: | Policy Research Working Paper |
| Language: | English en_US |
| Published: |
World Bank, Washington, DC
2013
|
| Subjects: | |
| Online Access: | http://documents.worldbank.org/curated/en/2012/12/17103546/learning-experiments-never-happened-lessons-trying-conduct-randomized-evaluations-matching-grant-programs-africa http://hdl.handle.net/10986/12200 |
| id |
okr-10986-12200 |
|---|---|
| recordtype |
oai_dc |
| repository_type |
Digital Repository |
| institution_category |
Foreign Institution |
| institution |
Digital Repositories |
| building |
World Bank Open Knowledge Repository |
| collection |
World Bank |
| language |
English en_US |
| topic |
ACCESS TO CREDIT ACCESS TO INFORMATION ACCESS TO NETWORKS ACCOUNTING ADVERTISING APPLICATION PROCESS AUDITING BACK-UP BANK LENDING BANK LOAN BANK LOANS BANKS BLOG BRANCH BUSINESS APPLICATIONS BUSINESS COMMUNITY BUSINESS DEVELOPMENT BUSINESS DEVELOPMENT SERVICE BUSINESS DEVELOPMENT SERVICES BUSINESS ENVIRONMENT BUSINESS NEEDS BUSINESS OWNER BUSINESS OWNERS BUSINESS PLAN BUSINESS REGISTRATION BUSINESS SERVICE BUSINESS SERVICE PROVIDERS BUSINESS SERVICES BUSINESS TOOLS BUSINESS TRAINING BUSINESSES BUYERS CAPITAL EXPENDITURE CAPITAL INVESTMENTS CAPITAL MARKET CERTIFICATE CHAMBER OF COMMERCE CHAMBERS OF COMMERCE COMPANY COMPETITIVENESS CONSULTING SERVICES COST-SHARING CREDIT CONSTRAINTS CREDIT GUARANTEES CREDIT LINE CREDIT MARKET CREDIT MARKET FAILURE CURRENT MARKET PRICE DATA COLLECTION DAY-TO-DAY OPERATIONS DECISION-MAKING DISTRIBUTION SYSTEMS E-COMMERCE ECONOMIC GROWTH EDUCATIONAL BACKGROUND ELECTRICITY ELIGIBILITY CRITERIA ELIGIBILITY CRITERION ELIGIBLE APPLICANTS EMPLOYEE EMPLOYMENT ENTERPRISE ASSISTANCE ENTERPRISE DEVELOPMENT ENTERPRISE GROWTH ENTERPRISE SUPPORT ENTREPRENEURS ENTREPRENEURSHIP EQUIPMENT EQUIPMENT PURCHASES EQUITY INVESTMENT EXCESS DEMAND EXPENDITURE EXPORT DEVELOPMENT EXPORT PROMOTION EXTERNALITIES FINANCIAL SYSTEMS FIRM SIZE FIRMS GENDER GOVERNMENT FUNDING GOVERNMENT INTERVENTION GOVERNMENT OFFICES GRANT PROGRAMS HUMAN CAPITAL IMPACT ASSESSMENTS INFORMATION TECHNOLOGY INNOVATION INNOVATION POLICY INNOVATIONS INTELLECTUAL PROPERTY INTELLECTUAL PROPERTY RIGHT INTEREST RATE INTERNATIONAL BANK INVESTMENT OPPORTUNITIES INVESTMENT PROPOSALS LABOR MARKET LACK OF INFORMATION LEARNING LIMITED ACCESS LOAN LOAN APPLICATIONS LOCAL GOVERNMENTS MANUFACTURER MANUFACTURING MARKET DEVELOPMENT MARKET RESEARCH MARKETING MARKETING STRATEGIES MARKETING STRATEGY MATCHING FUNDS MATCHING GRANTS MEDIUM ENTERPRISE MEDIUM ENTERPRISES MICRO ENTERPRISES MICROENTERPRISES MICROFINANCE MICROFINANCE INSTITUTIONS ONLINE FORMS OPEN ACCESS OUTREACH OUTSOURCING PARTIAL CREDIT PRIVATE FUNDING PRIVATE INVESTMENT PRIVATE SECTOR PRIVATE SECTOR DEVELOPMENT PRIVATE SECTOR FIRM PROCUREMENT PROCUREMENT PROCESS PRODUCT DEVELOPMENT PRODUCTIVITY PROMOTION ACTIVITIES PUBLIC FUNDS RADIO RECEIPTS RED TAPE REGISTRY RESULT RESULTS SEARCH SERVICE PROVIDER SERVICE PROVIDERS SMALL ENTERPRISE SMALL FIRMS SME SME DEVELOPMENT SUPPLIER SUPPLIERS TARGETS TECHNICAL ASSISTANCE TECHNICAL SKILLS TECHNICAL SUPPORT TECHNOLOGY DIFFUSION TECHNOLOGY TRANSFER TELEVISION TELEVISION ADVERTISEMENTS TRAINING PROGRAMS TRUST FUNDS UNEMPLOYMENT USES VENTURE CAPITAL VENTURE CAPITALIST WAGES WEB WORKING CAPITAL AFRICA GENDER POLICY GENDER INNOVATION LAB WOMEN AND YOUTH EMPLOYMENT |
| spellingShingle |
ACCESS TO CREDIT ACCESS TO INFORMATION ACCESS TO NETWORKS ACCOUNTING ADVERTISING APPLICATION PROCESS AUDITING BACK-UP BANK LENDING BANK LOAN BANK LOANS BANKS BLOG BRANCH BUSINESS APPLICATIONS BUSINESS COMMUNITY BUSINESS DEVELOPMENT BUSINESS DEVELOPMENT SERVICE BUSINESS DEVELOPMENT SERVICES BUSINESS ENVIRONMENT BUSINESS NEEDS BUSINESS OWNER BUSINESS OWNERS BUSINESS PLAN BUSINESS REGISTRATION BUSINESS SERVICE BUSINESS SERVICE PROVIDERS BUSINESS SERVICES BUSINESS TOOLS BUSINESS TRAINING BUSINESSES BUYERS CAPITAL EXPENDITURE CAPITAL INVESTMENTS CAPITAL MARKET CERTIFICATE CHAMBER OF COMMERCE CHAMBERS OF COMMERCE COMPANY COMPETITIVENESS CONSULTING SERVICES COST-SHARING CREDIT CONSTRAINTS CREDIT GUARANTEES CREDIT LINE CREDIT MARKET CREDIT MARKET FAILURE CURRENT MARKET PRICE DATA COLLECTION DAY-TO-DAY OPERATIONS DECISION-MAKING DISTRIBUTION SYSTEMS E-COMMERCE ECONOMIC GROWTH EDUCATIONAL BACKGROUND ELECTRICITY ELIGIBILITY CRITERIA ELIGIBILITY CRITERION ELIGIBLE APPLICANTS EMPLOYEE EMPLOYMENT ENTERPRISE ASSISTANCE ENTERPRISE DEVELOPMENT ENTERPRISE GROWTH ENTERPRISE SUPPORT ENTREPRENEURS ENTREPRENEURSHIP EQUIPMENT EQUIPMENT PURCHASES EQUITY INVESTMENT EXCESS DEMAND EXPENDITURE EXPORT DEVELOPMENT EXPORT PROMOTION EXTERNALITIES FINANCIAL SYSTEMS FIRM SIZE FIRMS GENDER GOVERNMENT FUNDING GOVERNMENT INTERVENTION GOVERNMENT OFFICES GRANT PROGRAMS HUMAN CAPITAL IMPACT ASSESSMENTS INFORMATION TECHNOLOGY INNOVATION INNOVATION POLICY INNOVATIONS INTELLECTUAL PROPERTY INTELLECTUAL PROPERTY RIGHT INTEREST RATE INTERNATIONAL BANK INVESTMENT OPPORTUNITIES INVESTMENT PROPOSALS LABOR MARKET LACK OF INFORMATION LEARNING LIMITED ACCESS LOAN LOAN APPLICATIONS LOCAL GOVERNMENTS MANUFACTURER MANUFACTURING MARKET DEVELOPMENT MARKET RESEARCH MARKETING MARKETING STRATEGIES MARKETING STRATEGY MATCHING FUNDS MATCHING GRANTS MEDIUM ENTERPRISE MEDIUM ENTERPRISES MICRO ENTERPRISES MICROENTERPRISES MICROFINANCE MICROFINANCE INSTITUTIONS ONLINE FORMS OPEN ACCESS OUTREACH OUTSOURCING PARTIAL CREDIT PRIVATE FUNDING PRIVATE INVESTMENT PRIVATE SECTOR PRIVATE SECTOR DEVELOPMENT PRIVATE SECTOR FIRM PROCUREMENT PROCUREMENT PROCESS PRODUCT DEVELOPMENT PRODUCTIVITY PROMOTION ACTIVITIES PUBLIC FUNDS RADIO RECEIPTS RED TAPE REGISTRY RESULT RESULTS SEARCH SERVICE PROVIDER SERVICE PROVIDERS SMALL ENTERPRISE SMALL FIRMS SME SME DEVELOPMENT SUPPLIER SUPPLIERS TARGETS TECHNICAL ASSISTANCE TECHNICAL SKILLS TECHNICAL SUPPORT TECHNOLOGY DIFFUSION TECHNOLOGY TRANSFER TELEVISION TELEVISION ADVERTISEMENTS TRAINING PROGRAMS TRUST FUNDS UNEMPLOYMENT USES VENTURE CAPITAL VENTURE CAPITALIST WAGES WEB WORKING CAPITAL AFRICA GENDER POLICY GENDER INNOVATION LAB WOMEN AND YOUTH EMPLOYMENT Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa |
| geographic_facet |
Africa |
| relation |
Policy Research Working Paper; No. 6296 |
| description |
Matching grants are one of the most
common policy instruments used by developing country
governments to try to foster technological upgrading,
innovation, exports, use of business development services
and other activities leading to firm growth. However, since
they involve subsidizing firms, the risk is that they could
crowd out private investment, subsidizing activities that
firms were planning to undertake anyway, or lead to pure
private gains, rather than generating the public gains that
justify government intervention. As a result, rigorous
evaluation of the effects of such programs is important. The
authors attempted to implement randomized experiments to
evaluate the impact of seven matching grant programs offered
in six African countries, but in each case were unable to
complete an experimental evaluation. One critique of
randomized experiments is publication bias, whereby only
those experiments with "interesting" results get
published. The hope is to mitigate this bias by learning
from the experiments that never happened. This paper
describes the three main proximate reasons for lack of
implementation: continued project delays, politicians not
willing to allow random assignment, and low program take-up;
and then delves into the underlying causes of these
occurring. Political economy, overly stringent eligibility
criteria that do not take account of where value-added may
be highest, a lack of attention to detail in "last
mile" issues, incentives facing project implementation
staff, and the way impact evaluations are funded, and all
help explain the failure of randomization. Lessons are drawn
from these experiences for both the implementation and the
possible evaluation of future projects. |
| format |
Publications & Research :: Policy Research Working Paper |
| author |
Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David |
| author_facet |
Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David |
| author_sort |
Campos, Francisco |
| title |
Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa |
| title_short |
Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa |
| title_full |
Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa |
| title_fullStr |
Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa |
| title_full_unstemmed |
Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa |
| title_sort |
learning from the experiments that never happened : lessons from trying to conduct randomized evaluations of matching grant programs in africa |
| publisher |
World Bank, Washington, DC |
| publishDate |
2013 |
| url |
http://documents.worldbank.org/curated/en/2012/12/17103546/learning-experiments-never-happened-lessons-trying-conduct-randomized-evaluations-matching-grant-programs-africa http://hdl.handle.net/10986/12200 |
| _version_ |
1764422207849627648 |
| spelling |
okr-10986-122002021-04-23T14:03:05Z Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David ACCESS TO CREDIT ACCESS TO INFORMATION ACCESS TO NETWORKS ACCOUNTING ADVERTISING APPLICATION PROCESS AUDITING BACK-UP BANK LENDING BANK LOAN BANK LOANS BANKS BLOG BRANCH BUSINESS APPLICATIONS BUSINESS COMMUNITY BUSINESS DEVELOPMENT BUSINESS DEVELOPMENT SERVICE BUSINESS DEVELOPMENT SERVICES BUSINESS ENVIRONMENT BUSINESS NEEDS BUSINESS OWNER BUSINESS OWNERS BUSINESS PLAN BUSINESS REGISTRATION BUSINESS SERVICE BUSINESS SERVICE PROVIDERS BUSINESS SERVICES BUSINESS TOOLS BUSINESS TRAINING BUSINESSES BUYERS CAPITAL EXPENDITURE CAPITAL INVESTMENTS CAPITAL MARKET CERTIFICATE CHAMBER OF COMMERCE CHAMBERS OF COMMERCE COMPANY COMPETITIVENESS CONSULTING SERVICES COST-SHARING CREDIT CONSTRAINTS CREDIT GUARANTEES CREDIT LINE CREDIT MARKET CREDIT MARKET FAILURE CURRENT MARKET PRICE DATA COLLECTION DAY-TO-DAY OPERATIONS DECISION-MAKING DISTRIBUTION SYSTEMS E-COMMERCE ECONOMIC GROWTH EDUCATIONAL BACKGROUND ELECTRICITY ELIGIBILITY CRITERIA ELIGIBILITY CRITERION ELIGIBLE APPLICANTS EMPLOYEE EMPLOYMENT ENTERPRISE ASSISTANCE ENTERPRISE DEVELOPMENT ENTERPRISE GROWTH ENTERPRISE SUPPORT ENTREPRENEURS ENTREPRENEURSHIP EQUIPMENT EQUIPMENT PURCHASES EQUITY INVESTMENT EXCESS DEMAND EXPENDITURE EXPORT DEVELOPMENT EXPORT PROMOTION EXTERNALITIES FINANCIAL SYSTEMS FIRM SIZE FIRMS GENDER GOVERNMENT FUNDING GOVERNMENT INTERVENTION GOVERNMENT OFFICES GRANT PROGRAMS HUMAN CAPITAL IMPACT ASSESSMENTS INFORMATION TECHNOLOGY INNOVATION INNOVATION POLICY INNOVATIONS INTELLECTUAL PROPERTY INTELLECTUAL PROPERTY RIGHT INTEREST RATE INTERNATIONAL BANK INVESTMENT OPPORTUNITIES INVESTMENT PROPOSALS LABOR MARKET LACK OF INFORMATION LEARNING LIMITED ACCESS LOAN LOAN APPLICATIONS LOCAL GOVERNMENTS MANUFACTURER MANUFACTURING MARKET DEVELOPMENT MARKET RESEARCH MARKETING MARKETING STRATEGIES MARKETING STRATEGY MATCHING FUNDS MATCHING GRANTS MEDIUM ENTERPRISE MEDIUM ENTERPRISES MICRO ENTERPRISES MICROENTERPRISES MICROFINANCE MICROFINANCE INSTITUTIONS ONLINE FORMS OPEN ACCESS OUTREACH OUTSOURCING PARTIAL CREDIT PRIVATE FUNDING PRIVATE INVESTMENT PRIVATE SECTOR PRIVATE SECTOR DEVELOPMENT PRIVATE SECTOR FIRM PROCUREMENT PROCUREMENT PROCESS PRODUCT DEVELOPMENT PRODUCTIVITY PROMOTION ACTIVITIES PUBLIC FUNDS RADIO RECEIPTS RED TAPE REGISTRY RESULT RESULTS SEARCH SERVICE PROVIDER SERVICE PROVIDERS SMALL ENTERPRISE SMALL FIRMS SME SME DEVELOPMENT SUPPLIER SUPPLIERS TARGETS TECHNICAL ASSISTANCE TECHNICAL SKILLS TECHNICAL SUPPORT TECHNOLOGY DIFFUSION TECHNOLOGY TRANSFER TELEVISION TELEVISION ADVERTISEMENTS TRAINING PROGRAMS TRUST FUNDS UNEMPLOYMENT USES VENTURE CAPITAL VENTURE CAPITALIST WAGES WEB WORKING CAPITAL AFRICA GENDER POLICY GENDER INNOVATION LAB WOMEN AND YOUTH EMPLOYMENT Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the risk is that they could crowd out private investment, subsidizing activities that firms were planning to undertake anyway, or lead to pure private gains, rather than generating the public gains that justify government intervention. As a result, rigorous evaluation of the effects of such programs is important. The authors attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case were unable to complete an experimental evaluation. One critique of randomized experiments is publication bias, whereby only those experiments with "interesting" results get published. The hope is to mitigate this bias by learning from the experiments that never happened. This paper describes the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delves into the underlying causes of these occurring. Political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in "last mile" issues, incentives facing project implementation staff, and the way impact evaluations are funded, and all help explain the failure of randomization. Lessons are drawn from these experiences for both the implementation and the possible evaluation of future projects. 2013-01-29T17:53:31Z 2013-01-29T17:53:31Z 2012-12 http://documents.worldbank.org/curated/en/2012/12/17103546/learning-experiments-never-happened-lessons-trying-conduct-randomized-evaluations-matching-grant-programs-africa http://hdl.handle.net/10986/12200 English en_US Policy Research Working Paper; No. 6296 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo/ World Bank World Bank, Washington, DC Publications & Research :: Policy Research Working Paper Publications & Research Africa |