The Role of Competition to Improve the Delivery of Social Protection Programs:Evidence from a Randomized Experiment on Indonesia’s Raskin Program

Sudarno Sumarto

The SMERU Research Institute & TNP2K

*Based on Abhijit Banerjee, MIT, Rema Hanna, Harvard University, Jordan Kyle, Columbia, Benjamin A. Olken, MIT,and Sudarno Sumarto, SMERU and TNP2K (fortcoming, Journal of Political Economy): Private Outsourcing andCompetition: Subsidized Food Distribution in Indonesia.

BACKGROUND

Raskin (currently, called Rastra) is the largest Indonesian social assistance program― Introduced during the AFC-1998/99 to help poor and near poor cope with the

economic shocks― 53% of all public social assistance, or US 1.5 billion/year― Beneficiaries (poorest 30% households) entitled to 15 kg per month at a co-

pay of Rp. 1,600 per kg

Central government program but implemented by local officials― Monopoly distribution: Village heads have a monopoly over the distribution of

Raskin into/within village and to beneficiaries

― “Last-mile” of delivery: Local officials pick up several tons of rice from government distribution point each month, distribute to citizens, and collect and remit co-pays

2

MOTIVATION

Delivery of RASKIN program is often ineffective*― High Co-Pays: Households pay Rp. 2,276 per kg on average instead of Rp. 1,600

(42 percent mark-up on co-pay)― Low Quality: Rice is of very low quality, perhaps because of distributor

processing practices― High Leakages: much of the rice goes missing (Olken 2006, World Bank 2012)

GoI (VP- Boediono) interested to build evidence on whether ― Allowing outsiders to bid for the right to distribute Raskin would improve the

distribution

To build this evidence, the National Team for the Acceleration of Poverty Reduction (TNP2K) collaborated with J-PAL and DFAT to conduct a Randomized Control Trial evaluation (RCT).

*note that this data corresponds to conditions circa 2012/2013 3

LITERATURE: PRIVATE VERSUS PUBLIC?

How should the government organize the delivery of its services?― The traditional model is delivery by a government agency that has monopoly power

Large literature (e.g. Hart, Shleifer, and Vishney, 1997) discusses benefits and costs of deviating from this model

― Emphasize, however, that while there may be efficiency gains, product quality may decrease

Even if outsourcing would yield efficiency gains without quality problems, not clear if public would actually benefit

― Without sufficient competition, gains may accrue to contractor, not the public

― Vested interests (i.e. incumbents, unions, etc) may block or subvert selection process to protect or capture rents when reform is needed most (e.g., Levin and Tadelis 2010)

This said, perhaps greater information by intended recipients about the program and how it should function is sufficient/better than increased competition.

4

OUR PROJECT: AN RCT ON ALTERING THE RASKIN DELIVERY MECHANISM

Contracting-out:

― Communities offered the option to contract out local distribution of the Raskin rice

― Conducted through open procurement process with selection made by locally-chosen committee

Contracting-out with enhanced competition:

― In some contracting-out locations, extended the bidding period if 3 bids were not received initially to spur competition

Information only:

― Privatization process involves disclosure about how current government process works

― A third treatment has just this disclosure element, to separate out effects of outsourcing from greater information transparency

5

SUMMARY OF FINDINGS

Outsourcing improves efficiency―Transportation cost falls―Quality does not decline

Outsourcing with sufficient competition reduces the price―But not (really) outsourcing alone

Significant gains, but not huge―Even with enhanced competition, co-pay mark-up reduced by 11 percent on average―And only 20 percent of villages elected to switch distributors

We document that competition reduces co-pay markup the most where it was highest―More entry of new suppliers when baseline rents high – rents attract more competition―But, some evidence of blocking – either preventing bidding from taking place, or blocking the new

distributor from taking over – in areas where baseline distributors appear corrupt―On balance, competition effect dominates – and we find most substantial effects of programs on

reducing really high rents

6

ROADMAP

RCT Design, Setting and Treatment Details

Timeline, Data collection and Estimation Procedures

Experimental Results― Selection among bidders

― Effect of outsourcing

― Does competition have an additional effect?

Conclusion

7

EXPERIMENT DESIGN, SETTING AND TREATMENT DETAILS

8

EXPERIMENTAL DESIGN: RANDOMIZED EVALUATION IN BRIEF

When comparing outcomes between areas that do or do not enact a policy, the outcomes may be due to differences between the areas, not the policy itself

The Raskin Pilot overcomes this challenge by randomly assigning villages to different policies by lottery:

―Example: Some villages randomly assigned to “bidding treatment”, others to “control group”

―Villages in the treatment and control groups were statistically equivalent when the experiment began

Randomization ensures outcomes are due to the policy

9

EXPERIMENTAL DESIGN: STUDY CARRIED OUT IN 6 DISTRICTS

Used previous project’s sample to save time

Randomization unit: village―High enough to avoid

potential spillover

572 villages within 6 Districts*―Pemalang and Wonogiri

(Central Java),―Palembang and Ogan

Komering Ilir (South Sumatera),

―Bandar Lampung and Central Lampung (Lampung)

10

ns

*originally 600 villages. 28 dropped before randomization given security and access issues

EXPERIMENTAL DESIGN

Randomization among 572 (desa / kelurahan) as follows, stratified by district and previous experiment conducted in same locations

11

SETTING: STATUS QUO AND POTENTIAL IMPROVEMENT

Status quo:―Government drops off rice at distribution point – either district warehouse, subdistrict

office, or municipal office

―Village / municipal head is responsible for organizing distribution to beneficiaries – either in municipality, each hamlet, or door to door – and collecting copays

― Unfunded mandate, so municipalities mark up co-pay price to cover transportation costs

―Village / municipal head sometimes delegates to other officials (e.g., sub-head for social welfare)

What could be improved?― Price: how much is co-pay price marked up?

― Quality: rice often poor quality. Can be malfeasance (substitute bad rice for good rice; cut with stones), or laziness (don’t protest if gov’t warehouse gives you moldy rice)

― Other dimensions: How close to beneficiaries? Credit?

― Timeliness

12

13

Government drops off rice at distribution point

TREATMENT DETAILS: BIDDING PROCEDURE

Informational meeting to describe current last-mile process and select committee to administer bidding

―Selected by community meeting – government and municipal legislative officials, informal leaders, and ordinary citizens

Posters to recruit bidders throughout subdistrict

Bidders have 10 days to submit bids―Incumbent distributor allowed to submit a bid (66% of cases)

Bidding committee meets to decide on how to select bids―Open rule, but some suggestions include price, ownership of equipment/capital, bidder

experience, reputation, etc.―Bidding committee rejected non-serious bids (~12 percent)

Bidders present bids, Q&A, allowed to revise bids ―Note: because allowed to revise bids, not strictly a first-price sealed bid auction

Committee selects new distributor or remains with status quo

Re-evaluation meeting held after 6 months

14

15

Recruiting poster and bidding processPosters to recruit bidders throughout subdistrict

TREATMENT DETAILS: WHAT IS IN A BID?

Dimensions of bid in bidding form:― Price

― Timing of required purchase

― Who else would help in distribution

― Distribution location and method

― Plan for those who could not afford to buy

― Plan to get money to pay for rice at distribution center

Additional information:― Biodata

― Relevant experience

16

TREATMENT DETAILS: INDUCING EXTRA COMPETITION

Idea is to induce additional competition where less than 3 bids were submitted

―IO literature showing that 3 is very different from 2 (e.g. Bresnahan and Reiss 1991).

In extra competition locations a minimum of 3 bids was required

―If fewer than 3 bids at end of 10 days, bidding period extended by another 10 days

17

TREATMENT DETAILS: INFORMATION ONLY TREATMENT

Bidding treatment also results in an increase in transparency― Current Raskin distributors describe the distribution process― Local committee evaluates and understands it

Since transparency may also have an effect, we ran an information-only’ treatment in some locations

―Same meetings as bidding – but no opportunity for bidding

―These meetings generate as much discussion as bidding meetings

Analyze effects of bidding relativeto information-only

18

TIMELINE, DATA COLLECTION AND ESTIMATION PROCEDURES

19

TIMELINE

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb

Treatment Implementation

Re-Evaluation

Baseline Survey

Endline Survey

2013 2014

20

DATA COLLECTION

Administrative data: ― Bids, committee meetings information

Survey data:― Household surveys for price, quantity, quality, etc

― Survey of distributors conducted at endline

― Administrative data on bids, committee meetings information

― Survey of village officials

Two other sub-treatments: randomized women onto procurement committees and soft encouragement for committee to also monitor at 3 month mark

21

IMPLEMENTATION

Experienced facilitators engaged directly with the village leaders to facilitate the activity

Intensive Facilitator Activities:―Obtaining buy in from village head

―Explaining the bidding/monitoring activity to village authorities

―Handling questions and objections (with support of village authority)

―Facilitating the formation of local committees

―Preparing committees for activities22

BIDDING MEETING EVALUATION MATRIX

23

ESTIMATING EQUATION

Estimate equations as followsyis = αs + β(BIDDINGorINFO)is + γREGULAR_BIDDINGis + ωENHANCED_BIDDINGyis0

+ φyis0 + ɛis

― i is municipality and s is stratum― BIDDINGorINFO is dummy for receiving either bidding treatment or information

treatment― REGULAR_BIDDINGis is dummy for receiving regular bidding treatments― ENHANCED_BIDDINGis is dummy for receiving bidding with enhanced competition― yis0 is baseline value of y when available

γ and ω are key coefficients for isolating impact of bidding treatments relative to information-only placebo

Report p-values of β + γ = 0 and β + ω = 0 to test bidding treatments vs. pure control and γ = ω to test impact of enhanced competition per se

24

FLOW OF LOCATIONS THROUGH BIDDING PROCESS

25

RESULTS

26

MINIMUM BIDS INCREASES COMPETITION

• Min bids increases number of bidders by 30 percent (2.14 to 2.74) 27

DID THE PROCESS CHANGE DISTRIBUTORS?

28

No evidence of elite capture Having experience as a trader matters

COMMITTEES CHOOSE BETTER BIDS

29

Lower markup prices for the winning bid – about 20% difference in the mark-up Winners propose other services, such as distributing at sub-village level, rather than village But, they are more likely to want to be paid before distribution (as opposed to during

distribution)

OUTCOMES

Improvements in price come from competition

Outsourcing leads to improvements in quality but not price

No gains from information 30

EFFICIENCY GAINS VS REDUCTION IN RENTS?

Transport costs fall under bidding, but note that costs go up in transparency treatment relative to pure control

“Compensation” falls dramatically only with competition – consistent with fall in prices31

CONCLUSION

Contracting out can work – even in a rent-seeking environment― Prices and quality improve, but overall effects are not large, for example,

compared to the ID card experiment, which saw about a 30% increase in subsidy for eligible households

― Localities seem to make broadly sensible decisions among bidders

― And, they seem most likely to replace incumbent when existing rents are high

Key to rent reduction is competition, not outsourcing per se

Magnitudes of savings not enormous in this context – in part because of blocking by vested interests and thus only 20 percent of locations induced to switch distributors

32

33

THANK YOU

ESTIMATING THE CHOICE FUNCTION

34

Price is most important – but committees also care about other characteristics

Credibility― Having experience as a trader is worth

about Rp. 250 / kg in extra markup – about 30 percent of control mean

― Having means of transportation is worth about Rp. 140 / kg in extra markup

Credit – providing credit worth Rp. 127 / kg in additional markup (p-val 0.16).

No evidence of elite capture― Does not seem to be elite advantage once

you condition on bid characteristics