What do we know about improving health in Punjab? And

some lessons from IndiaDr. Jeffrey Hammer

Princeton University

IGC – CDPR Seminar, IslamabadFebruary 12, 2015

2

What do we know? Possible answers

• First is “not a lot”

• Second is: Maybe something from India? Or from theory and logic?

• Third is: Maybe something from surveys done in Pakistan?

• Fourth is: Not really. as I undercut answer three above with examples of peculiarities in Pakistani data.

• Can we start getting some reliable data by establishing collection systems?

3

Talk in three parts

• Some evidence from rural areas in India on the relative effectiveness of public and (publicly provided) private goods.

• Some meager (and debatable) evidence on the same thing in Punjab, Pakistan

• Complaint about the state of statistics and a suggestion or two on new opportunities for data sources, collection methods and organization.

4

The Total Sanitation Campaign in Maharashtra

• Program: change behavior to get rid of open defecation, don’t just build latrines

• Evaluation: Randomized Control Trial of promotion of 100% safe defecation practices at village level.

Chaudhury, Ghosh Moulik et al; Hammer and Spears (2014)

5

Challenges of initial design

• Secretary, Rural Development, wanted to try this out in the hardest possible conditions – very poor, very isolated, heavily tribal areas.

• He bet that if it could work there, it could work anywhere.

• Unfortunately, he lost.

• Only in 1 District out of 3 (Ahmednagar) did officials do anything

6

Lessons (before we even start)

• On Intent to Treat grounds it obviously failed. Implementation constraints are critical

• BUT, on Treatment on Treated grounds, things look much better

• So, it might work. But only where it works.

7

Effect of program on latrine coverage – only in Ahmednagar

0.2

.4.6

.81

cu

mula

tive

den

sity

0 .2 .4 .6 .8 1latrine coverage, fraction of village households

treatment control

8

So addressing sanitation (with big externalities) could work!

How about publicly provided primary health care?...

not so much

9

How can this be? How can publicly provided medical care NOT help?

• Vacancies (A budget, not an implementation problem)

• Absenteeism

• Low levels of knowledge and ability of public MBBS’s (sometimes relative to untrained – sometimes not)

• Abysmal effort exerted by public providers such that they can’t possibly find out what’s wrong with you

• Substitutability of small (ish) public sector with a much larger private sector around it

10

All leading to the question: What is the marginal impact of an expanded

public system on the market as a whole?

• On overall usage (quantity)?• On the accuracy of medical advice (quality) in

the market as a whole – public and private?

• This is a very hard problem – we can only address bits and pieces of it

11

“Weak links” in our knowledge

India

• Absenteeism – about 40% in 2003, not much change about a decade later

• Quality of care (knowledge)- Das et al, Das and Hammer – next slide

Pakistan

• ??? – one very good but small study (Hasanain et al) based on an IT intervention, not a national picture

• ??? Not that I know of

12

How bad can quality be?

Madhya Pradesh: Public doctors know more than anyone but put in so little effort, they give the worst advice and treatment

13

Diagnosis and TreatmentAsthma In Madhya Pradesh

0.13

0.20

0.01

0.07

0.32

0.41

0.23

0.03

0.23

0.310.31

0.25

0.04

0.11

0.30

0.39

0.21

0.01

0.27

0.32

Public Private Qualified Unqualified

Perc

ent o

f int

erac

tion

s w

ith

item

co

mpl

eted

RightWrong

14

BUT NONE OF THIS IS KNOWN OR EVEN ASKED IN PAKISTAN

15

Reasons to doubt effectiveness of public sector - almost no one uses it

Punjab 2012 2006Place of treatment Diarrhea Cough / Fever Diarrhea Cough / Fever

Government Hospital 7.89 7.97 6.61 8.61RHC/BHU/FWC 1.97 1.61 9.92 9.27Lady health worker 0.61 0.09 2.48 0.66All public sector 10.47 9.68 19.01 18.54

Private hospital 24.58 26.09 19.83 13.25Private doctor 36.12 36.43 31.40 39.07Other private 13.20 12.05 12.40 11.92All private sector 73.90 74.57 63.64 64.24

Not treated 15.63 15.75 17.36 17.22

Total 100 100 100 100

Public sector if treated 12.4 11.5 23 22.4

Private sector if treated 87.6 88.5 77 77.6

PDHS reported in Afzal, Hammer and Ghaus (2015) Public shrinking?

16

Even in villages where a facility is certainly available

Place of treatment in villages with a public facility available, PDHS 2012 (Punjab)

Diarrhea Cough / FeverGovernment hospital 7% 9%RHC / BHU / FWC 7% 9%Lady health worker 2% 1%

All public sector 17% 18%

Private hospital / clinic 20% 13%Private doctor 31% 39%Other Private 12% 12%

All Private Sector 64% 64%

Not Treated 20% 18%Total 100 100

Private share of those seeking treatment 79% 78%

Down from 88%

17

What shows up as possible correlates with child health in the Punjab?

• Education (of mothers for sure – maybe fathers)

• Standard of living (measured by possessions – not even a direct measurement)

• Maybe, just maybe, open defecation (but not nearly as convincing as we’d like to see)

18

Why is it so hard to show anything?

• Serious data quality issues

• “Errors in variables” larger than variance of variables

19

-20% -18% -16% -14% -12% -10% -8% -6% -4% -2% 0%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

Changes in "open defecation" by district in two data sources

MICS data 2008 to 2011

PDH

S da

ta 2

006

to 2

012

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%0%

10%

20%

30%

40%

50%

60%

70%

80%

Correlation of levels of “Open defecation” 2006 versus 2012 by district (PDHS)

2008

2012

21

Plea for better data

• Massive changes in rich world in type, sources and sizes of available data

• Organized in ways that are either easy to use or, at least, publicly available

• Much is being organized geographically – a continuously lengthening panel of routinely collected data

22

Can we start now to develop general use statistics?

• Could we request donors to ask questions in their surveys that policy makers in Pakistan have discussed and considered important? And maybe ensure quality?

• Could we request researchers to format data so that it can be absorbed into a larger system?

• Could we request ministries to do the same?