Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | charles-strickland |
View: | 217 times |
Download: | 1 times |
Governance and social service delivery: An agenda for research
and action
Buenos AiresSeptember 29, 2009
Ariel FiszbeinChief Economist, Human Development
World Bank
The challenge of achieving human development results
•More inputs ≠ better outcomes•Outcomes = f (inputs; technology, behaviors; context)
•Complex systems: outcomes influenced by decisions of many actors
Outline
1. Conceptual framework
2. Reform strategies
3. Indicators and measurement tools
4. An agenda for research and action
1. Conceptual framework
Governance of service delivery
•Governance: rules under which actors interact
•Can be seen as set of principal-agent relations defined by the:▫Incentives facing agents▫Accountability mechanisms available
to principals
Client Power
Voice/
Politic
s
Compact/Policies
Politicians & Policymakers
ServiceProvidersCitizens/Users
6
A set of principal-agent relations…
..characterized by … (Dixit 2002)
•Moral Hazard: Principal cannot attribute outcomes to (often unobserved) actions of agent –e.g. role of teacher effort in learning outcomes
•Adverse Selection: Agents have private information on their ‘type’ –e.g. information on real costs at the health clinic level
•Costly Verification: Outcomes are not easily observable –e.g. quality of medical care/teaching
Public policy as a function of provider & user/citizen behavior (Le Grand 1997, 2003)
•Provider motivation: Knights or Knaves?▫Knaves: self-interested, extrinsic motivation▫Knights: altruistic, intrinsic motivation
•Agency: Capacity for action and choice among citizens/service users▫Pawns: Passive▫Queens: Active
Ag
en
cy
Motivation
Queens
Pawns
Knights Knaves
MarketQuasi-Market
“Mom &
Dad”
Improve agency (information, client power)
Improve provider incentives (Pay-for-performance,
autonomy)
2. Reform Strategies
Reform Intended Goals Examples
I. Improve citizen/user access to information
•Influence policy•Enable informed user decisions•Make providers accountable
•Right-to-information•Information campaigns•Score cards
II. Link provider pay to performance
•Focus teachers on improving student learning• Focus providers on quantity & quality of care
•Teacher bonuses linked to student performance•Health providers paid according outputs and outcomes
Reform Strategies
I. Improving access to information • Are better informed
citizens able to influence decisions by policy makers?
• Can information empower users to: (1) make better decisions as consumers and (2) demand better service from providers?
Voice/
Politic
s
Politicians & Policymakers
Citizens
Client Pow
er
Providers
Users
Are better informed citizens able to influence decisions by policy makers?
•Providing information is low cost intervention
•When is information the binding constraint?
•Is political system permeable?
Examples
• Newspaper campaign in Uganda
• Recurso Peru
Newspaper Campaign in Uganda
– In 1995 in Uganda, only 13 percent of non-wage recurrent spending on primary education reached primary schools.
– From 1996-2001, amounts and dates of grant to local governments published in local and national newspapers.
Voice/
Politic
sPoliticians &
Policymakers
Citizens
Schools in Uganda received more of what they were due, post-intervention:
Source: Reinikka and Svensson (2001), Reinikka and Svensson (2003a)
IMPACT ON LEAKAGE:A school close to a newspaper outlet experienced 25 percentage points less leakage compared to a school one standard-deviation (30 km) further away from a newspaper outlet.
IMPACT ON LEARNING:A one standard deviation reduction (30 km) in distance led to a 0.09 standard deviation increase in average test scores.
Recurso – Peru • In 2005 Peru instituted a series
of interventions to increase transparency and accountability in the social sectors.
• However, despite an increase in stakeholder participation, quality of services remained stagnant. Is information a binding constraint?
•Recurso produced instruments (including videos distributed by television stations and media) designed to educate stakeholders about benchmarks in education and nutrition.
•Universal testing introduced for 2nd grade students, with results being provided back to parents starting in 2008
•Nutrition standards established and provided to communities.
Voice/
Politic
s
Politicians & Policymakers
Citizens
Can information empower citizens as users?
• In decentralized settings where state has low capacity, community monitoring could be essential
• Can community monitoring be promoted at low cost?
• When there is choice, can information influence demand for services?
Examples
• Community monitoring of health services in Uganda
• Community information campaign for education in India
Citizen report cards for health in Uganda• 2001 all public health services become free-of-charge.
Dispensaries are controlled at the district level.• What’s the effect of providing information on service
delivery status relative to other providers and gov’t standards to users?
Impact of Citizen Report Cards: •.17 z-scores increase in weight of infants and 33% reduction in infant mortality.
• Utilization of services was 16% higher in treatment facilities than controls.
Impact of Community Information Campaigns:
(Three Indian States: Karnataka , MP, UP)
•India decentralized school management to local level and established ‘village education committees’
•Can information in the hands of communities help in making schools better and improve learning outcomes?
Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.
School Community Information Campaigns
Campaigns to inform the community about its oversight roles in public schools & the services to which children are entitled. 8-9 meetings in a village over 2 months
Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.
Client P
ower
Providers
Users
Impact of Information Campaign - On Educational Inputs & Student Learning:
REGION Teacher Attendan
ce
Teacher
Activity
% Receiv
ed Unifor
m
Participation rate of parents in school inspections
Scholarships
Students received
Improvement in
Reading
UP 11 % Unchanged
18 % (girls)
.11 33 %(general caste)
27 %(Grade 3)
MP Unchanged 30% 14 % (general caste)
.02 26 %(SC/ SC)
14 %(Grade 3)
Karnataka
Unchanged Unchanged
N/A Unchanged N/A 15 %(Grade 4)
Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.
Strategy IILinking pay to provider
performance
1) Can paying health providers for results instead of inputs result in improved quality and quantity of health care?
2) Will linking teacher pay to student performance increase achievement?
Compac
t/Polic
ies
Politicians & Policymakers
Service Providers
Results-based financing (RBF) ≈ Pay-for-performance (P4P)
Provision of payment for the
attainment of well-defined
results
Transfer of money or material goods conditional on taking a
measureable action or achieving a predetermined performance target
(CGD, 2009)
DonorCentral governmentLocal governmentPrivate insurer
$Recipients of careHealth care providersFacilities / NGOsCentral governmentLocal governments
RBF takes many forms…
Payers Payees
Can the introduction of pay for performance for patient services impact the quality and performance of hospitals/health centers?
• Impact on equity of services?
• Will quality be affected by workers focusing on quantity of services provided?
• Can facilities maintain an adequate amount of autonomy?
• Issues of autonomy for facilities.
Impact Evaluations:
RBF in Argentina’s Health Sector: Plan Nacer
• PN (launched 2002) introduces RBF for maternal and child health care. Expanded to all provinces in 2006.
• Performance agreements made between Nation and Provincial Ministries of Health; & between provinces and providers;
• Capitation payments provide incentives for providers to expand coverage;
NACION
CO
NV
EN
I O
MA
RC
O
PROVINCIA
CO
MPR
OM
I SO
DE
GE
ST I
ON
EFECTORES
INSCRIPCIÓN Validación final de
Padrones Nacionales
Identificación e Inscripción
Administración Padrón Provincial
Identificación e Inscripción
NOMENCLADOR Diseño Valoración Brinda Prestaciones y las
factura
FINANCIAMIENTO
Paga Cápita: 60% por inscripción 40% por
cumplimiento de resultados sanitarios
Administración Cuenta Única
Paga prestaciones
Utiliza los fondos en: Recursos humanos Equipamiento Infraestructura Insumos
AUDITORIA
Padrón de Beneficiarios
Trazadoras Prestaciones
Prestaciones Registro en Historia
Clínica
•Accountability ensured through concurrent audits;
•Social accountability through communication campaigns, beneficiary satisfaction surveys.
•How does the introduction of RBF impact health indicators for pregnant women and children from 0 to 6 years of age?
•Does the program impact the coverage and quality of primary care for pregnant women and children from 0 to 6 years of age?
• What is the impact on service delivery quality? And, how can the incentive structure be maximized to promote improved service delivery and health outcomes?
Impact Evaluation of Plan Nacer
Other eligible Children
- Questions about the type, quality and satisfaction of services received andname of the health care agent providing care
Mother
Last Child Born (aged between 1 and 12
months)
Managers
Doctors
Exit Poll
- Information on the newly-born child shall be provided by the mother or personin charge of nursing the child
House
hold
Surv
ey
- Household usual socio-demographic indicators. History of pregnancies andchildbirths
Speci
fic
Medic
al R
eco
rds
- Questions related to three-four hypothetical visits to learn about good practice protocols.
Medical records recovery at the PHC Facilities
- Measuring encephalic perimeter
- Measuring height
- Measuring weight
Specific measures taken in the field (to the mother
and the latest born child)
- Weight and height
- Contraceptive methods
- Last visit of the child, PHCC he attended, name of the health agent
- Anemia test (Hemo cue)
- Child's APGAR index
- General features of the center (staff, infrastructure, equipment, inputs, resources, expenditures and sources of funds)
- Demographic and Labor and professional qualifications
- Questions related to their attendance to the center
- Socio-economic questions
- In PHCCs, number of consultations per type of patient (age, gender) and reason, per month during the last year.
- Reconstruction of medical care received
Serv
ice P
rovid
ers
- Mother's aerobic capacity; contraceptive methods; practices of hygiene and healthy behavior
- Information on the latest pregnancy and childbirth
- Weight and height of the mother and the child, child's APGAR index; mentalhealth
- Checklist all the contents of the medical records
Plan Nacer Questionnaire 2008
Advance from Rwanda Evaluation: Gertler, Basinga et al
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
InstitutionalDelivery
Tetanus Preventive 0-23months
Preventive 24-59 months
2006
08- no PBF
08 - PBF
RwandaMonetary incentives to health center
conditioned on 14 maternal and child health outputs
Payments allocated at discretion of management
Can merit pay for teachers improve student achievement?
Concerns
• Unintended consequences (e.g. “teaching to the test”, cheating)
• How will it affect teacher motivation?
• Attribution?• Political feasibility?
Examples
•Teacher Incentives in Pernambuco, Brazil
•Teacher Pay in Andra Pradesh, India
Teacher Pay in India – The Context• Very low levels of learning in India
-~60% of children aged 6-14 in India cannot read a simple paragraph, though 95% enrolled in school (PRATHAM, 2008)
• Large inefficiencies in delivery of education -25% teachers absent, less than half are teaching -Over 90% of non-capital spending goes to teacher salaries
-Higher ‘levels’ of pay not associated with better performance
-Strong unions, almost impossible to fireCan linking pay to performance improve school quality?Identification of the causal impact of teacher performance pay is a central limitation
Teacher Pay Incentives in Andra Pradesh (Muralidharan and Sundararaman, 2008)
State introduces three policies:• Group and individual pay incentives based on
average improvement in test scores of all students• Block grants to schools (extra teachers, more inputs)• Assessments conducted by an independent NGO
Impact evaluation results (two rounds):• Incentives increase student performance by 0.22 SD• 1st year both incentives work well. 2nd year individual
incentives work better• Additional inputs improve scores only by 0.08 SD• Teachers liked the program
Teacher Incentives: Pernambuco, BrazilBackground•School attendance increased. Quality still low. •Weak pay incentives (flat salary scale determined by tenure)•Low teacher motivation and high absenteeism•Poorly qualified teachers (do not pass student exams).•Pernambuco one of the lowest•performing states
Teacher Incentives - PernambucoNon-linear performance pay system (July 2008):
•School goals for achievement in Math and Portuguese by grade• Staff reaching at least 50% of the goal get bonus (amount varies with salary & % of goal achieved)
Impact evaluation examining teachers responses:
• Strength of incentives (measured by the targets set by an education quality index)?• Do teachers that come close to the bonus get encouraged or discouraged?• Sorting of teachers across schools?
3. Indicators and Measurement Tools
A measurement framework
Measuring Governance Performance
Many of these indicators can be collected through Quantitative Service Delivery Surveys….
• The facility is used as the unit of analysis▫ Could be complemented with a household/users survey
• QSDS collect quantitative information about▫ Physical infrastructure▫ Staff characteristics▫ Income and expenditures▫ Governance and management▫ Characteristics/Quality of service provision▫ Outcomes
Measuring quality of medical care – Medical Vignettes
Getting at Clinical Quality
Knowledge or “Competence”
of Doctors
Behavior – Actual
Performance – of Doctors
0
.1.2
.3.4
% W
ho a
sked
the
rele
vant
que
stio
n
Private MBBS Private, No MBBS Public
...And What They DoWhat They Know
% Asked (DCO) % Asked (Vignettes)
What Doctors Know… and What They Do – Evidence from India
Das and Hammer 2006
Stallings Classroom Observation Instrument
Trained enumerators take 1 minute “snapshot” of class every 6 minutes. Repeat on 2 successive days. Revisit every two months. Being utilized in the Pernambuco evaluation.
Measuring quality of teaching?
Percent of time officially allocated to schooling; when a teacher is present; and spent in teaching and learning activities
Beyond absenteeism: Effective supply of teaching
Sources: Egypt, Yemen and Lebanon from Lane and Millot (2002); Tunisia, Pernambuco, Morocco and Ghana from Abhadzi, Millot and Prouty (2006); Cambodia from Benveniste, Marshall and Caridad Araujo (2008); and Laos from Benveniste, Marshall and Santibanez (2007).
Percentage of Time Use in 4th Grade (common for all countries)
Country Discipline %
Interactive Learning
%
Passive Learning
%
Organizational Management
(non-teaching)
%
Pernambuco (Brazil)
1.75 52.89 18.61 25.50
Ghana 1.44 52.50 12.50 35.00
Morocco 4.20 62.85 20.03 17.78
Tunisia .88 61.70 26.32 11.98
Abadzi, Millot and Prouty (2006)
4. An Agenda for Research and Action
Where are the opportunities and what are the main challenges?
•Measurement: Inter and intra-national benchmarking
•Experimentation and evaluation•International platforms for information
exchange on reform strategies•Entry points for reform: matching reform
to politics
END
Absence rates among teachers and health workers
Note: Surveys were all fielded in 2002 or 2003. Sources: Chaudhury et al (2006) except for PNG, World Bank (2004) and Zambia, Das et al (2005).
0
10
20
30
40
50
Bangladesh Ecuador India Indonesia Papua NewGuinea
Peru Zambia Uganda
Primary schools Primary health facilities
Disbursed public spending on school grants that actually reach schools
Percent
GNI per capita (2000)
GNI per capita PPP
(2000)
Ghana 1997/98 51 330 1880
Kenya 2004 (secondary school bursary funds)
78 250 810
Madagascar 2002 88 2050 4610
Peru 2001 (utilities) 70 / 97 670 2280
PNG (2001/2002) 72 / 93 280 510
Tanzania 2002-2003 62 270 1250
Uganda 1991-1995/2001 <20 / 80
Zambia 2001-2007 (discretion/rule) 24 / 90 320 740Ye and Canagarajah (2002) for Ghana; Republic of Kenya (2005) for Kenya; Francken (2003) for Madagascar; Instituto Apoyo and World Bank (2002) for Peru; World (Bank 2004)
for PNG; MOF, Government of Tanzania (2005) for Tanzania; Reinikka and Svensson (2005) for Uganda; Das et al. (2002) for Zambia.
Percent of school grants that actually reach schools
Disbursed public spending on school grants and funds allocated for health facilities that actually reached intended recipient/facility
EducationPercent
Health Percent
Ghana 1997/98 51 20
Kenya 2004 20 62
Madagascar 2002 88 --
Peru 2001 (utilities) 70 / 97 --
PNG (2001/2002) 72 / 93 --
Tanzania 2002-2003 62 59
Uganda 1991-1995/2001 <20 / 80 --
Zambia 2001-2007 (discretion/rule) 24 / 90 --
Chad 2003 (regional/local health centers)
-- 27/ <1
Ye and Canagarajah (2002) for Ghana; Republic of Kenya (2005) for Kenya; Francken (2003) for Madagascar; Instituto Apoyo and World Bank (2002) for Peru; World (Bank 2004) for PNG; MOF, Government of Tanzania (2005) for Tanzania; Reinikka and Svensson (2005) for Uganda; Das et al. (2002) for Zambia; Wane for Chad (2004).