Date post: | 20-May-2015 |
Category: |
Health & Medicine |
Upload: | rbfhealth |
View: | 158 times |
Download: | 6 times |
1
Zimbabwe Results Based Financing
ProjectEvidence and Lessons
Brigadier General Dr. Gwinji, MOHCCDr. Goverwa -Sibanda, MOHCC
• Key messages, strategic context and overview of RBF program• Brigadier General Dr. Gwinji, MOHCC
• Preliminary results from impact evaluation and process evaluation• Dr. Sibanda, MOHCC
Outline of Presentation
• RBF is a health systems management tool intended to improve the efficiency of utilization of system inputs
• RBF in Zimbabwe piloted in 2 front runner districts in July 2011, then scaled up to 16 additional districts in March 2012
• Contextual background: Dramatic improvements in MCH indicators witnessed throughout the country (MICS 2014)
• Yet faster rates of improvement in RBF districts for key indicators• 13 percentage point improvement in the in-facility delivery rate• 12 percentage point improvement in post-natal care coverage• Significant improvement in the quality of ANC services
• Not all indicators show relative improvement under RBF• No differential gain in coverage of ANC services• Small gain in use of modern contraceptives
Main Takeaway Messages I
• Improvements in part due to• Team based incentives facilitating teamwork• Regular and structured supervision visits yielding feedback to improve
performance• Enhanced community participation
• Challenges to be addressed include• Facilities in remote areas or with small catchment populations• Capacity to fully operationalize the quality of care components• Demand side barriers related to religious and socio-economic factors
Main Takeaway Messages II
• Population: 13 million (2002 Census estimates) • Decline in public sector financing • Effect on management• supportive supervision
• Increase in household out-of-pocket expenditures due to various forms of user-fees
• Decline in outcomes & slow progress on some key health MDGs [MDGs 4 & 5]
Country & Project Context During Design Phase [2010-2011]
RBF aligned with and supports National Health Strategy and policy–equity in access to health services;◦ User fee removal (package of high impact services)◦ Rebuild the quality of care standards◦ Increase access to priority maternal, family planning and child
health services◦ Strengthen the referral system (promotes appropriate care
seeking at appropriate levels)◦ Decentralized service delivery and revitalized primary health care
Prioritized package of services directly linked to burden of disease for mothers, newborns and children under 5
RBF used to operationalize GoZ Results-Based Management Strategy and Results-Based Budgeting Pilot
RBF in the Zimbabwe Context
• Fee-for-services : for both quality and quantity – partial replacement of user fees
• Functions separated: purchaser, provider, regulator & external verifier
• Key role for community –Health Center Committees
1. Results-Based Contracting
• Strengthening planning and RBF management capacity: RBF national management team
• Purchasing, verification, strategic management
2. Management and Capacity Building
• Capture effect on health outcomes and various aspects of the health system
• Accountability through community tracer surveys (CBOs)
3. Monitoring and Documentation
Three Project Components
Rural Health Centers
District Hospital
Package of RBF Services
9. Tetanus TT2+10. ARVs to HIV+ preg. Women (PMTCT)11. Family planning short and long term
methods12. High risk perinatal referrals13. Vitamin A supplementation14. Children fully immunized15. Growth monitoring, children < 5yrs16. Cure discharged acute malnutrition
children < 5yrs (October 2012)
1. OPD new consultations2. First ANC visit during the first 16 weeks
of pregnancy (October 2012)3. Ante natal care 4 visits completed4. Post natal care 2 or more 5. Normal deliveries6. HIV VCT in ANC7. Syphilis RPR test8. IPT (x2 doses)
1. Normal deliveries in district hospital2. Deliveries with complications (caesareans excluded) and post partum
complications3. Caesareans performed4. Family planning: Tuba Ligations5. Counter referral note arrives at RHC (October 2012)
9
Data Quality and Verification
Verified data from health facility registers collected by Local Purchasing Units (LPUs) and entered in the RBF database is utilized for payments
Data flow integrated within national HMIS, no parallel system used
Quantity verification by the LPU undertaken every month
Quality verification by the DHE (for RHCs) and by PHE (for DH) undertaken every quarter
Client satisfaction performed by the CBOs every month and by the counter-verifier every quarter
04/12/2023
HMIS - DHIS 2
District Facility
Programme Database
HFOs
≈
Routine Reporting - T-SeriesEntry and Submission into DHIS2
Data Accessible in Programme
System:Same
Platform RBF Indicators Downloaded
Verified Data Entered Into System
Payment to Facil
ityUpdating
HMIS with Verified Data Verificati
on
Interaction between Programme Data and HMIS
04/12/2023
RBF Financing & Coverage
11
RBF Funding DFID and Government of Norway (US$35m) Ministry of Finance: US$5 million per year from 2014 US$ 28 million disbursed (including $5 million Government
counterpart funding) Population Coverage : 4,1 million Geographic coverage : 18 rural and 2 low-income
urban and peri-urban districts (Harare and Bulawayo)
July, 2011 to October, 2015
04/12/2023
Contract
Tracing clients and client satisfaction
Policy and Supervision
Policy and Supervision
Policy and Supervision
MoHCW
Provincial Health Executive
Health Facilities and HCC (415)
District Health Executive
National Steering Committee
District Steering Committee
CORDAID
Private Purchasing Agency (NPA)
Contract + Payment
Payment
Community Based OrganisationsClients
CORDAID
Local Purchasing Unit
Payment
Contract + Verification
Contract
External verification12
Governance & Institutional Arrangements
Impact and Process Evaluations: Answering Policy Relevant Questions
13
Baseline(2011)
Midline Impact Evaluation(Mar-Sept 2014)
Program Inception
Endline IE(TBD)
Process Monitoring and Evaluation (PME)(November 2013)
Routine Performance Review (Quarterly) – Operational Data
Technical Review(June 2012)
Mid-Term Review(January 2013)
Technical Adjustments: Prices and Services
Technical Modifications –clinical quality, streamlining verification, equity monitoring
2nd PME Round Planned for October 2014
14
The IE seeks to determine the causal impact of RBF on priority service utilization and related health indicators
◦ Treatment: Facilities and patients residing in districts that introduce the RBF program
◦ Comparison: Facilities and households in matched “business as usual” districts
◦ Districts matched on various characteristics including: Average catchment size of facility Proportion of staff positions filled Population rates over 2008 – 2010 of ANC coverage, in-facility
delivery rates, immunization coverage
Evaluation Design I
Participating Districts
16
Given the purposive selection of study districts, the evaluation must be quasi-experimental. Specifically,
◦ A difference-in-difference (“diff-n-diff”) estimator between matched districts in treatment and control (16 in each arm) estimates program impact
◦ To estimate actual impact: two years of program exposure (2012-2014) is contrasted with a two year period immediately (2008-2010) before the program
Evaluation Design II
17
Household information – Population representative surveys of health behavior,• including utilization, recall of procedures health outcomes,• including anthropometry, satisfaction with care, knowledge and mediating variables• socio-demographics
Baseline data for community and household utilized the 2011 DHS Follow up data at community, household structured to replicate and
supplement the earlier DHS Yields a sample of ~2800 recent pregnancies/births
Data Sources I
18
Facility survey – a comprehensive review of the structure, provision, and quality of care at clinic level Instruments
◦ Facility checklist◦ Health worker tool ◦ Exit interview tool (ANC, child illness) ◦ Direct observation (ANC, labor and delivery and child illness) ◦ Chart audit* of routine and complicated delivery• Only in the follow up
180 facilities in 2011 baseline (the NIHFA) and 231 in follow up
Technical support from USAID and UNICEF was critical in this undertaking
Data Sources II
19
• Data has just been collected and still being processed• However the preliminary results from both population
and facility data are now available…• Overall it is a story of strong gains in select health
indicators for the entire nation (consistent with the MICS results), with yet more rapid improvement in RBF districts
• RBF gains in both the quantity and quality of care, but not for all prioritized indicators
Preliminary Results I
20
• RBF led to gains in both the quantity and quality of care• A 13 percentage point increase in the in-facility delivery
rate• A 12 percentage point increase in post-natal care
coverage• More women receiving full package of ANC services
including urine tests, blood tests, tetanus shots• But not for all prioritized indicators• Little change in ANC coverage and contraceptive use–
baseline rates already high
Preliminary Results II
Dif-n-dif in Practice: Receipt of Urine Test during ANC Care
Start of RBF
2008 2009 2010 2011 2012 2013 20140.4
0.45
0.5
0.55
0.6
0.65
RBF pre-trend
RBF trend
comparison pre-trend
comparison trend
22
Outcome ImpactLevel of
significance
Any modern contraception 0.05 0.21Receipt of any antenatal care (ANC) 0.02 0.43Time of first ANC -0.19 0.22Number of ANC visits 0.40 0.15Blood pressure checked during ANC 0.03 0.55Urine sample collected during ANC 0.16 0.02Blood sample collected during ANC 0.08 0.14Receipt of tetanus vaccine during pregnancy 0.08 0.05Number of tetanus vaccine 0.33 0.05Any iron supplements received during pregnancy 0.00 0.99Receipt of any postpartum care (PPC) 0.12 0.05A PPC within 2 months of delivery 0.12 0.06PPC by a skilled provider 0.14 0.02Facility delivery 0.13 0.00Skilled delivery 0.14 0.00
Summary Regression Measures of Impact
Most impacts measured in proportional terms. Statistically significant results with in red.
23
Complementary Results from Facility Level Data
Patient exit interviews◦ ANC interviews (n=1105) indicate significant improvements in care
processes such as measured abdomen and urine sample◦ Child health interviews (n=1612) indicate significant improvement in
measurement and growth monitoring◦ Client satisfaction is also higher (although at marginal significance,
p=.128) Facility measures◦ Improvements in select facility conditions after RBF:
8 percentage point increase in availability of bio-med waste disposal◦ Increases in supervision and community involvement after RBF:
Increase in number of HCC meetings (2.7 more per year) Increase in external assessments of staff (2.4 more per year) 34 percentage point increase in presence of facility work plan
Moving Beyond the ‘Black Box’
RBF Investments ProgramOutcomes
25
Need to go beyond numbers ◦ Capture rich experiences and lessons from frontlines of PBF
implementation & context Community engagement and support (HCC) Geographic influences (supervision aspect, cross-catchment area
patient movement) Health facility management skills and dynamism Extent of mentorship and clinical supervision by district/province
Explain contextual factors that matter the most & account for variation in provider performance
Supporting impact evaluation and not substituting it –better understanding of the our intervention
Process Monitoring & EvaluationRationale
Providers at high performing health facilities (HPF) were reported to be highly motivated.
Regular and well structured supportive supervision improved the capacity to earn more subsidies e.g one (HPF) had an increase in subsidies of 53% to 94% between the first and second quarters.
Team work improved communication...we are now working as a team to ensure that the T5 (data
compilation form) is correct and accurate and we are able to send it to the district in time. We are motivated to work as we get paid for our effort. --( HF Staff Member)
Higher client satisfaction attracted more patients (patient choice)
Select Findings from Qualitative Study
When low performance is found, facilities attribute reasons to:Smaller catchment populations & at times
geographic remoteness Irregular supportive supervisionSocio-economic and religious factors in the
population
Challenges Affecting HF Performance
28
Revisit the remoteness criteria & facility catchment issue RBF performance payment calculation formula being
reviewed (higher remoteness bonus) Content & structure of quality supervision checklists
Written feedback by district managers mandatory Modified supervision checklist – more process measures
of clinical care introduced District Health Executive supervision contract revisited –
ensure incentive to regularly supervise remote facilities Integrate RBF into HR management
Policy Actions Considered & Next Steps
29
Thank You!
30
Extra slides
31
What is the impact of RBF on maternal and child health services utilization and outcomes?
What is the impact of RBF (counter verification) on health management information system (HMIS) and on supportive supervision?
What is the effect of RBF on community participation and social determinants of health?
What is the effect of RBF on health workers’ attitude, job satisfaction, retention and attrition, etc?
What is the effect of RBF on patient/client satisfaction and in health seeking behavior?
Policy Relevant Questions - Zimbabwe
32
Given the purposive selection of study districts, the evaluation must be quasi-experimental. Specifically,
◦ A difference-in-difference (“diff-n-diff”) estimator between matched districts in treatment and control (16 in each arm) estimates program impact In a diff-n-diff estimator, the observed trends in outcomes for the
comparison districts stand for what would have happened in the RBF districts if not for the RBF program
The validity of this approach can be assessed by comparing pre-RBF trends of same outcomes in RBF and comparison districts before RBF program
To estimate actual impact: two years of program exposure (2012-2014) will be contrasted with a two year period immediately (2008-2010) before the program
Evaluation Design II
Dif-n-dif in Practice: In-facility Delivery Rate
Start of RBF
2008 2009 2010 2011 2012 2013 20140.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
RBF pre-trend
RBF trend
comparison pre-trend
comparison trend
Dif-n-dif in practice: Rate of ANC coverage (any ANC)
After introduction of RBF
2007 2008 2009 2010 2011 2012 20130.88
0.9
0.92
0.94
0.96
0.98
1
RBF pre-trend
RBF post-trend
comparison pre-trend
comparison post-trend
35
Key areas PME assessed
• Factors affecting health provider performance
• Factors influencing changes on the demand (community) side
• Compliance of the RBF stakeholders to the project’s implementation guidelines.
Moving beyond the ‘Black Box’ II
PME Study sites
Binga, Chipinge, Kariba, Mazowe and Zvishavane.
Selected three facilities in each of five districts
Sequential mixed method deployed
Participants selected included mothers of reproductive age, male community members, influential leaders, health center committee members, district management team, as well as clinic health staff
Sampling & Approach
38
RBF:◦ Strengthens relationships between health care providers &
communities and improves access to services at community level (e.g. community ambulance)
◦ Fosters innovations and entrepreneurship among health workers◦ Improves district managers supportive supervision◦ Improves health facility infrastructure (service delivery environment)◦ Improves staff morale and promotes team work
Summary of Qualitative Evidence
Staff shortage “When one of us attends a workshop only one person is left to deal with registers and the workload is huge and most of the mistakes we make are due to fatigue. We are now losing patients to other institutions…”HF staff
“There are a number of clinics surrounding therefore there is competition for example X clinic is GVT owned and well equipped, has drugs and well-staffed so patients prefer going there than spending more time in long queues here where there is staff shortage”. -- HF Staff Member Shortage of drugs e.g. vaccines Inadequate infrastructure
Challenges (cont.)