ASIAN DEVELOPMENT BANK PCR: CAM 27410€¦ · ASIAN DEVELOPMENT BANK PCR: CAM 27410 PROJECT...

ASIAN DEVELOPMENT BANK PCR: CAM 27410

PROJECT COMPLETION REPORT

ON THE

BASIC HEALTH SERVICES PROJECT (Loan 1447-CAM[SF])

IN CAMBODIA

March 2004

CURRENCY EQUIVALENTS

Currency Unit – riel (KR)

At Appraisal At Project Completion (15 May 1996) (31 December 2002)

KR1.00 = $0.0002621 $0.0004 $1.00 = KR2,300 KR3,835

For the purpose of calculations in this report, a rate of $1.00 = KR3,900 is used. This was the rate generally prevailing at the time of the project completion.

ABBREVIATIONS ADB – Asian Development Bank BHSP – Basic Health Services Project BME – benefit monitoring and evaluation CHSPP – contracting health services pilot project DHMT – district health management team HNI – Health Net International HSC – health service contractor ISC – integrated supervisory checklist MOH – Ministry of Health NGO – nongovernmental organization PCU – project coordination unit PHD – Provincial Health Department SDR – special drawing rights TA – technical assistance

NOTES

(i) The fiscal year (FY) of the Government ends on 31 December. (ii) In this report, "$" refers to US dollars.

CONTENTS

Page

BASIC DATA i MAP v EXECUTIVE SUMMARY vii

I. PROJECT DESCRIPTION 1

II. EVALUATION OF DESIGN AND IMPLEMENTATION 1 A. Relevance of Design and Formulation 1 B. Project Outputs 2 C. Project Costs 7 D. Disbursements 8 E. Project Schedule 8 F. Implementation Arrangements 9 G. Conditions and Covenants 9 H. Related Technical Assistance 9 I. Consultant Recruitment and Procurement 9 J. Performance of Consultants, Contractors, and Suppliers 10 K. Performance of the Borrower and the Executing Agency 10 L. Performance of the Asian Development Bank 11

III. EVALUATION OF PERFORMANCE 11 A. Relevance 11 B. Efficacy in Achievement of Purpose 11 C. Efficiency in Achievement of Outputs and Purpose 11 D. Preliminary Assessment of Sustainability 12 E. Environmental, Sociocultural, and Other Impacts 12

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 13 A. Overall Assessment 13 B. Lessons Learned 13 C. Recommendations 14

APPENDIXES 1. Project Framework 15 2. Comparison of Key Planned and Actual Outputs by Project Component 20 3. Evaluation of the Contracting Health Services Pilot Project 24 4. Capacity Building, Training, and Staff Development 37 5. Estimated and Actual Project Costs and Financing Plan 38 6. Disbursement of Funds, 1996–2003 39 7. Project Implementation Schedule, 1996–2003 40 8. Compliance with Major Loan Covenants 42 9. Technical Assistance Completion Report 53 10. Consulting Services 55 11. Assessment of Overall Project Performance 56 12. Benefit Monitoring and Evaluation 57

BASIC DATA A. Loan Identification 1. Country 2. Loan Number 3. Project Title 4. Borrower 5. Executing Agency 6. Amount of Loan 7. Project Completion Report Number

Cambodia 1447-CAM(SF) Basic Health Services Kingdom of Cambodia Ministry of Health SDR13.826 million CAM 798

B. Loan Data 1. Appraisal – Date Started – Date Completed 2. Loan Negotiations – Date Started – Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness – In Loan Agreement – Actual – Number of Extensions 6. Closing Date – In Loan Agreement – Actual – Number of Extensions 7. Terms of Loan – Interest Rate – Maturity – Grace Period

26 Feb 1996 15 Mar 1996 15 May 1996 17 May 1996 20 Jun 1996 29 Jul 1996 27 Sep 1996 0 30 Jun 2002 5 Aug 2003 2 1% per year 40 10

8. Disbursements a. Dates Initial Disbursement

29 Nov 1996

Final Disbursement

5 Aug 2003

Time Interval

86 months

Effective Date

27 Sep 1996

Original Closing Date

30 Jun 2002

Time Interval

69 months

ii

b. Amount ($'000) Category or Subloan

Original Allocation

Last Revised

Allocation

Amount

Canceled

Net Amount Available

Amount

Disbursed

Undisbursed

Balance Civil Works Furniture, Equipment, and Vehicles

4,111.9

2,600.4

5,089.6

727.5

0

0

5,089.6

727.5

5,051.900

726.000

38.300

1.500 Materials (Medical Supplies and Drugs)

3,556.5

1,283.7

0

1,283.7

1,272.600

11.100

Staff Training/Fellowships 523.3 389.5 0 389.5 350.500 39.000 Consulting Services 161.8 88.2 0 88.2 87.800 4.000 Surveys and Pilot Tests 4,786.8 9,322.2 0 9,322.2 9,435.500 -113.300 Incremental Project Implementation and Costs 896.4 1,066.8 0 1,066.8 1,078.800 -12.000 Operation and Maintenance 141.4 12.9 0 12.9 .035 13.000 Service Charge 402.3 358.1 0 358.1 358.000 0 Unallocated 2,972.0 22.2 0 22.2 0 22.200 Total 20,152.8 18,360.7 0 18,360.7 18,361.000 0.016

9. Local Costs (Financed) - Amount $3.10 million - Percent of Local Costs 54 - Percent of Total Cost 15 C. Project Data

1. Project Cost ($ million)

Cost Appraisal Estimate Actual

Foreign Exchange Cost 14.40 15.26 Local Currency Cost 5.60 5.75 Total 20.00 21.01

2. Financing Plan ($ million) Cost Appraisal Estimate Actual Implementation Costs Borrower-Financed 5.00 2.65 ADB-Financed 20.00 18.00 Total 25.00 20.65 Service Charge Borrower-Financed 0.00 0.00 ADB-Financed 0.36 0.36 Total 0.36 0.36

ADB = Asian Development Bank, IDC = interest during construction.

iii

3. Cost Breakdown by Project Component ($ million)

Component Appraisal Estimate Actual Civil Works 6.50 7.65 Land Acquisition 0.20 0.00 Furniture, Equipment, and Vehicles 2.70 0.73 Materials (Medical Supplies and Drugs) 4.40 1.27 Staff Development 0.50 0.35 Consulting Services 0.20 0.09 Surveys, Evaluation, and Pilot Tests 4.70 9.46 Project Implementation 0.90 1.08 Operation and Maintenance 0.70 0.02 Service Charge 0.00 0.36 Contingencies 3.70 0.00 Total 25.00 21.01

4. Project Schedule

Item Appraisal Estimate Actual

Date of Contract with Consultants Sheladia Associates Aug 1997 Macro International Sep 1997 Cambodian Red Cross-Medical Engineering Center Sep 1999 Project coordination unit manager Jan 1997 Feb 1997 Chief of Administration and Finance/Procurement

Specialist Jan 1997 Jan 1997 Contracting Specialist Jan 1997 May 1997 Medical Equipment Maintenance Planner Jul 1998 Mar 2000 Civil Works Engineer Jan 1997 Aug 1997 Health Facilities Architect Sep 1997 Sep 1998 Completion of Engineering Designs Apr 1997 Jul 1997 Civil Works Contract Date of Award Jan 1998 May 1998 Completion of Work Dec 2001 May 2001 Equipment and Supplies First Procurement Jul 1997 Jul 1998 Last Procurement Dec 2001 Jun 2002 Completion of Equipment Installation Jan 2002 Aug 2002 Start of Operations Completion of Tests and Commissioning Nov 2001 Nov 2002 Beginning of Start-Up Jan 2002 Jul 2002 Other Milestones Reallocation of Loan Proceeds 10 Dec 1998 First Extension of Loan Closing Date 20 Mar 2001 Reallocation of Loan Proceeds 18 Apr 2000 Extension of Health Services Contractors Jan 2003 Second Extension of Loan Closing Date 16 Jan 2003 Actual Loan Closing Date 5 Aug 2003

iv

5. Project Performance Report Ratings

Ratings Implementation Period

Development Objectives

Implementation Progress

(i) From 20 Nov 1996 to 31 Dec 1998 S S (ii) From 1 Jan 1999 to 31 Dec 2000 HS HS (iii) From 1 Jan 2001 to 31 Aug 2003b S S HS = highly satisfactory, S = satisfactory.

D. Data on Asian Development Bank Missions

Name of Mission

Date

No. of Persons No. of Person-Days

Specialization of

Membersa

Fact-Finding 20 Nov–9 Dec 1995 4 54 a, d, e, f Appraisal 26 Feb–15 Mar 1996 3 43 a, d, f Inception 4–11 Oct 1996 1 8 a Review Mission 1 12–19 Mar 1997 1 8 a Review Mission 2 4–8 Nov 1997 1 5 a Review Mission 3 25–31 Mar 1998 1 7 a Review Mission 4 10–17 Nov 1998 1 8 a Review Mission 5 (Special Loan Admin.) b

22–27 Mar 1999 1 6 b

Review Mission 6 19–27 Apr 1999 1 9 a Review Mission 7 15–23 Sep 1999 2 18 a, c Review Mission 8 16–24 Mar 2000 2 18 a, c Midterm Review Mission 10–21 Jul 2000 2 18 a, c Review Mission 9 19–23 Feb 2001 2 10 a, b Review Mission 10 b 22–30 Nov 2001 1 9 a Review Mission 11 28 Feb–8 Mar 2002 1 9 a, c Review Mission 12 18–28 Sep 2002 2 22 a, c Project Completion Reviewc 4–19 Sep 2003 2 24 a, c a a = health specialist, b = senior project specialist, c = project analyst, d = project economist, e = program

economist, f = counsel. b In conjunction with the review of another project. c The mission consisted of Wan Azmin, health specialist/mission leader, and Adelaida Mortell, national project

officer/mission member.

vii

EXECUTIVE SUMMARY

In the mid 1990s, the Ministry of Health (MOH) identified the basic issues in primary health care as inadequate physical facilities; lack of management capacity, particularly at district level; and inadequate financing by the Government. The Ministry formulated the Health Coverage Plan (HCP) as a step forward in restructuring its delivery of health care services for the rural population. The Project's principal objective was to reduce preventable mortality and morbidity, particularly among poor women and children, in five provinces: Kampong Cham, Kampong Chhnang, Prey Veng, Svay Rieng and Takeo provinces. Collectively these five provinces comprised about a third of the country's population, with 4.1 million people. The aim of HCP is to improve accessibility and coverage through the minimum package of activities (MPA) in health centers and complementary package of activities in referral hospitals. There was a need for an efficient and effective to deliver primary health services through the MPA at the periphery. The Project aimed to assist the Government by (i) strengthening the physical infrastructure at the commune and district levels; (ii) improving management capacity; (iii) testing mechanisms to increase the efficiency of health service delivery; and (iv) increase capacity of MOH and ensuring the benefits to the community. At Project completion, the impact are (i) infant mortality rate has reduced from 115 per 1,000 live births to 78 per 1,000 live births, (ii) under-five mortality rate, from 181 per 1,000 live births to 82.8 per 1,000 live births; and (iii) maternal mortality ratio, from 650 to 368 per 100,000 live births. A Project Coordination Unit (PCU) was established as planned to co-finance the costs of PCU management and administrative and financial functions. A joint PCU manager, administration unit, and technical units for the Asian Development Bank (ADB) and World Bank were established separately within the PCU. The PCU coordinated closely with the Provincial Project Unit (PPU) established in Project’s provincial health departments (PHDs) in Kampong Cham, Kampong Chhnang, Prey Veng, Svey Rieng, and Takeo provinces. The Project, fortified by a technical assistance, strengthened the management capacity and supervision of PHDs and district management health committees at the operational districts on the health centers and referral hospitals through integrated supervisory checklists. In strengthening the community health services, the Project satisfactorily constructed and renovated 229 health facilities in five provinces. Basic and essential drugs packed under MPA drug kits were provided to health centers and referral hospitals in the Project areas. There was increased accessibility in the delivery of primary health care services. Under the Project, the Contracting for Health Services Pilot Project (CHSPP) was successfully designed and tested the hypothesis that government might deliver health services more effectively and efficiently through administering contracts to non-governmental entities for services. Taking a number of non-intervention areas as controls, two models of contracting for health services were tested at district level: (i) contracting out where the health service contractors had full responsibility for delivery of all district health services in accordance with the Health Coverage Plan and MOH technical protocols; and (ii) contracting in where the health service contractors provided only management of the district health care services, with the actual health staff remaining MOH civil servants. There were 4 control districts, 3 contracting in districts, and 2 contracting out districts. The 2001 evaluation survey found that government contracting of the provision of health services to non-governmental entities is feasible, cost effective, high performing, and equitable, that is, effectively targets and benefits the poor. Contracting of health services can deliver interventions proven to reduce infant, child and maternal mortality to more people, more rapidly, than conventional government service delivery mechanisms. However, as the local private

viii

sector capacity is still quite limited, international NGOs could have an important role to play in developing a cadre of local groups capable of implementing future health service contracts. The Project conducted a baseline household survey as part of the National Health Survey (NHS) for benefit monitoring and evaluation in 1998. The Government conducted a separate Cambodia Demographic and Health Survey in 2000. A comprehensive household survey was also carried out at Project completion in 2002. The key findings as objectively verifiable indicators are (i) health centers with MPA has increased from 9.2% to 81.9%; (ii) referral hospitals with surgical capacity increased from 18.5% to 40.7%; (iii) fully immunized children increased from 33.3% to 45.2%; (iv) antenatal care in institutions increased from 40.1% to 52.6%; (v) vitamin A supplement increased from 48.6% to 61.6%; (vi) contraceptive prevalence increased from 21.1% to 31.7%; and (vii) treatment of illness/injury in public health facilities and outreach services increased from 7.1% to 17.6%. Based on the 11 objectively verifiable indicators, contracting out districts have the maximum mean increase of 320.4 %, Contracting in districts with 179.7%, and the control districts with 99.6%. In addition to much higher increases in health care coverage, the contracting out districts achieved a very substantial decrease in out-of-pocket health care expenditures by the poor, with a reduction over baseline of 70%. The reduction in out-of-pocket costs was greater among the poor than among the overall population, indicating that this intervention successfully reached and benefited the desired target group. The Project has targeted on the poorest segments in the rural population by ensuring that progress on project indicators related to coverage and health facility is computed for the poorest 50% of the population. In 1997, the lower 50% socioeconomic group in the contracting out districts spent the same or more for health care per capita than the higher income group, while they now spend considerably less. The Project also supported the strengthening of equipment maintenance and repair, particularly for the referral hospitals. An international consultant, recruited by MOH for six person-months, developed, assisted the implementation, and evaluated a provincial system for medical equipment maintenance for the comprised health centers, referral hospitals, and provincial hospital. The system, assisted by the Cambodian Red Cross, was revised based on the field experience and implemented nation-wide. Some basic training for provincial and district staff was organized locally.

I. PROJECT DESCRIPTION

1. The Project would focus on providing basic health services to the rural poor and disadvantaged, including women and children, in the provinces of Kampong Cham, Kampong Chhnang, Prey Veng, Svey Rieng, and Takeo. The Project took place in these provinces because of their easy access from Phnom Penh, their lack of basic health care delivery system, and their receipt of only minimal external assistance. The intent of the Asian Development Bank (ADB) assistance was to fund health care infrastructure, including facilities, and medical equipment and supplies, and to strengthen management capacity. The Project’s main objective was to enhance basic health care services so as to reduce preventable mortality and morbidity by (i) strengthening physical infrastructure at the commune and district levels, (ii) improving management capacity, and (iii) testing mechanisms that could increase the efficiency of health service delivery.1 The Project had four components as follows:

(i) Strengthening community health services (a) constructing or renovating health centers, (b) providing equipment and essential drugs (c) maintaining selected health facilities.

(ii) Strengthening district health offices (a) improving their management capacity (b) strengthening the supervision of health centers, and (c) strengthening selected referral hospitals.

(iii) Introducing health sector reforms by pilot testing the following innovative approaches to health care delivery: (a) contracting out health services (b) contracting in management services, and (c) setting up community loan schemes for health emergencies

(iv) Providing support for central office management support by (a) setting up a project coordination unit (b) carrying out benefit monitoring and evaluation, and (c) strengthening equipment maintenance and repair capabilities

II. EVALUATION OF DESIGN AND IMPLEMENTATION

A. Relevance of Design and Formulation

2. In 1997, the Ministry of Health (MOH) formulated the Health Coverage Plan,2 a step toward the reorganization of the basic health care delivery system. The system would be managed by means of operational districts. Each operational district would provide basic health services to a population of approximately 100,000 people. Each health center would deliver a minimum package of activities that would include promotive, preventive, and curative services to approximately 10,000 targeted women and children and to the poor. The district’s referral

1 ADB. 1994. Technical Assistance to Cambodia for Basic Health Services Project. Manila. 2 MOH. 1997. Guidelines for Operational Districts. Phnom Penh. The system would be managed by operational

districts, where each operational district would provide basic health services to a catchment population of approximately 100,000 people. Each health center will deliver minimum package of activities, comprising promotive, preventive, and curative services to the community of 10,000 people, targeted women, children, and the poor. The referral hospital provides complementary package of activities.

2

hospital would provide a complementary package of activities. The strengthening of the system for basic health care delivery is consistent with ADB’s country strategy and program, and particularly with the country’s objectives. The Project is in accordance with ADB's operational strategy on human resources development, including reducing poverty and improving the health status of Cambodia's rural poor. ADB supported the Government's commitment to human development. The Government’s aim was to address its broad economic goals as applied to the health sector by (i) involving nongovernmental organizations (NGOs) and the private sector in health service delivery, (ii) reducing the financial burden on the poor resulting from illness, and (iii) improving accessibility to and distribution of health facilities. Improvements started gradually at referral hospitals and health centers because of a lack of resources, but participation and the level of ownership are high. The midterm review in July 2000 strengthened the Project’s development objectives. Project relevance was high at appraisal and continued to be high at completion.3 B. Project Outputs

3. The evaluation of implementation is based on (i) the indicators set at appraisal, (ii) the July 2000 midterm review, (iii) the completion reports prepared by MOH, (iv) the benefit monitoring and evaluation data on the health sectors, and (v) a comparison of key planned and actual project outputs by component (Appendix 2).

1. Strengthening Community Health Services

4. The Project was to strengthen community health services by (i) constructing and renovating selected health centers, selected operational districts, and referral hospitals, (ii) providing basic medical equipment and essential drugs, and (iii) maintaining health centers, operational districts, and referral hospitals.

a. Constructing and Renovating Health Centers

5. The Project plan was to support the construction of 251 new health centers, the renovation of 75 existing health centers, provision of essential drugs and equipment, and the establishment of a facility and equipment maintenance capability in the project provinces. The Project also planned to renovate 20 district referral hospitals, including upgrading their emergency obstetric and trauma care facilities. However, the Project’s scope was reduced to constructing 187 new health centers, renovating 20 existing health centers and 13 referral hospitals. In addition, two operational districts were also renovated. The civil works were satisfactorily completed in July 2000 according to schedule by Cambodian contractors closely supervised by the international engineering firm. Health center staff received training in the minimum package of activities under CAM-1368: Basic Skills Project4 (Part B: Health Personnel Training). Hospital development has just started, but training in the complementary package of activities could not be developed because of a lack of resources. A total of 212 health facilities in Project provinces were constructed or renovated under four blocks (civil works package). Twenty-five new health centers were constructed under block one, 48 under block two, 61 under block three, and 53 under block four. Ten existing health centers were renovated under block two. One operational district was renovated under block two and one under block three. Two referral hospitals were renovated under block two and 11 under block four. 3 ADB. 2002. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to

Cambodia for Health Sector Support Project. Manila. 4 ADB 1995. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to

Cambodia for Basic Skills Project. Manila.

3

6. The scope of the civil works was reduced to accommodate the higher than expected expenditures for civil works and contracting of health services, for example, at appraisal the unit costs for constructing a health center were $15,000, but actual costs were $22,000 in block one and $32,000 in block four. This was partly because the original cost estimates for health centers had excluded the cost of such items as wells, site works, latrines, and fences. The appraisal estimates for renovating the referral hospitals were also low, because they assumed that the extent of repairs would be small. But many referral hospital buildings were dilapidated and required more works than had been anticipated. In addition, the costs of the supervising engineering firm were also underestimated. Even though the number of health centers to be constructed was reduced, funds also had to be reallocated to cover the costs of block four civil works. 7. The Project increased the number of health centers in the Project provinces. The use of health centers for curative care increased more than doubled and the use of referral hospitals decreased. The number of consultations per health center ranged from 100 to 400 per month. User fees varied from KR100 to KR500 per outpatient visit, depending on the community health committees and the location of the health center. A significant increase was observed in child immunization, prophylactic vitamin A supplementation, antenatal care, (including tetanus immunization), and contraceptive use.

b. Providing Equipment, Furniture, Supplies, and Essential Drugs

8. Basic medical equipment, furniture, medical supplies, and essential drugs were procured and delivered to the project health centers, referral hospitals, and operational districts as planned. The first procurement by international competitive bidding for basic medical equipment and essential drugs for 70 health centers was carried out in early 1998. The second procurement under international competitive bidding for medical equipment for 156 health centers, awarded in November 1999, encountered delays because transportation by ship, port clearance, and distribution by MOH’s central medical stores took longer than expected. In addition, further delays were encountered because some items were missing and had to be shipped later. MOH also encountered problems with suppliers that could not supply all the required medical equipment within one order and the poor quality of many items that broke down within a year and became unusable.

c. Maintaining Selected Health Facilities

9. The Project supported the maintenance of health center and referral hospital buildings, and landscaping. Some health centers and referral hospitals were affected by the 2000/2001 floods and were repaired under the CAM-1824: Emergency Flood Rehabilitation Project (Part E: Health).5 The maintenance of project facilities was facilitated by the development of guidelines, and the training of staff in each province in March 1999.

2. Strengthening District Health Offices

10. The Project supported the strengthening of district health offices (now known as operational districts6) by (i) improving their management capacity; (ii) strengthening their

5 ADB. 2000. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to

Cambodia for Emergency Flood Rehabilitation Project. Manila. 6 Headed by an operational district chief. The number of staff varies from 80 to 300 depending on the location.

4

supervision of health centers, and (iii) strengthening selected referral hospitals. The Project strengthened the capacity of district-level managers by (i) training them in management, budgeting, and related skills; (ii) developing and implementing supervisory systems, including integrated supervisory checklists (ISCs) for use in the supervision of health center and referral hospital staff; and (iii) supporting district health management teams (DHMTs) by means of training modules. The technical assistance (TA)7 (i) strengthened the capacity of DHMTs, (ii) helped establish supervisory systems for health centers and referral hospitals, and (iii) introduced the concepts and methods of benefit monitoring and evaluation.

a. Improving Management Capacity

11. All DHMTs in the project districts completed the four-module, 2-week basic management course by December 1998. The four modules covered planning, finance and budgeting, supervision, and essential drugs management.

b. Supervisory Systems

12. The Project developed ISCs for the supervision of (i) DHMTs’ management activities by provincial health department (PHDs); (ii) supervision of health centers; and (iii) supervision of referral hospitals. The ISCs were implemented nationwide commencing in March 2000. In addition, the Project successfully developed and implemented a two-week basic management course attended by all DHMTs in the project provinces that now been implemented nationwide. By project completion, supervisory systems, including training in the use of the ISCs; means of transportation; and support for other travel costs; were in place in every project district. Thus, the Project has succeeded in establishing a health service system at the district level where none had previously existed, and with significantly improved coverage indicators.

c. Providing Support for District Health Management Teams

13. The Project provided office equipment, supplies, and petty cash to support supervisory activities by operational districts and DHMTs. Where necessary, DHMT offices were renovated, either separately or as part of the renovation of referral hospitals.

3. Introducing Health Sector Reforms

14. The Project planned to pilot test innovative approaches before they were implemented nationwide using a small but well-defined area. Originally the Project planned to contract health services in 10 districts, 5 districts for contracting out and 5 districts for contracting in. In addition, the Project would select 5 districts to act as controls. 15. The Project successfully tested innovative approaches to health service delivery through an operational research component in which the Government contracted with private organizations to provide health care services. Two different models were tested: contracting out and contracting in. A community loan scheme for health emergencies was, however, dropped. Under contracting out, the health service contractors (HSCs) have full authority and funds for staffing, management and operational costs. Under contracting in, the HSCs provide technical support to the operational districts, but MOH is still responsible for the running costs for health services and for staff salaries. The control districts remained under the normal MOH

7 ADB. 1996. Technical Assistance to Cambodia for Managing Basic Health Services Project. Manila.

5

management system. A budget supplement was provided for contracting in and control districts to permit an unbiased comparison.

a. Contracting Health Services Pilot Project

16. The contracting health services pilot project (CHSPP) started 6 months later than originally planned with a baseline survey and the development of objectively verifiable indicators. An MOH working group, assisted by an international ccontracting specialist, developed terms of references and guidelines for the contracting out and contracting in districts. The contracting in and control districts were provided with supplementary budget to augment the minimal government budget allocations. Meetings were held at operational district and PHD levels to raise public awareness about the pilot. Only 12 districts turned out to be suitable for CHSPP: 4 for contracting out, 4 for contracting in, and 4 as controls. However, no acceptable bids were received for three districts, and because of funding constraints, a decision was made not to rebid the districts. Consequently, the final districts to be included were two for contracting out, three for contracting in, and four as controls. By December 1998, contracts were awarded to international NGOs with prior experience in Cambodia. The districts were Memut (under Save the Children Funds/Australia), and Ang Rokar (under the Association of Medical Doctors of Asia, based in Japan) for contracting out, and Cheung Prey (under Save the Children Funds/Australia), Peareng (under Health Net International, based in the Netherlands), and Kirivong (under Enfants et Developpement or Children and Development, later named Save the Children France). The control districts were Preah Sdach, Kamchay, Bati and Kroch Chhmar (Appendix 3).

b. Contracting Out

17. The contracting out HSCs were mobilized in January 1999, and took over authority for managing their operational districts by March–April 1999. The contracting out HSCs purchased their own drugs and consumable supplies following ADB’s procurement guidelines. The contracting out districts and the provincial PHDs agreed on mutually acceptable working arrangements and their performance has been satisfactory.

c. Contracting In

18. The contracting in HSCs took over authority for managing their operational districts by March–April 1999. The HSCs reinforced discipline among existing MOH staff. The contracting in has resulted in cost recovery and has provided additional income to staff as an incentive. This has proven essential, as the government salary alone is inadequate to motivate staff. The contracting in districts relied on drugs and equipment provided through normal MOH channels, supplemented by small purchases using the supplementary budget funds provided by MOH. The contracting in districts also used the budget supplement to finance outreach activities, and to purchase drugs and materials when MOH supplies were insufficient or delayed. ADB carried out field audits of the use of the budget supplement to ensure transparency and accountability. Extensive meetings and field visits were conducted to facilitate the transition to contracting in, as the presence of a private contractor in between two layers of MOH staff creates a complex situation. 19. The use of health centers in the contracting in districts was satisfactory and better than in the control districts. The services provided by referral hospitals have improved. The low level of the government operating budget has severely limited what contracting in can accomplish, as

6

the contracting in districts are receiving proportionately less than other districts in the same province. 20. The monitoring committee chaired by the Director General of Health has direct supervision over the contracting in districts, facilitating an understanding at the provincial and district levels of the new roles and relationships. The relationships between contracting in and PHDs were more complex than in contracting out, as both MOH and HSCs must deliver health services at the district levels collaboratively. Some operational districts continued to report directly to, and receive instructions from the PHD, while some contracting in contractors have bypassed the PHD in communicating with MOH.

d. Control Districts

21. The budget supplements to the control districts were used to finance outreach activities, provide staff incentives, and purchase drugs and materials when MOH supplies were insufficient or delayed. As for the contracting in districts, field audits were conducted to ensure transparency and accountability. The control districts have developed good accounting procedures and received regular replenishment of the budget supplements. 22. The Project conducted operations research on the feasibility, benefits, and cost-effectiveness of the government contracting the provision of district health services to NGOs. Contracting was conducted in two different forms in a total of five districts, with another four districts as controls. Baseline data was collected through household and facility surveys prior to the start of the intervention, and the surveys were repeated in mid-2001. An external evaluation found that contracting significantly improved health service coverage, with disproportionate benefits to the poor; was more cost-effective than conventional government service delivery; and substantially decreased out-of-pocket expenditures by the poor, even when a user fee system was in place. 23. The CHSPP evaluation survey concluded that contracting for health services is feasible, cost-effective, and equitable and that it benefited the rural poor in particular. The contracting out model was preferable to the contracting in model. The contracting model could be applied to (i) outreach services, (ii) health centers and referral hospitals and their catchment areas, (iii) an individual district or districts in a province, and/or (v) entire provinces. Based on these results, the Government has decided to continue with contracting health services but on a larger scale with participation by other development partners, including the World Bank, the Department for International Development, and the Government of Belgium. More than 1 million people in 11 operational districts in some of the poorest and most remote parts of the country will benefit under the ongoing Health Sector Support Project (footnote 3).

e. Community Loan Scheme for Health Emergencies

24. The Project also planned to pilot test a community loan scheme that would prevent families from becoming destitute as a result of catastrophic medical expenses. However, MOH and ADB agreed to drop the scheme early during project implementation because of (i) the lack of a budget for this activity, because the contracting health services would be using the entire budget for this subcomponent; (ii) the health services involved in contracting had already introduced a type of community scheme; and (iii) the time was premature for introducing such a loan scheme.

7

4. Providing Support for Central Office Management

a. Project Coordination Unit

25. The Project established a project coordination unit (PCU), cofinanced with the World Bank,8 for procurement, disbursement, accounting, training MOH staff in technical skills, and reporting. A joint PCU manager, Administration Unit, and technical units for ADB and the World Bank were established separately within the PCU. The total number of PCU staff was increased from 8 to 17 with ADB’s concurrence. Provincial project units were established in each PHD in the project area. The provincial project unit coordinators maintained close relationships with the PCU and with operational districts.

b. Carrying Out Benefit Monitoring and Evaluation

26. To establish objectively verifiable indicators for assessing the benefits realized, the Project recruited an international firm, Macro International, to carry out demographic health surveys and a domestic firm, the Foundation for Advice on Water and Agronomy Consultants for Development for data collection. Macro International was recruited for short-term and specialized tasks, including developing a sample design and survey instrument, training interviewers, and entering and analyzing data. The survey results were satisfactory. PCU assisted in the printing in Cambodian context and financial reporting. The Foundation for Advice on Water and Agronomy Consultants for Development conducted the collection, collation, and initial analysis of the data. 27. The Project assessed the effectiveness of the new approaches to service delivery based on contracting using a baseline household and health facility survey conducted during the first year of project implementation to permit comparison of project provinces with other provinces in Cambodia. The baseline household survey was completed in 1998 as part of the 1998 national health survey. The Government carried out a nationwide demographic and health survey in 2000 and a comprehensive household survey in 2002.

c. Strengthening Equipment Maintenance and Repair Capabilities

28. The Project was also intended to help MOH improve the maintenance and repair of hospital medical equipment. An international consultant, recruited by MOH for 6 person-months developed a system for use in the provinces to maintain medical equipment, assisted in its implementation, and evaluated it. The role of the private sector, NGOs, MOH maintenance and repair staff, and mechanisms for planned preventive maintenance were established. The system covered health centers, referral hospitals, and provincial hospitals. It was revised as necessary based on initial field experience and implemented nationwide. The Medical Engineering Center of the Cambodian Red Cross helped the consultant train health staff in the provinces. No study visits took place and the funds were reallocated to the CHSPP. C. Project Costs

29. The total project cost was estimated at $24.9 million equivalent at appraisal, with ADB to finance $20 million, and the Government to finance $5 million. The foreign exchange component was estimated at $14.4 million, fully financed by ADB. The local component was $10.5 million, with ADB to finance $5.6 million, and the Government to finance $5 million. Because of a lack of

8 World Bank. 1996. Cambodia Disease Control and Health Development. Washington, DC.

8

familiarity with ADB procedures, the executing agency, MOH, had some difficulties with cost estimates. A number of expenditures were higher than expected: (i) surveys, pilots, and evaluation for the CHSPP cost $9.47 million rather than the expected $4.70 million, (ii) civil works amounted to $7.6 million rather than the anticipated $6.5 million, (iii) project implementation cost $1.08 million rather than the estimated $0.90 million. In addition, depreciation of the SDR against the dollar reduced the allocated budget by about $1.7 million from $20 million equivalent to $18.36 million equivalent. 30. The construction cost per health center was $22,000 in block 1 and $32,000 in block 4, while the appraisal estimate was $15,000. The actual cost included wells, site works (e.g., drainage) latrines, fences, and so on, which was excluded from the original estimate. The actual costs for renovating referral hospitals were higher because of the need for more extensive repair work than anticipated. The costs of the supervising engineering firm were also higher than estimated. Originally the CHSPP was to comprise five contracting out districts, five contracting in districts, and five control districts. Even with the reduction to contracting out districts, three contracting in districts, and four control districts, the actual cost was more than double. The cost of the baseline household survey was also about double that estimated at appraisal. D. Disbursement

31. All loan proceeds, $18.361 million equivalent, were disbursed (Appendix 6), mainly through the imprest account and through direct payment in accordance with the financial regulations of the Ministry of Economy and Finance and ADB’s Guidelines on Imprest Fund and Statement of Expenditures Procedures. Loan proceeds were fully utilized prior to the project end date. The turnover rate and replenishment of the Project’s loan fund imprest account were high, although the Project experienced some delays in replenishment of the counterpart funds. The initial ceilings approved by the Ministry of Economy and Finance for the imprest and PCU accounts were too low to meet the Project’s disbursement needs. After discussions during the midterm review, the ceiling was increased from $150,000 to $200,000, but replenishments were lower than expected because of MOH’s and the Ministry of Economy and Finance’s approval processes. 32. On 10 December 1998, ADB approved the reallocation of loan funds and increased the original allocation of $4.60 million for CHSPP to $9.40 million. To this end, $2.45 million was reallocated from the unallocated category; $1.60 million from the furniture, equipment, and vehicles category; $250,000 from the staff training fellowships category; and $100,000 from consulting services category. On 18 April 2000, ADB approved another reallocation for the civil works. The total estimated cost for civil works was $2.72 million: $1.80 million from loan funds and $0.93 million from counterpart funds. To fund the increased cost of civil works, $1,400 was reallocated from furniture, equipment, and vehicles category; $758,000 from the medical supplies and drugs category; $200,000 from surveys and pilot tests; and $339,000 from the unallocated category. Reallocation was also done for consulting services by an adjustment of $35,000 from the medical supplies and drugs category. E. Project Schedule

33. The Project was to be implemented for five years, from January 1997 to December 2001. Implementation of all activities commenced later than planned due to an underestimated time required to establish the PCU. Once the PCU was operational, however, implementation proceeded satisfactorily and all components were completed at Project completion. The Loan

9

was extended twice from 30 June 2002 to 31 December 2002, to harmonize the Project physical completion date with the CHSPP contract agreement; and from 31 December 2002 to 31 march 2003 for smooth transition and continuation of the CHSPP under the newly approved Health Sector Support Project (HSSP). The CHSPP contracts were extended for one year with retroactive financing under the HSSP to allow sufficient time for competitive awards of new contracts. F. Implementation Arrangements

34. The implementation arrangements were in accordance with the design at appraisal. A special PCU was established within MOH. As the World Bank-financed Cambodia Disease Control Project commenced at the same time, it was decided to cofinance the costs of PCU management, administrative, and financial functions. A joint PCU manager, and administration and finance unit within the PCU while separately funded technical units for ADB and World Bank were established. International and domestic consultants were recruited to strengthen the PCU’s capacity. The PCU was responsible for day-to-day operations, including coordination with various agencies, procurement, financial management, consultant recruitment, fellowship awards, submission of reports, and monitoring of project implementation. At the provincial level, provincial project units were established in each provincial health department, whose staff were civil servants. The Project provided funds, office equipment, furniture, and service vehicles for the provincial project units. Implementation of the CHSPP was monitored by a steering committee chaired by the director general of health. G. Conditions and Covenants

35. MOH generally complied with the major loan covenants stipulated in the Loan Agreement. No covenants were modified, suspended, or waived. Appendix 8 provides a comprehensive assessment of compliance with the covenants. H. Related Technical Assistance

36. Appendix 9 summarizes the findings of the advisory TA on "the Management of Basic Health Services." The main objectives of the TA were (i) to provide DHMTs with relevant management skills, (ii) help establish supervisory systems for health centers and referral hospitals, and (iii) introduce the concepts and methods of benefit monitoring and evaluation. Following ADB’s Guidelines on the Use of Consulting Services, an international consultant (health systems specialist) was recruited for 50 person-months, and a domestic consultant (materials preparation specialist) was recruited for 27 person-months. The TA consultants submitted the inception, interim, draft final, and final reports as scheduled. ADB supervision of the TA was adequate and timely. The TA fully met its objectives and scope, particularly in developing ISCs for health centers, referral hospitals, and operational districts. I. Consultant Recruitment and Procurement

1. Recruitment of Consultants

37. It was envisaged at appraisal that international and domestic consultants to assist Project implementation will be recruited. Under the Project, four consulting firms, four long-term international consultants, and a number of short-term consultants have been recruited for the implementation of the Project (Appendix 10). The recruitment of consultants was in accordance with ADB's Guidelines on the Use of Consultants. Except for the minor delay in the recruitment

10

of an international procurement specialist, there were no major problems were encountered. Overall, consulting services used in the Project were highly beneficial to Project implementation.

2. Civil Works

38. The procurement of civil works, conducted under local competitive bidding following ADB's Guidelines for Procurement, was packaged into four blocks. A total of 212 health facilities was constructed or renovated in five provinces with no major delay or problems. The international supervisory and engineering firm provided assistance with developing technical specifications for bid documents, monitored the quality of construction, and provided recommendations to MOH in relation to the release of progress payments. The civil works were completed satisfactorily under close supervision by the international engineering firm.

3. Equipment, Furniture, and Essential Drugs

39. Medical equipment, furniture and essential drugs were procured following ADB’s Guidelines for Procurement and delivered to the project facilities. The procurement of basic medical equipment and essential drugs under international competitive bidding encountered delays because of prolonged shipping times, delays at the port, and delays in distribution by the central medical stores. The equipment and drugs were repackaged into packages approved by the Ministry of Economy and Finance and procured under international shopping and local purchase in late 2000. J. Performance of Consultants, Contractors, and Suppliers

40. With the exception of the international procurement specialist, all consulting services performed satisfactorily and contributed significantly to project implementation. The international procurement specialist's poor performance was mainly due to unfamiliarity with ADB procedures. The performance of the international supervisory and engineering firm (Sheladia Associates) was satisfactory and ensured that the civil works were of high quality. The civil works packages were completed satisfactorily under close supervision and included penalties for delays. Most contractors were willing to learn, and through careful planning and attention to detail rose to the challenge and succeeded in building facilities on time and to specification, and in addition increased their capacity for quality construction. The facilities constructed are of notably better quality than similar facilities constructed by other external funding agencies. The supply of basic medical equipment for the referral hospitals was generally unsatisfactory. The end users complained that many items broke down within a year and were unusable. K. Performance of the Borrower and the Executing Agency

41. Government ownership of the Project was strong, particularly of the health sector reforms. The Borrower provided adequate counterpart funds, even during the economic downturn. Cooperation between Government and ADB review missions was professional, and performance by MOH is considered satisfactory. Documentation of various aspects of project implementation, such as procurement, consulting, staff development, monitoring, and disbursements, was satisfactory and was made available to ADB. The PCU prepared progress reports and submitted them to ADB regularly. Coordination with international development partners, particularly the World Bank and the World Health Organization, was satisfactory. The PCU staff's initial inexperience with project management and implementation and nonfamiliarity with ADB procedures were offset by their willingness to learn through seminars, workshops, and counterpart training with international consultants. Nevertheless, their unfamiliarity with project

11

management concepts and procedures for contracting services caused considerable delays during initial project implementation. L. Performance of the Asian Development Bank

42. The performance of ADB, which included 14 missions, was considered satisfactory, and good relationships with MOH were maintained throughout project implementation. ADB staff conducted review missions regularly and the Project benefited from ADB field visits to the project sites. A healthy relationship between ADB and the PCU was maintained through regular communication. Relationships between MOH and ADB staff strengthened as project implementation progressed, and resulted in successful working relationships for the follow-up Health Sector Support Project. ADB’s Resident Mission in Phnom Penh provided considerable support throughout project implementation.

III. EVALUATION OF PERFORMANCE

A. Relevance

43. The Project’s emphasis on human development and poverty reduction was fully aligned with and supportive of Cambodia's efforts to restructure and strengthen its health care delivery system. The Project’s components have contributed to the strategic thrusts of health care reforms, especially in the area of human resources development,9 and focused on the disadvantaged, on the poor, and on marginalized population groups. The midterm review strengthened the Project’s development objectives. The original scope of the Project was highly relevant, and was still highly relevant, particularly in terms of poverty alleviation, ADB's main goal, at project completion. B. Efficacy in Achievement of Purpose

44. The Project has significantly contributed to health care reforms, led by the CHSPP, which increased accessibility to primary health care and strengthened management capacity at the provincial and district levels. The CHSPP showed that contracting health services to NGOs was feasible and increased accessibility within a short time. Contracting health services, including promotive, preventive and curative services, could reduce infant, child, and maternal mortality among a larger segment of the rural population than conventional public sector service delivery mechanisms. The CHSPP also showed that efficiency gains in the provision of health services do not come at the expense of equity. Rather, improved efficiency appears to lead to better access to health services by the poor, relieving them of the burden of health care expenditures.10 Packages of basic medical equipment, furniture, essential drugs and medical supplies strengthened primary health care through minimum packages of activities provided at health centers and complementary packages of activities at referral hospitals. Public-private partnerships were established by contracting health services. The Project was considered efficacious.

9 Ministry of Health. 2002. Health Strategic Plan, 2003–2007. Phnom Penh. 10 ADB. 2002. Achieving the Twin Objectives of Efficiency and Equity: Contracting Health Services in Cambodia .

Manila.

12

C. Efficiency in Achievement of Outputs and Purpose

45. Project implementation was efficient, with no major problems.11 The implementation arrangements and management structures and procedures were in accordance with those agreed on in the loan agreement and at appraisal. The loan proceeds, with adequate counterpart funds, were utilized efficiently for project implementation and the loan achieved its expected outputs. The annual turnover ratio improved over time from 0.89 in 1997 to 1.49 in 2003. The audited financial statements were satisfactory and were submitted to ADB in a timely manner. Under the CHSPP, out-of-pocket expenditures by the poorest 50% of the population have declined significantly and the poor were using the public health facilities. Direct observation of health facilities and interviews with patients indicated that the quality of care and health care coverage had increased significantly. D. Preliminary Assessment of Sustainability

46. The Project has made significant contributions to the development of the health sector in Cambodia, including by providing good quality health centers, by improving maintenance at health centers and referral hospitals, and by enhancing management and supervisory systems at MOH and at the provincial and district levels. However, MOH will continue to need external assistance because of the large areas of the country that still need to develop basic health infrastructure. The health care infrastructure provided under the Project is most likely sustainable, although additional resources are required to increase capacity so as to achieve expected improved health status. 47. The CHSPP was an operations research activity and not a continuous intervention. The contracted districts could return smoothly to conventional government management or could be run using another innovative approach the government might adopt without disruption of service delivery. The sustainability of the research itself is contingent upon MOH’s use of the evaluation findings to develop new approaches to service delivery nationwide. E. Environmental, Sociocultural, and Other Impacts

48. The Project has contributed to environmental awareness through the incorporation of relevant topics in management skills training and health education. Construction used environmental friendly designs and was carried out in accordance with Government regulations. Management of medical wastes commensurate with the appropriate medical technology and level of curative care services was provided at referral hospitals. The Project has significantly increased the rural population’s access to basic health services and enhanced their well-being, particularly that of women, children, and the rural poor. At the start of the Project, only 10% of the population in the five provinces were served by health centers. By the midterm review, more than 67% had such access, and this was expected to reach 85% by project completion. The services provided are those of greatest benefit to the poor and to women and children, namely, treatment of common diseases at a minimal cost (with full exemption for the indigent), immunization of children, and provision of family planning and prenatal care. In addition, job opportunities increased for women trained as nurses and midwives who could provide primary

11 Economic analysis using: ADB. 1999. Economic Analysis of Health Sector Projects: A Review of Issues, Methods.

and Approaches. Manila; ADB. 2000. Handbook for the Economic Analysis of Health Sector Projects. Manila. –was not possible because of a lack of reliable baseline data. The economic internal rate of return and financial internal rate of return could not be estimated because they were not computed at appraisal, but some financial assessments were made based on tracer studies as described in Appendix 3.

13

health care at health centers and referral hospitals. The Project’s institutional, environmental, social, and other impacts are considered substantial.

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS

A. Overall Assessment

49. The Project achieved the development targets set at appraisal, and is considered successful (Appendix 11). The extremely poor baseline indicators in the project provinces have improved significantly. At the start of the Project, many health centers had temporary structures and were dilapidated, and only 20% of health centers provided the minimum package of activities. The operational districts were poorly organized and their staff had minimal management and supervisory skills and support. About 22% of the referral hospitals had 60 usable inpatient beds, and only referral hospitals in the provincial capitals provided minimal surgical services. Maternal and child health care was minimal, while only about half of those in the poorest 50% of the population received adequate treatment for their illnesses. 50. Local contractors constructed the health centers and renovated referral hospitals closely supervised by an international engineering firm. The construction was timely coordinated for provision of medical equipment, drugs and medical supplies, and essential trained personnel. Adequate commissioning facilitated the start-up of services and improved the utilization of services through effective supervisory systems by operational districts for health centers and by PHDs for operational districts. The Project contributed to the reduction of preventable mortality and morbidity and has benefited about 4.1 million of the rural population. The number of operational health centers in the project provinces increased from 78 (22.3% of MOH’s Health Coverage Plan) to 285 (81.9% of MOH’s Health Coverage Plan). Utilization of health centers for curative care increased from 7.1% to 17.6% and reduced the use of referral hospitals for such care. The utilization of promotive and preventive services also increased, including child immunization, prenatal tetanus immunization, antenatal care, vitamin A supplementation for children, and contraceptive use. 51. The Project developed and initiated basic management training for DHMTs in the project provinces that was extended nationwide. The Project developed three ISCs for (i) supervision of health centers by DHMTs, (ii) supervision of referral hospitals by PHD staff, and (iii) supervision of DHMTs by PHD staff. The supervisory instruments and training strengthened capacity building at the provincial and district levels. 52. The Project examined the feasibility, benefits, and cost-effectiveness of contracting health services to NGOs at the district level. Two contracting models were tested in five districts, with four other districts used as controls. Household and health facility surveys were conducted to collect baseline data and were repeated in mid–2001. The results of an external evaluation found that contracting significantly improved health care coverage, especially for the poor, was more cost-effective, and decreased out-of-pocket expenditures by the poor. B. Lessons Learned

53. Contracting health services to nongovernmental entities is feasible, cost-effective, and equitable and effectively targets and benefits the poor. This innovative approach has rapidly provided services to more people, and reduced infant, child, and maternal mortality. The contracting out model was preferable to the contracting in model. The contracting model could be applied to (i) outreach services, (ii) health centers and referral hospitals and their catchment areas, (iii) a district or districts in a province, and/or (v) entire provinces. In addition,

14

appropriately designed and managed user fees systems can increase utilization by the poor and decrease their out-of-pocket expenditures on health care. 54. The development of health care facilities, particularly referral hospitals, requires clear operational policies at the planning and design stages. Design should incorporate site works, utilities (including power and water supply), and adequate cost estimates. The construction, which should be coordinated with the supply of equipment, drugs, and trained personnel, should be monitored closely.

55. Decentralization of health service management to provincial levels is crucial where small but adequate amounts of resources are provided to enable the management and supervisory systems at the district levels to implement managerial decisions and address problems identified. C. Recommendations

1. Project–Related

56. Project objectively verifiable indicators to be identified at baseline or at early project implementation, monitored closely during implementation, and verified after Project completion.

57. The covenants as agreed under the loan and project agreements should be maintained.

58. The contracting of health services should be continued, particularly in provinces with large poor rural populations, and poor health services performance. Local NGOs should be encouraged to participate; a user fee system and an equity fund for hospitalization under the contracting of health services should implemented; and the roles of PHDs, the relationships with HSCs, and the mechanisms to ensure a positive stake in the contracting should be defined. 59. MOH should continue to improve its maintenance program for health centers, referral hospital buildings, and medical equipment. 60. A project performance audit is proposed during or after mid–2005 to help ensure that project facilities are fully operational and to measure the Project’s impacts.

2. General

61. Project appraisal should (i) ensure thorough assessment of the executing agency’s institutional capacity, (ii) estimated the costs of construction and equipping of health centers and referral hospitals, and (iii) planning of project activities and practical project scheduling. 62. For project implementation, both ADB and the executing agency need to monitor (i) implementation progress through a well-designed quarterly reporting system; and (ii) agreed project indicators, including contract awards and disbursements.

Appendix 1 15

PROJECT FRAMEWORK

Design Summary

Performance

Indicators/ Targets

Monitoring

Mechanisms

Assumptions and Risks

1. Goal Improve health status of the rural

population by reducing preventable mortality and morbidity, particularly among women, children and the poor, in the targeted provinces

By 2002 Infant mortality rate reduced by 12% from 115 to 101 live births per 1,000, under five mortality rate reduced by 12% from 181 to 159 live births per 1,000, and maternal mortality rate reduced from 650 to 455 live births per 1,000.

Demographic and health surveys National health survey 2002

The Government continues to regard health as a high priority. Political and civil stability maintained. Aggregate level of international support for health is maintained. Current annual economic growth rate is maintained through project period.

2. Purpose Increase accessibility to improve health care services for the rural population, especially women and the poor.

Increase the use of health services by the rural population, especially women and the poor

By 2002 The number of pregnant women having at least one antenatal care visit performed by a trained health worker increased from 48% to 95%. The number of women of reproductive age receiving at least two doses of tetanus toxoid increased from 33% to 80%. Deliveries attended by trained health worker (from 40% to 70%). Children 12–23 months have been immunized prior to their first birthday (from 50% to 90%). Eighty percent of children aged 6 months to 5 years have received vitamin A capsules as supplement in the last 6 months. Community use of MOH facilities for outpatient services (health centers and referral hospitals), other than immunization, increases in total admissions to referral hospitals (compared with admissions to currently existing hospitals) for diseases other than tuberculosis.

Demographic and health surveys National Health Survey 2002

Royal Government of Cambodia continues to regard health as a high priority. Political and civil stability maintained. Aggregate level of international support for health is maintained. Current annual economic growth rate is maintained through project period.

Establish healthy lifestyle among the rural population, particularly women, children, and the poor.

Increase positive health seeking behavior among the rural population, particularly women, children, and the poor.

By 2002 Married women of reproductive age, 15–45 years, know at least three modern methods of contraception and a source of supply for each (from 36% to 50%). Fifty percent of mothers know

Demographic and health surveys National health survey 2002

Cambodia continues to regard health as a high priority. Political and civil stability maintained. Aggregate level of international support for health is maintained. Current annual economic growth rate is maintained throughout the

16 Appendix 1

Design Summary

Performance

Indicators/ Targets

Monitoring

Mechanisms


percent of mothers know

warning signs of acute respiratory infection that should be seen by a trained health worker and know how to prepare oral rehydration salts.

project period.

3. Output 3.1. Component 1: Strengthening Community Health Services

Subcomponent 1.1 Constructing and Renovating Health Centers and Referral Hospitals Increase accessibility to affordable basic, curative, and preventive care services for the rural population, particularly women, children, and the poor.

Construct 251 new health centers and renovate 75 existing health centers, and renovate 20 referral hospitals. Basic medical equipment and essential drugs will be provided as minimum package of activities kits will be provided to health centers and complementary package of activities kits to referral hospitals.

Eighty percent of health centers are fully operational, defined as (i) having an adequate supply of essential drugs (80% of drugs in stock), (ii) having at least two health workers at each health center on a full-time basis, and (iii) having been supervised eight times in the last year. Eighty percent of referral hospitals are also fully operational, defined as (i) having an adequate supply of essential drugs (80% of drugs in stock), and (ii) having been supervised eight times in the last year. Fifty percent of referral hospitals with capacity to perform caesarian sections, defined as having trained staff, equipment, facilities, and having performed at least five in the last year. Fifty percent increase from baseline in the score health centers obtained on a standardized supervisory instrument related to the quality of patient care, the availability of family planning services, outreach activities, cleanliness, and health center management. Fifty percent increase from baseline in the score district hospitals/referral hospitals obtained on a standardized supervisory instrument related to the quality of patient care and hospital management. Eighty-five

Demographic and health surveys Quarterly progress reports Review missions

Sites for health centers and referral hospitals are accessible during rainy season so that construction can continue. Quality of local contractors is available to construct and renovate health centers and referral hospitals.

Appendix 1 17

Design Summary

Performance

Indicators/ Targets

Monitoring

Mechanisms


percent of diagnosed

tuberculosis cases receive complete short course therapy and become sputum negative.

Subcomponent 1.2 Providing Equipment and Essential Drugs Basic medical

equipment and essential drugs will be provided as minimum package of activities kits to health centers and complementary package of activity kits to referral hospitals.

Provision of minimum package of activities kits (equipment and essential drugs) to health centers, complementary package of activity kits (including basic, surgical, obstetrics, and emergency equipment) to 20 referral hospitals, x-ray units and generators to 20 referral hospitals.

Quarterly progress reports Review missions

Basic medical equipment lists for minimum package of activities and complementary package of activity kits are being updated. Requirements for equipment are based on the level of services defined by the Health Coverage Plan. Timely delivery of equipment and drugs by the central medical stores and supervised by MOH.

Subcomponent 1.3 Maintaining Selected Health Facilities Establish preventive

maintenance and repair of health care buildings. Health staff will emphasize appropriate cleanliness.

Newly constructed and renovated health centers and renovated referral hospital buildings are well maintained.

Supervisory and maintenance reports

MOH inventory document is regularly updated and administered by MOH. Its budget for maintenance and repair of health facilities buildings is available and adequate.

3.2. Component 2: Strengthening of District Health Offices Subcomponent 2.1 Improving Management Capacity

Strengthen capacity, management, and supervision of MOH, PHDs, and operational districts. Establish district health management teams for effective district health management

Improve health sector performance at the district and provincial levels


MOH continues to decentralize and devolve health management to PHDs and operational districts.

Subcomponent 2.2 Strengthening the Supervision of the Health Centers Improved

supervision of health centers

Increased accessibility and improved minimum package of activities


Continued support by MOH and PHDs to operational districts

Subcomponent 2.3 Strengthening Selected Referral Hospitals

Improved supervision of referral hospitals

Increased accessibility and improved complementary package of activities


Continued support by MOH and PHDs to operational districts

18 Appendix 1

Design Summary

Performance

Indicators/ Targets

Monitoring

Mechanisms


3.3. Component 3: Introducing Health Sector Reforms

Subcomponent 3.1 Contracting Out Health Services

Health service contractor with full authority and funds for staffing, management, and operational costs provides services.

Improved health services delivered were based on the contract. As specified in the contract, utilization of health services per capita by the poor increased.


All relevant levels of government remain committed to the alternative service delivery mechanisms.

Subcomponent 3.2 Contracting In Management Services Health service

contractor provides technical support to the operational districts, but the running costs for health services remain the responsibility of MOH. The control districts are under MOH’s management system.

Improved health services delivered based on the contract As specified in the contract, utilization of health services per capita by the poor increased. Service delivery targets as set by MOH


MOH and PHDs continue to provide drugs, equipment, and operating budget to contracted operational districts.

Subcomponent 3.3 Setting up Community Loan Scheme for Health Emergencies Establish a

community loan scheme to prevent families from becoming destitute as a result of catastrophic medical expenses

Reduce poverty in the community

Quarterly progress reports Review mission

Communities actively participate in health activities and organize such a scheme

3.4. Component 4: Support for Central Office Management

Increase MOH capacity for project management

Project activities, management skills, procurement, and accounting training effectively carried out to meet physical and financial reporting requirements.

All quarterly progress reports produced and schedules are of acceptable quality All activities performed according to project schedule

Quarterly progress reports Financial audited and audited statements

Key MOH staff can participate in and contribute to project implementation.

Subcomponent 4.1 Setting up a Project Coordination Unit Strengthen management and technical skills of MOH staff. Develop objectively verifiable indicators for the Project.

Improve capacity for procurement, disbursement, accounting, and so on by training MOH staff in technical skills and in reporting on project implementation.

Project implementation according to agreed schedule


MOH can assign, train, and retain qualified and capable staff.

Appendix 1 19

Design Summary

Performance

Indicators/ Targets

Monitoring

Mechanisms


Subcomponent 4.2 Carrying Out Benefit Monitoring and Evaluation

Strengthen capacity of MOH Develop objectively verifiable indicators for the Project

Establish a practical system of monitoring and evaluation Improve quality of routine data collected for monitoring and evaluation Monitoring and evaluation information is used for policy and program planning. Assess effectiveness of contracting-out and contracting-in.

Eleven objective verifiable indicators were developed for the Project.

Baseline survey Household survey Health facility survey Field visits and spot checks

MOH can assign, train, and retain qualified and capable staff.

Subcomponent 4.3 Strengthening Equipment Maintenance and Repair Capabilities MOH’s capacity for equipment maintenance and repair increased.

Planned, preventive medical equipment maintenance and repair plan included as part of MOH’s annual operational plan

Supervisory and maintenance reports

MOH’s budget for maintenance and repair of equipment is available and adequate.

Inputs by Subcomponent

Civil Works $6.50 million Land Acquisition $0.20 million Equipment, Furniture, and Vehicles $2.70 million Medical Supply and Drugs $4.40 million Staff Development $0.50 million Consulting Services $0.20 million Surveys and Evaluation $0.50 million Pilot Tests $4.20 million Project Implementation $0.90 million Operation and Maintenance $0.70 million Physical Contingencies $2.00 million

Price Contingencies $1.70 million Service Charges $0.40 million Total $25.00 million

Contract awards Withdrawal applications Quarterly progress reports Review missions Project accounts

Availability of counterpart funds.

MOH = Ministry of Health, PHD = provincial health department.

20 Appendix 2

Comparison of Key Planned and Actual Outputs by Project Component

Component/Activity At Appraisal Actual

I. Strengthening Community Health Services The Project will strengthen community health services by (i) constructing or renovating health centers, and renovating referral hospitals, (ii) providing basic medical equipment and essential drugs, and (iii) maintaining health centers

The Project would support the construction of 251 new health centers and renovation of 75 existing health centers, the provision of essential drugs and equipment, and the establishment of a facility and equipment maintenance capabilities in the project provinces. The Project also planned to renovate 20 district referral hospitals, including upgrading the emergency obstetric and trauma care.

Civil Works Package The Project scope was reduced to constructing 187 new health centers, renovating 20 existing health centers, and renovating 13 referral hospitals. In addition, two operational districts were also renovated. This scope change was to accommodate the higher than expected expenditures for civil works and contracting of health services. The project was completed satisfactorily in relation to health centers, referral hospitals, and operational districts.

Health center staff received training in the minimum package of activities under Loan 1368-CAM: Basic Skills Project (Part B: Health Personnel Training).1

Equipment, Furniture, Supplies, and Drugs Basic medical equipment, furniture, medical supplies, and essential drugs were procured and delivered to the project health centers and referral hospitals as planned. The health centers were well utilized, with consultations ranging from 100 to 400 per month, and collected user fees ranging from KR100 to KR500 per outpatient visit depending on the community health committees and the location of the health center.

Maintenance of Health Centers and Referral Hospitals Maintenance of health centers and referral hospitals buildings was particularly effective in terms of cleanliness. Some health centers and referral hospitals were affected by the 2000/2001 floods and were repaired under Loan 1824-CAM: Emergency Flood Rehabilitation Project (Part E: Health).2

II. Strengthening District Health Offices The Project plans to strengthen the district health offices (now known as the operational districts) by i) improving their management capacity, (ii) strengthening the supervision of health centers, and (iii) strengthening selecting referral hospitals.

The Project would strengthen the capacity of district-level managers by (i) training them in management, budgeting, and related skills; (ii) developing and implementing the use of supervisory checklists for use in supervising health center and

Managing Basic Health Services This technical assistance3 (TA 2567- CAM) was provided to (i) strengthen the capacity of DHMTs in relation to relevant management skills, (ii) help establish supervisory systems for health centers and referral hospitals, and (iii) introduce the concepts and methods of benefit

1 ADB. 1995. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to

Cambodia for Basic Skills Project. Manila. 2 ADB. 2000. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to

Cambodia for Emergency Flood Rehabilitation Project. Manila. 3 ADB. 1996. Technical Assistance to Cambodia for Managing Basic Health Services. Manila.

Appendix 2 21


referral hospital staff; and (iii) providing support to DHMTs by providing office supplies and equipment, plus funds for supervision and by renovating offices.

monitoring and evaluation.

Improving Management Capacity A 2-week management training course for DHMTs covered planning, finance, budgeting, supervision, and essential drugs management. All DHMTs in the project districts completed the training course by December 1998.

Supervisory Systems Integrated supervisory checklists have been developed for supervising (i) DHMTs' management activities, (ii) health center staff, and (iii) referral hospital staff. The integrated supervisory checklists were implemented nationwide commencing in March 2000.

Support for DHMTs Office equipment and supplies, and petty cash have been provided to support supervisory activities and DHMT offices. Where necessary, DHMT offices have been renovated, either separately or as part of the renovation of the referral hospitals.

III. Introducing Health Sector Reforms The Project will pilot test innovative approaches before they are implemented nationwide. To be pragmatic and for easy replication of the achievements, the Project will focus its resources on a small but well-defined area. The Project will select five districts to use as a control, five districts that will contract out health services, and five districts that will use contracting in.

The Project planned to test innovative alternative approaches to health service delivery which the Government contracts with private organizations to provide health care services. Two different models were to be tested: contracting out and contracting in. A community loan scheme for health emergencies would also be established. Under contracting out, the health services contractors have full authority and funds for staffing, management, and operational costs. Under contracting out, the health services contractors provide technical support to the operational districts, but running costs for health services and staff salaries remain the responsibility of MOH. However, a budget supplement would be provided to allow for unbiased comparison.

Contracting Health Services Pilot Project The CHSPP started 6 months later than originally planned. A baseline survey was carried out, and evaluation indicators were developed. An MOH working group, assisted by an international contracting specialist, developed terms of reference and guidelines for the contracting out and contracting in districts. It was decided to provide a supplement budget to augment the minimal government budget allocations for contracting in and control districts. Meetings were held at operational district and provincial health department levels to promote public awareness. Only 12 districts were found to be suitable for CHSSP, four in each category (control, contracting out, contracting in). However, no acceptable bids were received for three districts, and because of funding constraints, it was decided not to rebid. Consequently, the final composition of districts was two for contracting out, three for contracting in, and four controls. By December 1998, contracts were awarded to international NGOs with prior experience in Cambodia.

22 Appendix 2


The dis tricts were Memut (under Save the Children Australia) and Ang Rokar (under the Association of Medical Doctors of Asia) for contracting out; Cheung Prey (under Save the Children Australia), Peareng (under Health Net International based in the Netherlands), and Kirivong (under Save the Children France). The control districts were Preah Sdach, Kamchay, Bati, and Kroch Chhmar.

Contracting Out The contracting out health services contractors were mobilized in January 1999, and took over the management of their operational districts by March–April 1999. They purchased their own drugs and consumable supplies following ADB's procurement guidelines. The contracting out districts and the provincial health departments have arrived at mutually acceptable working arrangements and achieved satisfactory performance.

Contracting In The contracting in health services contractors took over the management of their operational districts by March–April 1999 after mobilization in January 1999. They reinforced discipline using existing MOH staff, and the budget supplement. The contracting in system has established cost recovery, and provided income to staff as incentives. This has proven essential as government salaries alone are inadequate to motivate staff. The contracting in districts rely on drugs and equipment provided through normal MOH channels, supplemented by small purchases using budget supplement funds. The contracting out model was preferable to the problematic contracting in model. The contracting model could be applied at (i) outreach services, (ii) health centers and referral hospitals and their catchment areas, (iii) a district or districts in a province and/or (iv) entire provinces.

Community Loan Scheme for Health Emergencies The Project will pilot test a community loan scheme to prevent families from becoming destitute as a result of catastrophic medical expenses. The Project will hire an NGO that is already running a rural microcredit scheme.

Community Loan Scheme MOH and ADB agreed to drop the scheme early during project implementation because of (i) the lack of funds for this activity given that the contracting health services would be using all the funding for this subcomponent, (ii) the contracting health services had introduced a form of community scheme.

Appendix 2 23

Component/Activity At Appraisal Actual IV. Providing Support for Central Office Management

A. PCU The Project will establish a PCU cofinanced with the World Bank for procurement, disbursement, accounting, technical skills training of MOH staff, and reporting to both ADB and the World Bank.

A joint ADB-World Bank PCU was established as planned to cofinance the costs of PCU management and administrative and financial functions. The total number of PCU staff was increased from 8 to 17 with ADB's concurrence. Provincial project units were established in each provincial health department in the project area. The coordinators of the latter maintained close relationship with the PCU and with operational districts.

B.

Benefit Monitoring and Evaluation

The Project would assess the effectiveness of the new approaches to service delivery on contracting based on the baseline household and health facility survey to be conducted in the first year of project implementation. The purpose would be to compare project provinces on the basis of objectively verifiable indicators established for the Project.

The baseline household survey was completed in 1998 as part of the 1998 national health survey. The Government carried out a national demographic and health survey in 2000. A comprehensive household survey was carried out in 2000.

C. Strengthening Equipment Maintenance and Repair Capabilities

The purpose was to improve maintenance and repair of medical equipment in MOH. The Project would provide a consultant to formulate a policy that would address the role of the private sector and NGOs, MOH maintenance and repair staff, and mechanisms for planned preventive maintenance. The Project would fund study visits and implement an equipment maintenance plan in the project provinces.

An international consultant recruited by MOH for 6 person-months, developed, assisted in the implementation, and evaluated a provincial system for medical equipment maintenance. The system covered health centers, district hospitals, and provincial hospitals. The system was revised based on field experience and implemented nationwide. The Medical Engineering Center of the Cambodian Red Cross assisted the consultant. Training was organized locally by the consultant. The study visits did not take place. The resulting excess budget was reallocated to the CHSPP.

ADB = Asian Development Bank, CHSPP = Contracting Health Services Pilot Project, DHMT = district health management teams, MOH = Ministry of Health, NGO = nongovernment organization, PCU = project coordination unit.

24 Appendix 3

EVALUATION OF THE CONTRACTING OF HEALTH SERVICES PILOT PROJECT

A. Design

1. The contracting health services pilot project (CHSPP) supports a pilot program to test innovative alternative approaches to health service delivery through operational research on the effectiveness and efficiency of contracting with nongovernment organizations (NGOs) and private sector groups to provide basic health services on the Government’s behalf.1 Two different models of contracting were tested and evaluated in relation to their effectiveness in health system development and service coverage in operational districts, with selected government-run districts serving as a point of comparison. The arrangements were as follows:

(i) Under the contracting out model, the contractor has full authority and funds for staffing, management and operational costs, and medicines and consumable supplies,

(ii) Under contracting in model, the contractor receives funds for providing technical

support to the operational district. Running costs for health services, medications and other consumable supplies, and staff salaries are the responsibility of the Ministry of Health (MOH). The contractor may transfer staff within the operational district and make recommendations regarding staff issues to the provincial health department (PHD), but hiring or termination are the responsibility of the PHD and MOH.

(iii) The control districts remain under the normal MOH management system, with

the district management health teams (DHMTs) responsible for districts’ health activities. The control districts receive the same management and supervisory support as other MOH districts. A small budgetary supplement was provided to enable an unbiased comparison.

The original plan was to contract 10 districts (5 for contracting in and 5 for contracting out) and to select 5 districts as control districts. The original budget to support the health service contracts was US$ 4.28 million. 2. An initial assessment of selected operational districts was conducted, and only 12 operational districts were suitable, including four for contracting out, four for contracting in, and four for controls. However, the number of districts was further reduced after tendering because of a lack of bids. The final number of districts was two for contracting out, three for contracting in and four for controls. A baseline survey was conducted that showed that operational districts received low levels of government budgetary support. As a result, the project designers decided to provide an annual budget supplement of $0.25 per capita per operational district for running costs for the contracting in and control districts. 3. The limitations of the CHSPP’s design were as follows:

(i) The inputs and their corresponding costs were underestimated, and the local private sector and NGOs lacked the capacity to participate,

1 ADB. 1996. Report and Recommendation of the President to the Board of Directors on the Proposed Loan to

Cambodia for Basic Health Services Project. Manila.

Appendix 3 25

(ii) The chain of command between PHDs and contractors was not clear. This affected contracting in most severely, because contractors depended on the PHDs for operating budgets, medical supplies, and staff. District-level relationships between contracting in and DHMTs were also poorly defined, leading to considerable confusion, ambiguity, and conflict.

(iii) The original design and budget did not anticipate the use of international NGOs

as contractors, which caused a number of difficulties. However, the international NGOs have served an important function as pioneers in this pilot and could contribute considerably to the development of local organizations capable of implementing such contracts in the future.

(iv) The methods for procuring medical equipment and drugs as specified in the

Project Administration Memorandum were not suited to the nature and value of the goods, causing significant delays and service delivery constraints.

4. While the contracting out model proved feasible to implement with no major changes required, the contracting in model was impossible to test as originally designed. Specific flaws were the management of a district health system depending fully on MOH for its operating budget, drugs, equipment, and supplies. Contracting in was successfully implemented after significant changes and efforts by contractors and MOH officials. B. Implementation

5. A contracting specialist, assigned to the CHSPP from February 1997 to December 1998 developed evaluation indicators, conducted a baseline survey in the selected districts, and supported an MOH working group that developed the terms of reference and guidelines for the contracting in and contracting out models. The MOH working group and Project Coordination Unit (PCU) introduced the contracting concepts and helped PHDs and operational district staff and civil authorities in the provinces prepare for the changes after 1998. Invitations to bid started in August 1998, 16 bids were received in October 1998, and evaluation of bids was conducted in December 1998. Two contracts were awarded for contracting out (Save the Children Funds/Australia) in Memut in Kampong Cham province and the Association of Medical Doctors of Asia in Ang Rokar in Takeo province) and three contracts were awarded for contracting in (Save the Children Funds/Australia in Cheung Prey in Kampong Cham province, Health Net International in Prereang in Prey Veng province, and Save the Children, France in Kirivong in Takeo province. The four control districts were Preah Sdach and Kamchay Mear in Prey Veng province, Bati in Takeo province, and Kroch Chhmar in Kampong Cham province. MOH made health centers and referral hospitals, basic medical equipment, furniture, and essential drugs available to the contractors according to pre-determined specifications. All districts under the CHSPP were to achieve a defined package of health services and level of population coverage for specific services by the end of four years. Baseline data were to be obtained for all districts, including the controls. Repeat surveys were planned to compare progress achieved in health system development, health service coverage, and cost-effectiveness across contracting in, contracting out, and control districts. 6. The CHSPP evaluation survey highlighted the following strengths and achievements:

(i) The staff of MOH and the PCU acquired skills in contract management and monitoring, including trouble-shooting and resolving conflicts between health

26 Appendix 3

service contractors and PHDs and/or DHMTs. MOH and PCU staff was also able to help contractors deal with staff unrest during the early transition period.

(ii) The PCU established an effective monitoring system, closely supervised by a

monitoring group headed by the director general of health. The monitoring system was capable of identifying and remedying service delivery problems.

(iii) The contractors rapidly developed district health systems and improved the

health sector’s performance by means of innovative approaches that would have been difficult or impossible for conventional MOH districts to implement. Contracting out districts have been able to ensure the availability of the necessary number and type of staff, maintain staff accountability, and enforce regulations. Although more constrained than contracting out, contracting in districts have been able to effect significant improvement in staff accountability and regulation. The control districts were able to develop action plans and improved the reporting requirements.

C. Weaknesses and Constraints

7. The CHSPP evaluation survey raised the following weaknesses and constraints:

(i) The number of staff in health centers and referral hospitals in contracting in and control districts was inadequate.

(ii) The contracting in districts received low levels of operating funds from the

provincial treasury and PHDs and a small amount under MOH's Priority Action Program.

(iii) The PHDs experienced a reduction in their control over staff and resources as a

result of the contracting of operational districts and obtained no tangible benefits in return.

(iv) The control districts required the PCU’s assistance to develop plans for the use

of the budget supplements. Accountability for the use of funds was generally poor in the control districts;

(v) The better-resourced National Health Programs are de facto autonomous entities

with their own direct funding by external funding agencies. The programs continued to function in a vertical manner, bypassing the DHMTs in conducting direct supervision at the health center level and issuing instructions to health center staff.

(vi) The international NGOs acting as contractors in some cases encountered

difficulties initially for both parties, partly because of their superiority attitude towards MOH, PCU, the PHDs and operational districts.

(vii) The procurement of basic medical equipment and essential drugs became

problematic because of unfamiliarity with the Asian Development Bank’s (ADB’s) procurement guidelines. The contractors expected tax-exempt procurement supported by ADB and approved by the Ministry of Finance. The contracting in contractors and the control districts encountered significant delays in the

Appendix 3 27

procurement and distribution of the equipment and drugs carried out by MOH’s central medical stores.

D. Review of Health Services Contractors2

1. Capacity Building, Staff, and Systems Development

8. The contractors carried out the following key activities in relation to capacity building:

(i) Situational analysis exercises were carried out for the particular district’s health situation and health care facilities in preparation of an annual operational plan.

(ii) Half-day sessions on problem analysis. (iii) Assessment of training needs for the minimum package of activities for health

center staff and the complementary package of activities (under MOH’s 1997 Health Coverage Plan) for referral hospital staff. Clinical training was provided for staff of PHDs, operational districts, referral hospitals, and health centers staff.

(iv) Weekly presentations and lectures organized by technical referral hospital staff.

(v) Training on logistics and management of pharmaceutical drugs and medical

supplies.

(vi) Training of more than 200 village health volunteers, particularly for community-based surveillance system.

(vii) Monthly district health management committee meetings and sharing of

information in the operational health district (OHD). 9. In addition, visiting trainers from the National Institute of Public Health, PHDs, the National Tuberculosis Center, regional training centers, and NGOs also provided training.3 Training was also organized to improve health delivery systems and systems development by means of (i) community health action committees in the contracted districts, (ii) health center management and feedback committees for health centers, (iii) technical working group in referral hospitals, and (iv) district health advisory teams. 9. Management teams organized by the contractors (i) reviewed districts’ morbidity and mortality status; (ii) assessed patient management in health centers and referral hospitals; (iii) reviewed the rationalization of drug use enforced through the use of the MOH integrated supervisory checklists and the expanded program on immunization supervisory checklist for current year; (iv) prepared reports following monitoring visits to health centers and referral hospitals; (v) assisted in the population census; (vi) participated in the joint DMU-contractor inventory review and in the current year work plan jointly prepared by project staff, DMU, referral hospital and health center staff; (vii) enforced staff discipline; and (viii) participated in monthly

2 Based on the quarterly reports submitted by the contractors for 2001–2002 on activities carried out by the health

service contractors in the contracting out and contracting in districts. 3 International NGOs, including Save the Children, UK, Reproductive and Child Health Alliance, Aide Odontologique

Internationale, Population Services International, Medicine Without Frontier, and local NGOs including Action for Health, Nak Aphiwat Sahakum .

28 Appendix 3

district health management committee meetings and ensured that information was shared across operational districts.

2. Delivery of Health Care

10. To improve the delivery of health care services in the contracted districts, the contractors carried out the following activities: (i) promoting services available at health centers and referral hospitals and referral services; (ii) mobilizing community participation by training traditional birth attendants and other volunteers by means of regular refresher courses organized by Project and DMU staff; (iii) providing outreach services for remote, inaccessible villages; (iv) providing health services at schools, e.g., launching a diarrhea/cholera control campaign? and developing educational materials; (v) running health education and information, education, and communication activities to disseminate information on such topics as diarrhea and home-made oral rehydration salts; and (vi) coordinating with local governments to arrange for regular garbage collection.

11. Key activities for improving the quality of care included (i) peer discussions on the management of difficult cases; (ii) provision of basic equipment, furniture, and supplies for minor surgery in referral hospitals; (iii) feedback on performance goals from staff; (iv) patient counseling and prosthetics provided by CIOMAT; (v) weekly presentations and lectures by referral hospital technical staff and visiting trainers from National Institute for Public Health, regional training centers, PHDs, and NGOs; and (vi) monthly meetings to resolve health care delivery problems.

3. Community-Based Activities and Services

12. Because of its cost-effectiveness in reducing infant, child, and maternal mortality, the contractors emphasized preventive health care. They focused on maternal and child health and encouraged active participation by the community through elected health center management and feedback committees. The contractors also set up district health technical advisory committees. Finally, health action groups discussed issues raised by means of suggestion boxes placed at health facilities. 13. The health centers organized regular classes for mothers and well-baby clinic sessions that emphasized personal hygiene; safe delivery at health facilities; importance of antenatal care; tetanus toxoid immunization, and breast feeding immediately after delivery; prevention and treatment of anemia and neonatal tetanus; danger signs during pregnancy and awareness of high-risk mothers; use of trained midwives during delivery; and monitoring of all deliveries. 14. The contractors also organized mass education campaigns on general hygiene and sanitation, whooping cough, measles, malaria, dengue fever, immunizations, tuberculosis, acute respiratory infections, and cholera. In addition, the participation of contractors increased the collection of health data. For this purpose the contractors trained village health volunteers to maintain community based surveillance system.

4. Cost Sharing and User Fee System

15. The contracting in model, as originally designed, failed even before implementation. This was mainly because contracting in managers were unable to hire, fire, or transfer staff or prevent interference by MOH or the PCU. With modifications, the contracting in model implemented by Health Net International (HNI) has successfully implemented a user fee and

Appendix 3 29

staff performance contract system. 4 With a combination of economies of scale and user fees, HNI can provide satisfactory district-wide health services for about $3.50 to $4.00 per capita per year (recurrent costs). Pereang district, which is under HNI contracting in, has a population of about 180,000, 15 health centers, 1 operational district, and 1 referral hospital. HNI’s cost-sharing innovations were based on a user fees system linked to performance-based staff incentives and agreements with health workers in public health care facilities not to provide services on a private basis. This system reduced overall out-of-pocket expenditure in the district by 40%, while access to health services by the poorest 50% of the population improved significantly. HNI replaced its initial individual contracts with health workers with sub-contracts with health facility managers who had demonstrated good management skills. HNI paid the health center sub-contractors a lump sum based on actual achievement in key result areas. This innovative approach by HNI was an instant success and allowed for further improvement of all output indicators in the district. E. Survey Methodology and Findings

1. Methodology

16. The National Institute for Public Health,5 MOH’s research arm, conducted the household and health facility surveys under the direct supervision of MOH and an international consultant. The sampling methodology and survey instruments conformed to those of the baseline survey to ensure comparability of the two datasets. The sample for the household survey consisted of the same random sample of villages as for the baseline survey. The sample consisted of a total of 270 villages, 30 in each of the 9 districts. The questionnaire used was identical in content and Khmer wording with the baseline instrument. The National Institute for Public Health also conducted surveys of referral hospitals and health centers in the 9 districts using an instrument identical to the original baseline survey. The final total was 17 health centers (100%) in contracting out districts, 29 health centers (60%) in contracting in districts, and 25 health centers (60%) in the control districts.

2. Summary of Major Findings

a. Health Service Coverage

17. In 1990, at baseline less than 20% health centers were functional with almost zero utilization, while during evaluation in 2001, both contracting in and contracting out districts had 100% of their health centers operational, and the figure was 69% for the control districts. However, referral hospitals had just started being operational, thus the impact of improved referral hospital services has not yet been fully realized. In relation to the use of each type of facility by individual household members in the previous year, use of the public health system was greatest in the contracting out districts and lowest in the control districts. One control district, Bati, was an exception. Utilization there was higher because of its close proximity to Phnom Penh and because health centers were managed by a non-contracted NGO. Table A3.1 shows the quality of care provided at health centers in terms of percentages of maximum possible points at evaluation in the contracting in, contracting out, and control districts.

4 Health Net International. 2001. Memorandum with Annexes of the Health Financing Support Visit in support of the

Peareng Contracting-In Project, Prey Veng Province, Cambodia. Prey Veng. 5 NIPH is the official research institute of the MOH, and has previous experience in conducting large-scale surveys,

including the 1998 National Health Survey.

30 Appendix 3

Table A3.1: Quality of Care at Health Centers

Total Pointsa

Baseline Evaluation

Percentage of Maximum Possible

(Evaluation Only) Control Districts 52

Bati 0 346 51 KamChai Mear 17 362 56 Kroch Chhmar 0 302 50 Preah Sdach 0 317 52

Contracting-In 61 Cheung Prey 0 428 63 Kirivong 0 518 63 Peareng 6 417 58

Contracting-Out 73

Ang Rokar 6 427 71 Memut 0 445 76

a Maximum possible points vary by district, depending on the number of health centers Source: National Institute for Public Health, 2001.

b. Maternal Health

18. The increase in antenatal care was roughly quadruple in the contracting out districts, more than double in the contracting in districts, and about one-and-a-half times above the baseline in the control districts, although the significant increase in two of the control districts was due to the Safe Motherhood Initiative funded by the United Nations Population Fund. Tetanus toxoid immunization coverage at health centers and referral hospitals and during immunization outreach increased significantly and by a similar magnitude in the control and contracting in districts, but much more in the contracting out districts. The increase in deliveries attended by trained personnel was minimal, and consisted mostly of women giving birth in their homes attended by trained government midwives. While most women believed that institutional delivery was safer, the poor state of rural roads and the high costs of transportation are significant barriers to deliveries at an institution.

c. Family Planning

19. The national family planning program began in 1995 under Cambodia’s maternal and child health program. The National Mother Child Health Center of the MOH conducted a survey in 1995 about people’s knowledge and use of contraception found that only 34.1% of married women knew one modern method of contraception. Rural women’s use of modern methods of contraception was 5.7% in 1995, 15.2% in 1998, and 17.6% in 2000. Changes in knowledge about birth spacing in CHSPP districts in 2001appeared to be significantly greater than in other districts nationwide. The overall increase in knowledge about modern methods of contraception and how to access them was greatest in contracting out districts; however, the greatest progress in narrowing the gap in knowledge between better off and poorer women occurred in contracting in districts. Women in the contracted districts were more likely to be using a modern method of birth control and to obtain that method from the public sector. The survey found that higher use in contracted districts was entirely due to the public sector.

Appendix 3 31

d. Child Health

20. In 1998, complete immunization coverage was only 38.8% nationwide and 38.2% in rural areas.6 Immunization coverage was limited because of the lack of funds, an ineffective cold chain and the weakness of outreach programs. The increase in fully immunized children was highest in contracting out districts and higher in contracting in than in control districts. In relation to vitamin A capsule consumption among children age 6 months to 59 months, in the contracting districts the increases were greatest among children from poorer families.

e. Curative Care

21. While curative care is not a prominent contributor to mortality reduction, the expenditures involved are an important cause of poverty, debt, and landlessness. Average out-of-pocket per capita spending is $30, or 11% of gross domestic product. Significant increases took place in the contracted districts, but little change was apparent in the control districts except in Bali district. 22. The two contracting out and two of the contracting in districts had no user fee system, while only one contracting in district had a user fee system that was administered by the contractor. Among the control districts, two had officially approved MOH user fee schemes and two districts had just started.

23. On average, the survey found that annual, private, out-of-pocket health expenditures per capita for the poorest half of households decreased significantly in the contracting out districts and in one contracting in district. A significant decrease also occurred in one control district that received assistance from a non-contracted NGO. The reduction in expenditures by the poor in contracting out districts was greater than the reduction in expenditures by the overall population, indicating that this approach is successfully targeting and reaching the intended beneficiaries. In addition to the direct costs of health care, the lack of readily available curative care of acceptable quality increases the amount of productive time lost due to illness. The contracting out districts saved people time because of the greater availability of an acceptable level of curative care at the district level.

f. Quality of Care

24. The assessment of the quality of care at health centers was based on the following criteria: (i) equipment, supplies, and record-keeping for immunization care, family planning, childbirth, and consultation; (ii) presence of functioning health centers; and (iii) presence of a documented referral system. Contracting in districts improved more than control districts and contracting out districts outperformed both. 25. The quality of care at referral hospitals was measured based on the following criteria: (i) presence of assigned staff; (ii) availability of drugs, equipment, and supplies; (iii) maintenance of hygiene and infection control; (iv) use by maternity patients; (v) use by children under the age of 5; (vi) use for tuberculosis; (vii) correct medical and nursing treatment of pediatric diarrhea; (viii) correct medical and nursing treatment of pediatric respiratory infections; (ix) correct medical and nursing care during labor and delivery; (x) management systems for quality control; and (xi) presence of a referral system.

6 National Institute for Public Health. 1998. National Health Survey 1998. Phnom Penh.

32 Appendix 3

Table A3.2: Quality of Care at Referral Hospitals

Total Points

Baseline Evaluation

Percentage of Maximum Possible

(Evaluation Only) Control Districts 80 261 55

Bati 60 160 34 KamChai Mear 50 316 66 Kroch Chhmar 160 276 58 Preah Sdach 50 293 61

Contracting In Districts 98 411 86 Cheung Prey 160 362 76 Peareng 75 438 92 Kirivong 60 434 91

Contracting Out Districts 50 380 80 Ang Rokar 75 399 84 Memut 25 362 76

26. As has been consistently true with all indicators, the contracted districts out-performed the control districts. However, the contracting in districts overall had a higher score than the contracting out districts, the reverse of the usual result observed. A major reason for this was that the referral hospital in one of the districts, did not have tuberculosis services.

g. Summary of Objectively Verifiable Project Indicators

27. Contracted districts consistently out-performed control districts with respect to the predefined coverage indicators, and contracting out districts performed considerably better than contracting in districts. Table A3.3 shows the changes in indicators by type of distric t and indicates that the contracting out model was the most effective of the three.

Table A3.3: Summary of Increases in Evaluation Indicators by Model Type (percentage)

Indicator Control

Districts Contracting In

Districts Contracting Out

Districts Antenatal Care 160.1 233.3 401.5

Delivery Attended by Trained Staff 26.0 — —

Delivery in a Health Facility n.a. 225.1 142.0

Tetanus Toxoid Immunization 149.1 148.6 400.0

Knowledge of Birth Spacing—All 307.4 317.4 599.5

Knowledge of Birth Spacing—Poor 271.0 301.4 559.5

Use of Birth Spacing 93.4 104.5 122.6

Expanded Immunization Program 55.7 81.8 158.1

Vitamin A Capsules —All -25.1 18.1 20.9

Vitamin A Capsules —Poor -24.1 29.9 23.9

Curative Care—Poor 81.7 490.5 1,096.0

Mean Increase in Key Indicators 99.6 179.7 320.4

Appendix 3 33

F. Health Care Costs, Resource Allocation, Cost-Efficiency, and Cost-Effectiveness

1. Project Costs and Expenditures

28. At appraisal the budget estimated for contracts under CHSPP: (i) $2.251 million for five CO districts; (ii) $0.700 million for five CI districts, a total $2.951 million. The planned budget supplement for five control districts was $600,000. During implementation, the actual contracts awarded to two CO districts total at $7.776 million. The budget supplement estimated for four control districts and three CI districts totaled at $1.041 million ($0.554 million for three CI districts and $0.488 million for four control districts). However, the contracts awarded were based on 1998 census per capita costs: (i) Memut at $5.60/capita/annum, amounting to $2,465,816; and (ii) Ang Rokar at $4.41/capita/annum amounting to $1,900,000. The total amount for CO was $4.366 million. The contracts for CI districts cover only costs specifically associated with the contractor and contractor related costs such as training: (i) Cheung Prey at $1.38/capita/annum, amounting to $927,319; (ii) Kirivong at $1.39/capital/annum amounting to $1,158,300; and (iii) Pereang at $1.84/capita/annum, amounting to $1,325,000. The total amount for three CI districts was $3. 399 million. The budget supplement of $0.25/capita/annum was in addition to the above costs. Although in theory the budget supplement in the CI districts was to support operating costs, in practice, the contractors used part of the budget supplements as incentives. The control districts received similar budget supplement of $0.25/capita/annum. The actual costs of the contracts awarded were significantly much higher than expected and there were no local NGOs or private sector groups.

2. Cost-Effectiveness

29. The ratio of cost-effectiveness for the 2.5-year period was computed by dividing the mean percentage of increase in key health coverage indicators by average total expenditure per capita (Table A3.4). During the period, the contracting out model was the most cost-effective of the three models, delivering a 30% increase in health service coverage for every dollar spent per capita. Contracting in, with a 26% increase in health service coverage, was less cost-effective than contracting out, but more cost-effective than the control districts.

Table A3.4: Cost-Effectiveness (percent)

Item Control

Districts Contracting In

Districts Contracting Out

Districts A. Average Percent Change in All

Coverage Indicators 99.60 179.70 320.40

B. Average Total Expenditure Per

Capita 4.65 6.73 10.82

C. Cost-Effectiveness Ratio (A/B) 0.21 0.26 0.30 Source: Table A3.3; Government, ADB, and NGO data. 3. Contribution to Poverty Reduction

30. The evaluation study found a significant increase in the use of public sector facilities matched by a decline in out-of-pocket costs, except in two contracting in districts. Private out-of-

34 Appendix 3

pocket expenditures decreased by 61% to 77% for the poorest 50% of households in contracting out districts. The decrease was 40% in one contracting in district, while an increase of 36% was apparent for the poorest 50% of households in two contracting in districts. In three control districts, private out-of-pocket expenditures decreased between 11% and 33%, but increased by 132% in one control district.

4. Recurrent Costs of Contracting

31. The evaluation study estimated the average annual total recurrent health expenditures per capita by source of funds in 2001 for contracted and control districts. The contracting in and contracting out districts spent less total (public plus private) resources per capita on health than the control districts. Even though the contracting in districts spent more than 1.5 times the amount of government and external funding agency funds than the control districts and the contracting out districts spent nearly 2.5 times as much, this was more than offset by the decrease in households’ out-of-pocket expenditures as shown in table A3.5.

Table A3.5: Average, Annual, Total Recurrent Health Expenditures per Capita by Source of Funds, 2001

($)a

Source of Funds Control Districts

Contracting In Districts

Contracting Out Districts

Private Out-of-Pocket $24.99 $23.56 $18.17

Government, External Funding Agencies, NGOs

$ 1.86 $ 2.82 $ 4.50

TOTAL $26.85 $26.38 $22.67 a Source: Average for private expenditure is from the 2001 household survey. Government, external funding agency,

and NGOs expenditures are estimated for 2001 based on expenditures for the first six months of 2001. 32. The highest government, external funding agency, and NGO cost, $4.50 per capita per year, is reasonable by international standards. Furthermore, because of the research nature of the CHSPP, expenditures were higher than might otherwise have been the case because of the scattered location of the districts, which prevented economies of scale. Future contracting could reduce costs by geographical concentration and a consequent increase in economies of scale. In addition, contracting out? already the most cost-effective model? could achieve further cost reductions by incorporating the user fee and staff performance contract system successfully demonstrated in one contracting in district. Thus satisfactory district health services could probably be provided for about $3.50 to $4.00 per capita per year for recurrent costs. 33. As the control districts had access to fewer funds than the contracting in and contracting out districts, the question arises as to whether they might have been able to match the latters’ performance (in terms of service coverage, if not cost-effectiveness) had more resources been available to them. In this connection, note that the control districts as a group expended only 36.1% of the budget supplement funds available to them. This raises some doubt as to whether, in the absence of external management assistance, the absorptive capacity to use additional resources effectively is present.

Appendix 3 35

G. Conclusion and Recommendations7

1. Conclusions

34. The CHSPP evaluation survey concluded that contracting health services is feasible, cost-effective, equitable, and benefited many rural poor people in particular. The contracting out model was preferable to the contracting in model.

35. The survey noted the following:

(i) An appropriately designed and managed user fees system could increase health care utilization by the poor and decrease their out-of-pocket expenditures on health care, as well as reduce expenditures by the government and external funding agencies. The user fees were established with community consultation and applied transparently. A ban on private practice by MOH staff was enforced, and incomes generated at health centers and referral hospitals were used to pay staff based on their performance and on objectively verifiable indicators.

(ii) The health centers with well-trained staff, including medical assistants,

significantly increased their utilization for curative care.

(iii) The use of international NGOs as health services contractors was an expedient solution for the CHSPP; however, this would not be feasible on a permanent nationwide basis.

(iv) The fixed price reimbursement basis of the contracts limited MOH’s managerial

control over the health services contractors. The purpose was to reduce the workload of the PCU accounting unit, delays, and cumbersome paperwork for the contractors.

(v) The prolonged vacancies in key positions was a serious concern for MOH and was caused by ineffective recruitment efforts, inadequate salaries, or intentional cost saving.

(vi) MOH-contractor relationships developed under the CHSPP have benefited

operational districts, but not PHDs.

(vii) The health service contractors encountered difficulties in procuring small amounts of basic medical equipment and drugs, which hampered health service delivery.

2. Recommendations

36. The evaluation survey recommended that the contracting of health services should be expanded in the follow-on ADB-assisted project. The priority areas are provinces with large populations and a high proportion of poor residents where the performance of health care services is below the national average. 37. The existing contracted districts should be reviewed based on PHDs’ capacities and resources and contractors’ performance. Adequate time should be allowed for any transitions, whether a return to government control or to a new contractor, to minimize service disruptions. 7 Based on “Achieving the Twin Objectives of Efficiency and equity: Contracting Health Services”, ADB, March 2002.

36 Appendix 3

Where a new contract award is required, a 6- to 12-month extension of the existing contract may be required to allow adequate time for the bidding process, award, and hand-over between health service contractors. 38. For future health service contracts, the survey recommends the following:

(i) Carrying out a careful appraisal of all NGOs in the health sector in the area prior to issuing bid documents and deciding on a case-by-case basis whether to either phase these out after the contractor assumes responsibility or to include them as subcontractors and design the contractor’s terms of reference to exclude specific responsibilities.

(ii) Ensuring that the PHDs’ role is clear and that mechanisms are in place to ensure

that they have a positive stake in the contractor’s success.

(iii) Having international NGOs partner with a counterpart local NGO with a gradual transfer of skills and responsibility to the local NGO during the contract period.

(iv) Instituting a user fee with staff performance contracts for using the proceeds and

enforcing a ban on private practice. An equity fund for hospitalization costs should be included as part of this scheme.

(v) Posting a medical assistant in each health center with specific responsibility for

consultation. While this will be more costly in terms of health center staff salaries, it is likely to generate considerable savings in hospitalization by ensuring prompter, earlier, and more accurate treatment at the primary level.

(vi) Working on a cost-plus-fee basis for personnel and on fixed prices for all other

items to permit both the necessary degree of managerial control and ease of payment.

(vii) Specifying a ceiling for recurrent costs. The experience of the pilot suggests that

$4 per capita would be a reasonable cap.

(viii) Including funds for the contractor to procure basic medical equipment, drugs, and medical supplies. This could involve a pre-approved mechanism for rapid purchase of drugs by contractors that includes a competitively awarded, indefinite quantity contract with one or more suppliers pre-qualified on the basis of price and quality.

Appendix 4 37

CAPACITY BUILDING, TRAINING, AND STAFF DEVELOPMENT

Number of Staff Trained Training Courses Planned Actual

Duration (Days)

Type of Staff Trained

Training of Trainers for Basic Management Training in the provinces

Not specified 20 14 Trainers from the Project

Basic Management Course for District Health Management Teams

Not specified 150 14 District health managers

Training of Trainers on the Use of the Health Center Integrated Supervisory Checklist

Not specified 36 3 Trainers

Use of the Integrated Supervisory Checklist for Health Center Supervision

Not specified 95 3 District health managers

Use of the Integrated Supervisory Checklist for Operational Dis trict Supervision

Not specified 30 2 Provincial health directors

Use of the Integrated Supervisory Checklist for Referral Hospital Supervision

Not specified 30 3 Provincial health directors

Minimum Package of Activities Module 1

Not specified 180 14 Health center staff

Medical Equipment Maintenance

Not specified 12 3 Provincial health department staff

Source: Ministry of Health.

38 Appendix 5

ESTIMATED AND ACTUAL PROJECT COSTS AND FINANCING PLAN ($ million)

Appraisal Estimate Actual

Item Foreign Exchange

Local Currency

Total Cost Foreign Exchange

Local Currency

Total Cost

A. Base Cost Civil Works 2.60 3.90 6.50 3.29 4.36 7.65 Land Acquisition 0.00 0.20 0.20 0.00 0.00 0.00 Furniture, Equipment, and Vehicles 2.40 0.30 2.70 0.68 0.05 0.73 Medical Supply and Drugs 3.50 0.90 4.40 1.27 0.00 1.27 Staff Development 0.20 0.30 0.50 0.16 0.19 0.35 Consulting Services 0.20 0.00 0.20 0.09 0.00 0.09 Surveys and Evaluation 0.20 0.30 0.50 7.60 1.87 9.47 Pilot Tests a 2.10 2.10 4.20 0.00 0.00 0.00 Project Implementation 0.60 0.30 0.90 0.42 0.66 1.08 Operation and Maintenance 0.00 0.70 0.70 0.00 0.02 0.02 Subtotal (A) 11.80 9.00 20.80 13.51 7.15 20.66 B. Contingencies Physical 1.20 0.80 2.00 0.00 0.00 0.00 Price 1.00 0.70 1.70 0.00 0.00 0.00 Subtotal (B) 3.20 1.50 3.70 0.00 0.00 0.36 C. Service Charge 0.40 0.00 0.40 0.36 0.00 0.36 Total Project Cost 14.40 10.50 24.90 13.87 7.15 21.02 Percentage of Total 57.60 42.40 100.00 66.00 34.00 100.00 Note: Figures may not total exactly because of rounding. a Expenditures combined with surveys and evaluation.

Appendix 6 39

DISBURSEMENT OF FUNDS, 1996–2003 ($ million)

Asian Development Bank Ministry of Health Year Quarter Amount Cumulative (A) Amount Cumulative (B)

Total (A) + (B)

1996 I 0.00 0.00 0.00 0.00 0.00

II 0.00 0.00 0.00 0.00 0.00 III 0.00 0.00 0.00 0.00 0.00 IV 0.50 0.50 0.00 0.00 0.50

1997 I 0.00 0.50 0.00 0.00 0.50 II 0.00 0.50 0.00 0.00 0.50 III 0.34 0.84 0.00 0.00 0.84 IV 0.27 1.11 0.00 0.00 1.11

1998 I 0.61 1.71 0.29 0.29 2.00 II 0.06 1.78 0.00 0.00 1.78 III 0.39 2.17 0.00 0.00 2.17 IV 2.03 4.20 0.00 0.00 4.20

1999 I 0.30 4.50 0.53 0.824 5.32 II 0.23 4.73 0.00 0.00 4.73 III 1.93 6.65 0.00 0.00 6.65 IV 1.28 7.93 0.00 0.00 7.93

2000 I 0.36 8.29 1.36 2.18 10.47 II 1.34 9.63 0.00 0.00 9.63 III 1.11 10.74 0.00 0.00 10.74 IV 1.59 12.33 0.00 0.00 12.33

2001 I 1.09 13.42 0.42 2.60 16.02 II 1.00 14.42 0.00 0.00 14.42 III 1.52 15.94 0.00 0.00 15.94 IV 1.14 17.08 0.00 0.00 17.08

2002 I 0.00 17.08 0.00 0.00 17.08 II 0.58 17.66 0.00 0.00 17.66 III 0.40 18.06 0.00 0.00 18.06 IV 0.25 18.31 0.00 0.00 18.31

2003 I 0.06 18.36 0.05 2.65 21.01

Total 18.36 2.65 21.01

PROJECT IMPLEMENTATION SCHEDULE, 1996–2003

Implementation Period (Years) Year 1 (1996)

Year 2 (1997)

Year 3 (1998)

Year 4 (1999)

Year 5 (2000)

Year 6 (2001)

Year 7 (2002)

Year 8 (2003) Description

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

I. Strengthening Community Health Services

A. Constructing and renovating Health Centers

B. Providing Equipment and Essential Drugs

C. Maintaining Selected Health Facilities

II. Strengthening District Health Offices

A. Improving Management Capability: Management and Budget Training District Office Renovation

B. Strengthening Supervision of Health Centers

Supervisory Skills Training

C. Strengthening Selected Referral Hospitals: Renovation of Referral Hospitals Equipment and Essential Drugs

III. Introducing Health Sector Reforms Through Pilot Testing

Pilot Test Design

40 A

ppendix 7

Implementation Period (Years) Year 1 (1996)

Year 2 (1997)

Year 3 (1998)

Year 4 (1999)

Year 5 (2000)

Year 6 (2001)

Year 7 (2002)

Year 8 (2003) Description

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Evaluation and Auditing

A. Contracting Out Health Services

B. Contracting In Management Services

C. Setting up a Community Loan Scheme for Health Emergencies D R O P P E D

IV. Providing Support for Central Office Management

A. Setting up a Project Coordination

Unit

B. Carrying Out Benefit Monitoring and Evaluation:

Benefit Monitoring and Evaluation Survey Health Facility Survey

C. Strengthening Equipment Maintenance and Repair: Fellowship for Equipment Maintenance D R O P P E D National Workshop for Equipment Maintenance

D. Equipment Maintenance

– Appraisal – Actual Source: Ministry of Health.

Appendix 7

41

42 Appendix 8

COMPLIANCE WITH MAJOR LOAN COVENANTS

Covenant Reference in Loan Agreement Status of Compliance

1.

The Borrower shall (i) maintain, or cause to be maintained separate accounts for the Project; (ii) have such accounts and related financial statements audited annually, in accordance with appropriate auditing standards consistently applied, by auditors acceptable to ADB; (iii) furnish to ADB, as soon as available but in any event not later than twelve (12) months after the end of each related fiscal year, certified copies of such audited accounts and financial statements and the report of the auditors relating thereto (including the auditors' opinion on the use of the Loan proceeds and compliance with the covenants of this Loan Agreement, as well as on the use of the procedures for imprest account and statement of expenditures), all in the English language; and (iv) furnish to ADB such other information concerning such accounts and financial statements and the audit thereof as ADB shall from time to time reasonably request.

Section 4.06 (b)

Complied with.

2. The Borrower shall enable ADB, upon ADB's request to discuss the Borrower's financial statements for the Project and its financial affairs related to the Project from time to time with the Borrower's auditors, and shall authorize and require any representative of such auditors to participate in any discussions requested by ADB, provided that any such discussion shall be conducted only in the presence of an authorized officer of the Borrower unless the Borrower shall otherwise agree.

Section 4.06 (c) Complied with.

3. Without limiting the generality of the foregoing, the Borrower shall furnish, or cause to be furnished, to ADB quarterly reports on the carrying out of the Project and on the operation and management of the Project facilities. Such reports shall be submitted in such form and in such detail and within such a period as ADB shall reasonably request, and shall indicate, problems encountered during the quarter under review, steps taken, or proposed

Section 4.07 (b) Complied with

Appendix 8 43


program of activities, and expected progress during the following quarter.

4. Promptly after physical completion of the Project, but in any event not later than six (6) months thereafter or such later date as may be agreed for this purpose between the Borrower and ADB, the Borrower shall prepare and furnish to ADB a report, in such form and in such detail as ADB shall reasonably request, on the execution and initial operation of the Project, including its cost, the performance by the Borrower of its obligations under the Loan Agreement and the accomplishment of the purposes of the Loan.

Section 4.07 (c) Complied with. Submitted on 9 July 2003.

5. Except as ADB may otherwise agree, the Borrower shall establish immediately after the effective date, an imprest account at the National Bank of Cambodia. The imprest account shall be established, managed, replenished, and liquidated in accordance with ADB's Guidelines on Imprest Fund and Statement of Expenditures Procedures dated November 1986, as amended from time to time, and detailed arrangements agreed upon between the Borrower and ADB. The initial amount to be deposited into the imprest account shall not exceed the equivalent of $500,000.

Schedule 3, para. 8 Complied with. Imprest accounts were established on 31 March 1997.

6. MOH shall be the Project's executing agency. Within one month of the effective date, the Borrower shall establish a Steering Committee comprising representatives from the Ministry of Economy and Finance, Ministry of Planning, and the Council for the Development of Cambodia, as well as senior staff of MOH. The Steering Committee shall meet as often as required, and shall be responsible for overseeing the Project and the Disease Control and Health Development Project of the World Bank. A Project Director acceptable to ADB, to be nominated by the Borrower's Minister of Health, and whose appointment shall be subject to ADB's approval, shall have overall responsibility for the implementation of the Project.

Schedule 6, para. 1 Complied with. The Project Steering Committee was formed on 31 March 1997.

44 Appendix 8


7. Within two months of the Effective Date, a

PCU shall be established within the central office of MOH. The PCU shall be staffed with adequate numbers of qualified personnel throughout the Project implementation, and shall be headed by a PCU manager whose appointment shall be subject to ADB's approval, and whose continuation in the position shall depend on performance acceptable to ADB. The PCU manager shall be responsible for the day-to-day operations of the PCU. The objectives of the PCU are to ensure timely, efficient, and effective implementation of the Project, and to build capacity within MOH in the areas of project administration, financial management and procurement. The functions of the PCU shall include (i) implementing policy decisions and guidelines of the Steering Committee, (ii) ensuring proper record-keeping in respect of Project activities, (iii) ensuring timely procurement of good and services under the Project, (iv) supervising Project monitoring and evaluation activities, (v) accounting for Project funds, (vi) tracking Project implementation and remedying any delays, and (vii) assisting in effective coordination with the projects of other donors.

Schedule 6, para. 2 Complied with. The PCU was established on 31 March 1997.

8. Within the PCU, there shall be an Administration and Finance Unit responsible for procurement, disbursement, and accounting for the Project. In addition, there shall be an ADB Section responsible for the implementation of the Project. The ADB Section shall be headed by an ADB Project Coordinator whose appointment shall be subject to ADB's approval, and whose appointment depend on performance acceptable to ADB.

Schedule 6, para. 3 Complied with. Administration and Finance Unit and ADB Unit were established and the ADB coordinator was appointed on 28 February 1997.

9. In each provincial health department in the Project area, a PPU shall be established within one month of the effective date. The functions of each such PPU shall include (i) facilitating Project implementation through collaboration with the PCU and the central office of MOH; (ii) ensuring the timely flow of funds for Project activities, especially in respect of small items and expenditures

Schedule 6, para. 4 Complied with. PPU was established on 31 March 1997.

Appendix 8 45


such as per diems and transportation costs; and (iii) day-to-day monitoring of the Project at the provincial level. Each PPU shall be headed by the Director of the Provincial Health Department of the PPU's respective province, and at least one staff member of each respective provincial project officer to work full-time on Project activities. Within each PPU, office accommodation shall be provided to the provincial construction supervisor(s) appointed under the Project.

10. Prior to the commencement of construction under the Project, MOH shall have prepared a list of all staff to be assigned to each health center in the Project area, other than health centers in those districts covered by the pilot programs to be carried out under Part C (i) and (ii) of the Project.

Schedule 6, para. 5 Complied with. Sheladia Associates obtained a list of staff for each proposed health center from provincial health departments during field assessments by 31 March 1997.

11. Within twelve months of the effective date, the Borrower shall establish standard accounting guidelines satisfactory to ADB for Project accounting and other funds received at the provincial, district, and health center levels in respect of health services, and for streamlining disbursement procedures in respect of such Project and other funds. Such guidelines shall be widely disseminated and shall be available in every health facility in the Project area.

Schedule 6, para. 6 Complied with. Accounting guidelines were established at the central level and petty cash procedures were established for provinces by 31 March 1997.

12. The PCU shall be responsible for the preparation of all reports to be submitted to ADB in respect of the Project.

Schedule 6, para. 7 Complied with.

13. Approximately three years after the effective date, a comprehensive midterm review shall be carried out by MOH in cooperation with ADB. The midterm review shall (i) review the scope, design, implementation arrangements, and other relevant aspects of the Project in light of the Borrower's development strategies, the policy framework in the health sector and the social concerns of ADB; (ii) examine progress made in achieving the objectives of the Project; (iii) identify changes in Project conditions in respect of sectoral issues and resource management and allocation, and

Schedule 6, para. 8 Complied with. Undertaken from 10–21 July 2000.

46 Appendix 8


assess their impact on Project implementation and sustainability; (iv) assess implementation performance against the established schedule; (v) review compliance with the assurances given in the Loan Agreement; (vi) identify problems and constraints in Project implementation and formulate recommendations for corrective action; and (vii) examine the preliminary results of the pilot programs conducted under the Project, and identify appropriate changes in MOH policy based on these results. Following the conclusion of the midterm review, MOH shall promptly take all measures necessary to remedy any Project weaknesses identified.

14. Without limiting the generality of Section 4.02 of this Loan Agreement, and except as ADB may otherwise agree, the Borrower and MOH shall take all necessary measures to acquire, prior to the commencement of construction under the Project, all land, rights-of-way, and other property rights required for the Project. The Borrower and MOH shall ensure that in selecting the sites for the health centers to be constructed under the Project consideration is given to the safety and privacy of the patients utilizing the facility, particularly women patients.

Schedule 6, para. 9

Complied with

15. The Borrower and MOH shall take all necessary measures to mitigate possible adverse social impacts associated with the Project. Such measures shall include ensuring that all lands required for the Project are unutilized and free from settlement, or, if utilized or settled, that a resettlement plan in accordance with the laws of the Borrower and ADB's policy on involuntary resettlement is formulated, and that adequate compensation is promptly made to property owners or users whose land is expropriated or whose use of the land is disrupted by the Project. Such measures shall also include ensuring that an effective grievance procedure, and dispute resolution mechanism acceptable to ADB is established for use by persons affected by Project activities.

Schedule 6, para. 10 Complied with. Land resettlement procedure was established by 31 March 1997. Policy on land acquisition was developed. No land resettlement was required for any construction under the Project.

Appendix 8 47


16.

MOH shall ensure that the location, design, and construction of Project facilities take into consideration the preservation of the environment and natural resources of the Borrower. Optimal use of local materials, natural light and ventilation, and refuse handling and disposal measures for the hazardous medical waste, shall be included in the design of the Project facilities. MOH shall submit to ADB for approval the designs of the health centers and hospitals to be constructed under the Project.

Schedule 6, para. 11

Complied with. On 31 December 1997, Sheladia Associates, the supervision and engineering firm, was provided with a copy of this covenant and incorporated these concerns in the construction plans. Medical wastes management procedures commensurate with the level of medical technology and curative care services were provided at referral hospitals.

17. The Borrower and MOH shall take all necessary measures to ensure full and effective implementation of the pilot program to be carried out under Part C (iii) of the Project. The Borrower and MOH shall ensure that local communities are directly involved in all aspects of the design and operation of such a pilot program.

Schedule 6, para. 12 Complied with. The pilot program was implemented under a monitoring system whereby visits and interviews were conducted in selected villages on a quarterly basis.

18. The PCU shall be responsible for implementing the benefit monitoring and evaluation program to be carried out in connection with the Project. The BME program shall be based on guidelines contained in ADB's Handbook on Benefit Monitoring and Evaluation.

Schedule 6, para. 13 Complied with. Performance in the field was monitored regularly. A database of health facilities and their construction and activity status is being maintained and regularly updated. Progress indicators of overall project performance are monitored annually. A separated monitoring system in place for the contracting pilot districts is compiled into a database in the ADB Unit.

19. To evaluate the benefits of the Project and asses the effectiveness of the pilot program carried out under the Project, a baseline household and health facility survey shall be carried out in the Project area during the first year of Project implementation. Follow-on surveys shall be conducted at completion of the Project, using the same questionnaires and methodologies as used for the baseline survey. Additionally, a smaller household

Schedule 6, para. 14 Complied with. The project baseline survey (the national health survey) was completed in 1998 and a final report was published in November 1999. The final survey was conducted in December 202.

48 Appendix 8


survey with a sample size of approximately 3,000 households shall be carried out prior to the midterm review to examine progress on certain of the Project output measures in the Project area.

20. Health facility surveys shall be conducted at the commencement and completion of the Project, and at the time of the midterm review of the Project, in order to obtain information on Project indicators related to quality of care, supervision, and availability of inputs in health centers and hospitals.

Schedule 6, para. 15 Complied with. A health facility survey was conducted during project preparation. The PCU conducted an in-house review of the construction already completed and prepared a report detailing health center construction during the second quarter of 2000.

21. The results of such surveys shall be used to compare the performance of the basic health care system in the Project area with that in other provinces of the Borrower.

Schedule 6, para. 16 Complied with. A final household and health facility surveys for both ADB and World Bank project provinces and at the national level were conducted in December 2002. The findings suggested that the project provinces outperformed other provinces in Cambodia, particularly in relation to the utilization of public health facilities, and maternal and child health services.

22. In respect of the pilot programs to be carried out under Part C (i) and (ii) of the Project, MOH and an external auditor shall carry out spot-checks during the contract period, and a comprehensive evaluation of the pilot programs shall be conducted prior to the midterm review and at Project completion.

Schedule 6, para. 17 Complied with. The contracting health services were fielded from March–July 1999 and September–November 1999 to oversee contractor performance during the crucial initial phases, and a permanent domestic consultant position (ADB Unit technical liaison officer) has been established in the PCU to conduct on-going monitoring of contractor

Appendix 8 49


performance. A detailed report by the contracting specialist that provides sufficient information was handed over to the midterm review mission. An evaluation survey on the CHSPP was conducted from 7 June–6 November 2001 as part of the preparation for the new Health Sector Support Project, jointly funded by ADB, the World Bank, and the Department for International Development.

23. The Borrower and MOH shall take all necessary measures, including all measures stipulated in the pilot program implementation plan agreed with ADB, as such plan may be amended from time to time with the concurrence of ADB, in order to permit full and effective implementation of the pilot programs to be carried out under Part C (i) and (ii) of the Project. Such measures shall include, but not be limited to, taking the necessary actions to (i) provide winning bidders a list of all public health workers in the Project area; (ii) provide health care system to the winning bidders; (iii) allow winning bidders to charge user fees in accordance with MOH's policy; and (iv) in respect of the pilot program to be carried out under Part C (i) of the Project, permit MOH personnel in the relevant districts of the Project area to take a leave of absence from MOH service for the period of the contract to accept employment with the winning bidders, and give the winning bidders management authority over such personnel. The Borrower shall cause MOH to perform all of its obligations under each of the contracts entered into between MOH and the successful bidders under the pilot programs carried out under Part C (i) and (ii) of the Project.


24. Within six months of completion of the Project, and in addition to the Project completion report submitted under Section 4.07 (c) of the Loan Agreement, the

Schedule 6, para. 20 Complied with. Evaluation was conducted from 7 June–6 November 2001, and the report was submitted to MOH

50 Appendix 8


Borrower shall prepare a comprehensive

report assessing the execution and operation of the pilot programs carried out under Part C (i) and (ii) of the Project. Such report shall be signed by MOH, and shall inform ADB of the Borrower's purposes of the Loan, and the Borrower's intentions in respect of continuing and extending the health care delivery systems developed under the pilot programs.

for consideration as inputs to the follow-on Health Sector Support Project.

25. Within 18 months of the effective date, MOH shall have developed detailed guidelines acceptable to ADB for the regular cleaning, maintenance, and repair of all MOH buildings in the Project area. Within one year of the effective date, in each provincial health department in the Project area, at least one person shall be working full time managing such building cleaning, maintenance, and repair in accordance with the guidelines developed by MOH. Each such maintenance manager shall be responsible for (i) carrying out regular visits to each health facility in the manager's respective province to inspect the condition of the building and carry out minor repairs; (ii) initiating and supervising required preventive maintenance such as painting; (iii) noting major repairs required, estimating the cost of such repairs, and initiating appropriate action to effect such repairs; and (iv) providing a written maintenance report to the relevant district health officer and the provincial department following each site visit.

Schedule 6, para. 20 Complied with. MOH established the Hospital Services Department to be responsible for maintenance. Provincial health departments and operational district staff have been trained to maintain health facility buildings.

26. Within two years of the effective date, MOH shall develop detailed guidelines acceptable to ADB for medical equipment maintenance and repair. The guidelines shall include standard reporting forms for maintenance managers and suggestions for supervisors, and the guidelines shall be provided to all health workers in the Project area. MOH shall take all necessary measures to ensure that such guidelines are implemented.

Schedule 6, para. 21 Complied with. Two contracts were awarded to an expatriate adviser and to the Cambodian Red Cross on 31 March 1999. MOH has developed policy and guidelines on maintenance and repair for medical equipment, and the Hospital Services Department has been assigned this responsibility.

Appendix 8 51


27.

All maintenance and repair activities undertaken in respect of MOH buildings and medical equipment in the Project area shall be summarized and reported to ADB in the quarterly Project reports submitted in accordance with Section 4.07 (b) of the Loan Agreement.


Complied with. The PCU has regularly submitted quarterly progress reports on a timely basis.

28.

The Borrower shall make no new investments in construction or equipment in respect of those district hospitals which will be converted to secondary health centers under the Project, other than investments in tuberculosis wards and ordinary repairs and maintenance. The Borrower shall ensure that external donor agencies are informed of this commitment in the event that outside assistance is proposed for such district hospitals. In respect of those districts in the Project area covered by the pilot programs to be carried out under Part C (i) and (ii) of the Project, the Borrower shall make no substantial additional investments in health services, and shall ensure that external donors are informed of this commitment in the event that outside assistance is proposed for such districts.


Complied with.

29. Except as ADB may otherwise agree, the location and number of hospitals and health centers to be constructed under the Project shall be in accordance with the Borrower's existing Health Coverage Plan.


30. The Borrower shall continue to increase its health services budget each year with the goal of having its public health expenditures equal 10% of the national budget by the year 2000. The Borrower shall annually provide ADB with a copy of its official budget, and with calculations showing the proportion of the budget allocated to MOH.


31. Without limiting the generality of Section 4.02 of this Loan Agreement, the Borrower shall take all necessary measures to ensure that the funds required, in addition to the


52 Appendix 8


. proceeds of the Loan, for the carrying out of the Project are made available at the times and in the amounts agreed with ADB. The Borrower shall annually provide ADB with a copy of (i) MOH's proposed budget, and (ii) the Borrower's official budget, showing, respectively, the amount of funds requested and allocated in respect of the Project.

32. MOH shall prepare and submit to ADB for approval prior to its implementation a program for the overseas fellowships to be provided under the Project. The program shall include details of the content and costs of the proposed training sessions, the location of such sessions, the criteria for selection of candidates, and the proposed candidates for the fellowships, including details of their qualifications.

Schedule 6, para. 27 Complied with. Overseas fellowship was undertaken in Manila from 13–24 October 1997.

ADB = Asian Development Bank, CHSPP = contracting health services pilot project, MOH = Ministry of Health, PCU = project coordination unit, PPU = provincial project unit.

Appendix 9 53

TA COMPLETION REPORT

Amount Approved: $500,000 Technical Assistance (TA) No. and Name TA 2567-CAM: Managing Basic Health Services Revised Amount: $499,000

Executing Agency: Ministry of Health (MOH)

Source of Funding: Japan Special Fund

TA Amount Undisbursed $2,230.60

TA Amount Utilized $496,769.40

Date Completion Date Original

Actual

Closing Date

Approval 8 May 1996

Signing 30 May 1996

Fielding of Consultants 1 August 1996

Original 28 February 1998

Actual April 2002

Description

MOH's Health Coverage Plan identified the facilities and services to be provided by operational districts as the focus for decentralized management of basic health services. District health management teams (DHMTs) were to be established in every operational district and supervised by the provincial health department. However, management training was lacking and the management of the health care delivery system was weak. The major issues were (i) insufficiently trained staff, (ii) lack of a coherent supervision system, (iii) inadequate physical infrastructure for the district health offices, and (iv) little knowledge or understanding of benefit monitoring and evaluation (BME). Thus improving decentralized management of the delivery of health care services at the provincial and district levels was given a high priority.

Objectives and Scope

The objective of the TA was to develop MOH’s institutional capacity to efficiently and effectively manage the basic health care delivery system by strengthening the managerial capabilities of provincial and district health management teams to supervise, administer, and support community health facilities developed under the ongoing Loan 1447-CAM(SF): Basic Health Services Project (BHSP). Specifically, the TA was to (i) provide DHMTs with relevant management skills; (ii) help establish supervisory systems for health centers and referral hospitals, and (iii) introduce the concepts and methods of BME.

Evaluation of Inputs

The TA was implemented by one international consultant (a health systems specialist) and one domestic consultant (a materials preparation specialist). The design and objectives of the TA were appropriate, and the terms of reference were adequate and were successfully implemented by the consultants. The consultancy inputs and the outputs of the TA facilitated implementation of the BHSP, in particular, in carrying out start-up activities, monitoring project activities and establishing BME. The TA was supervised by means of review missions. The consultants' overall performance was satisfactory.

Evaluation of Outputs

The TA helped MOH expedite loan effectiveness and establish good teamwork on the part of the BHSP's Project Coordination Unit (PCU). The TA helped strengthen DHMTs’ management and supervision capacity by providing training that emphasized planning, supervision, finance and budgeting, and essential drug management. Field assessment was conducted and DHMT district assistance plans were developed to ensure optimal use of physical infrastructure provided under the BHSP. Integrated supervisory checklists (ISCs) for supervision of health center and referral hospital staff by DHMTs and provincial health department staff were developed, field tested, approved and implemented nationwide. Three ISCs with users guides and training materials were developed in both English and Khmer. The TA also helped the PCU establish a monitoring system to assess the commissioning of health centers and referral hospitals, the delivery of services by health services contractors, and the effectiveness of supervision. Periodic unannounced spot visits were made to health centers and referral hospitals to verify the level of activity and whether supervision was taking place as reported. The results of these visits were entered into a continuously updated database. The supervisory systems were established in every operational district except in those participating in the contracting health services pilot project and those in which another donor was providing management support. DHMTs gained considerable assurance in their new roles, and were able to open and manage new health facilities effectively. All health centers constructed under the Project have been successfully opened and are operational. Supervisors felt that they had benefited from the training provided by the BHSP and that the

54 Appendix 9

supervisory systems were appropriate and useful; however, they encountered difficulties in addressing problems identified during supervision because of a lack of resources. Their capacity to motivate health center staff is limited by their low salaries. The TA also helped MOH design, manage, and analyze the BHSP's baseline survey and implement BME. The baseline survey provided MOH and the donor community with the first-ever population-based data on morbidity, mortality, and related health indicators in Cambodia. The TA was closed later than originally planned because of the BHSP’s the late start-up.

Overall Assessment and Rating The TA is considered successful and has fully met its objectives and scope. Systems developed under the TA have been implemented in the BHSP's area as well as nationwide. DMHTs with basic management training, use of ISCs and support for supervision have reached 100% achievement from a zero baseline. The sustainability of the monitoring systems is being ensured through the creation of a new loan-funded position within the PCU.

Major Lessons Learned To make supervision more effective, the role and capacity of the supervisory teams should be strengthened by increasing access to areas covered by operational districts and by providing adequate resources to increase team members’ salaries.

Recommendations and Follow-Up Actions Training curricula and materials developed for training DMHTs on planning, finance and budgeting, and essential drug management will be institutionalized and used for areas outside the BHSP area as well. MOH will review the curricula and materials for their relevance, effectiveness, and use in future externally-aided projects, including the potential ADB follow-up project, and update them as necessary. MOH has institutionalized the ISC developed under the TA. Operational policies and guidelines for supervision of health centers, referral hospitals, and operational districts using ISCs will continue to be reviewed and updated based on MOH's Health Coverage Plan. MOH should continue to support supervision and monitoring of the delivery of health care services, in particular, benefits and health impacts at the district and provincial levels. The skills developed for BME and the baseline values for mortality, morbidity, and health status will help MOH develop the master plan for the health sector and establish monitoring indicators. These skills and baseline values will also be useful for trend analysis of health status in the country and for monitoring the impact of various policy and project interventions.

Source: Report prepared by Wan Azmin, health specialist, Education and Health Division, Agriculture (West) Division.

Appendix 10 55

CONSULTING SERVICES

Appraisal Estimates Actual

Consulting Services No. of Persons

Total Person-Months

No. of Persons

Total Person-Months

A. International Consultants

1. Firms

Sheladia Associates, Inc. 0 0 1 42.0 Macro International (demographic health surveys) 0 0 1 6.0 Cambodian Red Cross, Medical Engineering Center 0 0 1 6.0 Subtotal 0 0 3 54.0

2. Individual/s Project Coordination Unit Manager and Chief of 1 15 1 8.0 Administration, Finance and Procurement Specialist 1 30 2 48.0 Contracting Specialist (CHSPP) 1 33 1 36.0 Medical Equipment Maintenance Planner 1 6 1 6.0 Civil Works Engineer 1 36 1 42.0 Health Facilities Architect 1 6 1 9.0

Subtotal 6 126 7 149.0

B. Domestic Consultant

1. Firm Foundation for Advice on Water and Agronomy

Consultants for Development 0 0 1 7.0

Subtotal 0 0 1 7.0

2. Individuals Jointly Hired by ADB and the World Bank Project Coordination Unit Manager 1 21 2 70.0 Deputy Project Director and Disbursement Specialist 0 0 1 72.0 Chief of Procurement 1 60 1 71.5 Procurement Assistant 0 0 2 79.0 Accounting Assistant 0 0 2 72.0 Subtotal 2 81 8 364.5

c. Individuals, ADB Unit Project Manager 1 60 1 74.0 Technical Liaison Officer 0 0 1 42.0 Health Management Monitoring Specialist 0 0 1 30.5 Accounting Assistant 0 0 2 72.5 Construction Supervisors 5 48 0 0.0 Subtotal 6 108 5 215.0 Total 14 315 24 789.5

ADB = Asian Development Bank, CHSPP = contracting health services pilot project. Source: Ministry of Health.

56 Appendix 11

ASSESSMENT OF OVERALL PROJECT PERFORMANCE

Criterion Weight Assessment Rating Value

Weighted Rating

(weight x assessment)

1. Relevance

20% Relevant

2

0.60

2. Efficacy 25% Inefficacious

0

0.50

3. Efficiency 20% Inefficient

0 0.40

4. Sustainability 20% Unlikely

0

0.40

5. Institutional Development and Other Impacts

15% Moderate

2

0.45

Overall Rating 2.35

BENEFIT MONITORING AND EVALUATION

Objectively Verifiable Indicator At Appraisal

Baseline (Based on the 1998 National Health Survey)

(%)

At Midterm Review

(Based on the 2000

Communicable Diseases

Health Survey)

(%)

At Project Completion

(Based on the 2002 National Health

Survey)

(%)

Remarks

A. Input and Process 1. Eighty percent of health centers are fully

operational, defined as (i) having an adequate supply of essential drugs (80% of drugs are in stock), (ii) having at least two health workers staffing the health center on a full time basis, and (iii) having been supervised eight times in the last year.

– 22.3 68.8 100.0 The BHSP constructed 251 new health centers and renovated 75 existing ones. Outpatient attendance in 2002 was about 20 to 40/day.

2. Eighty percent of referral hospitals with at least 60 usable beds are fully operational, defined as (i) having an adequate supply of essential drugs (80% of drugs are in stock); (ii) having at least 2 health workers staffing the health center on a full time basis; and (iii) have been supervised 8 times in the last years.

– 22.2 29.6 100.0 The BHSP upgraded 11 and renovated 2 referral hospitals. Basic medical equipment was provided to 19 referral hospitals.

3. Fifty percent of referral hospitals with surgical capacity to perform caesarean sections, defined as having trained staff, equipment, and facilities and having performed at least five in the last year.

– 18.5 25.9 50.0 Limited by lack of trained staff and of power and water supplies.

4. Functional operation district offices: • Percentage of district health

management teams with basic management training

• Percentage of district health management teams with supervisory checklist.

– 0 – 100.0 The BHSP upgraded 2 operational districts. Cambodia currently has 75 operational districts.

B. Process 1. Fifty percent of married women aged 15-45

years know at least three modern methods of family planning and a source of supply for each.

• Total fertility rate

–

–

32.4

4.1

42.2

3.1

23.9

3.6

In 2002, 26.2% of women in urban, 23.5% in rural areas, and 33.3% for daily pill. Use of modern method was 17% among poor women.

Appendix 12

57



(%)

At Midterm Review

(Based on the 2000


Health Survey)

(%)



Survey)

(%)

Remarks

2. Fifty percent of mothers know warning signs of

acute respiratory infection that should be seen by a health worker, and know how to prepare oral rehydration salts.

– 49.1 49.9 69.0

3. Fifty percent increase from baseline in the score health centers obtain on a standardized supervisory instrument related to the quality of patient care, availability of family planning services, outreach activities, cleanliness, and health center management.

– – – – Supervisory visits at health centers did capture information based on scores.

4. Health centers provide minimum package of activities in the project areas.

– 9.2 67.4

–

5. Fifty percent increase from baseline in the score referral hospitals obtain on a standardized supervisory instrument related to the quality of patient care and hospital management.

– – – – Supervisory checklist for the complementary package of activities was incomplete due to incomplete guidelines.

6. Eighty-five percent of diagnosed cases of tuberculosis receive complete short course therapy and become sputum negative.

– – – – Data on tuberculosis services were limited to the number of patients with tuberculosis and bed utiliization.

C. Output 1. Eighty percent of women of reproductive age

have received two doses of tetanus toxoid.

33.0a 15.2 33 38.8 Rates were better in urban areas than in rural areas.

2. Ninety-five percent of pregnant women have at least one antenatal visit performed by a professional health worker.

48.0b 43.2 45.5 70.7 Rates were better in urban areas than in rural areas.

3. Seventy percent of deliveries are attended by Ministry of Health staff.

40.0b 34 34.5 43.5 Rates in urban better than in rural areas.

4. Ninety percent of children aged 12–23 months are fully immunized prior to their first birthday.

50.0c 38.9 31.3 42.9 Rates were almost equal for urban and rural areas.

58 A

ppendix 12



(%)

At Midterm Review

(Based on the 2000


Health Survey)

(%)



Survey)

(%)

Remarks

5. Eighty percent of children 6 months to 5 years

old have received vitamin A supplements in the last 6 months.

– 58.1 28.5 37.1 In 2002, 76.1% of targeted children consumed foods rich in vitamin A.

6. Community use of Ministry of Health facilities for outpatient services (health center and hospitals) other than immunization process to 0.7 per inhabitant per year.

– – – –

7. Treatment of illness/injury by trained health worker:

• all socio-economic groups • poor (poorest 50%)

– –

61.9 51.4

– –

– –

In 2002, patients showed positive health seeking behavior: 98.3% of the population sought first treatment, 18.8% sought second treatment, and 5.1% sought third treatment.

8. Treatment of illness/injury in: • health center or outreach center • any hospital • district referral hospital

– – –

7.1

31.5 22.4

– – –

– – –

D. Impact 1. Infant mortality rate per 1,000 live births 115.0a 89.0 95.0 78.0 MDG target by 2015 is 42.

2. Under 5 mortality rate per 1,000 live births 181.0a 115.0 124.0 82.8 MDG target by 2015 is 59.

3. Maternal mortality rate per 100,000 live births 650.0b 473.0 437.0 368.0 MDG target by 2015is 150.

– = not available. a National estimates. No household survey data are available. b Ministry of Health. 1995. Knowledge, Attitudes, and Practice Survey on Fertility and Contraception in Cambodia. Phnom Penh. c Nationwide estimates from health management information system, MOH.

Appendix 12

59

Date post:	15-Jun-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

ASIAN DEVELOPMENT BANK PCR: CAM 27410€¦ · ASIAN DEVELOPMENT BANK PCR: CAM 27410 PROJECT...

Documents