Community Systems Strengthening Programs in Cambodia: Findings from a Recent Evaluation for the Global Fund

Soumya Alva, Sujata Ram, Anne Langston ICF International, May 15, 2012

Motivation for the Evaluation The role of community health workers and CSOs is

key Frameworks discuss ways in which community

systems contribute to improving health outcomes Indicators to measure the role of CSS are relatively

new A gap in measuring the success of CSS programs in

relation to health outcomes

HCMC (Representatives of Commune Council, Health center Team, and VHSG)

Population in Community

NGOsOD/PHD

Supervisor

OD Referral Hospital

HBC/ Linked

Response

VMWMMW

VHSG

Community Representati

ves

Health Center

CDOTS Watcher

VHV

CSS Activities in Cambodia

Evaluation Questions Program Design

Program content aligned with national priorities/plans Inputs and processes to ensure proper steps and policies are

incorporated in programs from the beginning

Program Implementation Facilitating and inhibiting factors Coordination, communication, alignment Sustainability?

Program Results Equity of services Synergies across HSS, HIV, TB, malaria GF grants Effect on diagnosis, referral, treatment, follow-up, linkages with other

aspects of health system

Evaluation Framework Global Health Community Systems Framework

Discusses how community systems contribute to improving health outcomes in conjunction with national health and social welfare systems

6 core components of community systems across 10 SDAs GF Monitoring and Evaluation Toolkit for Health and

CSS programs Based on the 10 SDAs applied to the CSS context Presents indicators for health inputs, outputs (by SDA),

health outcomes and impact

Linking to the GF CSS FrameworkSix Core Components of the framework

Service Delivery Areas (SDA) Evaluation Themes

1. Enabling environment and advocacy

SDA 1: Monitoring and documentation of community and government interventionsSDA 2: Advocacy, communication, and social mobilization

Community participation

2. Community networks, linkages, partnerships and coordination

SDA 3: Building community linkages, collaboration and coordination

Knowledge of community structures, roles and responsibilities, coordination

3. Resources and capacity building

SDA 4: Human resources: skills building for service delivery, advocacy and leadershipSDA 5: Financial resourcesSDA 6: Material resources—infrastructure, information, and essential commodities (including medical and other products and technologies)

Human resources, capacity, sustainability

4. Community activities and service delivery

SDA 7: Community-based activities and services—delivery, use, quality

Access, demand and utilization of services, quality of services

5. Organizational and leadership strengthening

SDA 8: Management, accountability, and leadership Adherence to work plan

6. Monitoring and evaluation and planning

SDA 9: Monitoring and evaluation, evidence buildingSDA 10: Strategic and operational planning

Standardization of guidelines and procedures, flow of information and data quality, feedback mechanisms

CSS Inputs•Enabling environment and advocacy•Community networks, linkages, partnerships and coordination•Resources and capacity building•Community activities and service delivery•Organizational and leadership strengthening•Monitoring and evaluation, and planning

CSS ProcessesRole of VHSG, HCMC, VMW, MMW, VHV, HBC, LR, C-DOTS watchers, and associated community programs•Community based health education•Referrals•Early Diagnosis and Treatment•Follow up care•Monitoring health programs•Conduit for community concerns•Monitor demand for health services

CSS Outputs•No./% of villages with VHSG , No, of. VHSG members trained• No. of HBC teams, No. of Health centers with HBC teams, No. of PLWHA supported by HBC teams, No. of PLWHA receiving care•No. of C-DOTS watchers trained, No. of health centers supporting C-DOTS, No. of TB patients receiving DOTS from C-DOTS watchers•No. of VMW, VHV trained, No. of fever cases presented/ treated by VMW

CSS Outcomes•Disease specific health outcomes (based on services received)• TB (case notification, treatment success)• Malaria (ACT)• HIV (ART)• MCH (ANC, SBA)•Stakeholder perceptions of• Access• Coverage • Equity• Quality of services

•Coordination with the health system•Research•Monitoring and evaluation

Overall Impact/Goal: Reduced Morbidity and Mortality

Cambodia Socio-Economic Context

Data Collection (6 ODs & MOH) Qualitative:

Key Informant Interviews National level: MOH, PR, selected SRs, intnl. organizations OD level: SSR/NGO, OD/Provincial staff

Focus Group Discussions at OD level Community members (including women) Community health workers – VSHG, HCMC Committee

chair, VMW, C-DOTS watcher, HBC Quantitative:

GF investments in health CSS outputs and health outcomes

Findings

Program Design: GF Disbursement

GF Expenditure by Round

Program Design Volunteers are the key to program success But, a top down approach

Little involvement of health center staff, community members, volunteers, and PLWHA in program design

Poor formal feedback mechanism Need to standardize guidelines for community participation in

health Difficulty enlisting volunteers and low incentives (TB C-DOTS

program: length of the treatment and the lack of incentives result in low motivation)

Need for training and regular supervision

“In case I don’t like any services in the health center, I just tell the village chief.”

—Commmunity Health Worker

“In general, the community people never have any idea of designing or making up the project so we have our own strategy and just tell them the policy and method for them to follow.”

—Provincial Malaria Supervisor

Program Implementation: Successes High community participation levels Overlapping of responsibilities

Synergy between the programs - cadre of volunteers trained can increase education and access to treatment for other conditions

Partnership among community entities, with collaboration between NGOs, the OD, the health department, and health centers

Roles and responsibilities of various community structures clearly understood

VHSG are key, mixed opinions re: Health Center Management Committee (HCMC)

“The referral of patients from health center to the hospital is done on time and without discrimination between the rich and the poor.”

—Community Member

“The HCMC are not really different from VHSG, they cooperate with each other to spread information as well as activities implementation. But we also have one more role; we are the coordinator between them to make a meeting at the health center. When there are some problems, we cooperate with the commune, village and health center to solve them.

—Community Health Worker

Program Implementation: Concerns Reasons for inability to follow approved plans

Disbursement of funds, low staff capacity, delays caused by partner organizations/in drugs and supplies, poor infrastructure, and problems with reporting due to poor data quality

Hard to reach people living outside of the village, namely monks, rural community members, and migrant workers

Community level data are weak Poor reporting and use for strategic decision-making Sustainability: volunteer activities would cease if

incentives were removed

“…the groups who we haven’t fully reached include those living in the remote areas and mobile populations whom our programs could not meet. The groups who we should focus more on are only those who are poor and live in remote areas, the elderly people, and the pregnant women as they are the most vulnerable population.”

—NGO Project Coordinator

Home Based Care: 32,000 PLWHA, 800 Health Centers Supported

2006 2007 2008 2009 20100

100

200

300

400

500

600

700

800

900

No. of HBC teams No. of Heath centres covered by HBC teams

HIV (Linked Response), TB (C-DOTS), Malaria (Village Malaria Workers)

HIV: PLWHA support group successful Referral and testing of pregnant women through Linked Response

program Linkages with TB not very clear

TB: Increase in no. of health centers implementing C-DOTS: pilot in

2001/2002 to 839 health centers nationally in 2010 No clear pattern between case notification rates and no. of health

centers offering C-DOTS across ODs

Malaria: VMW play a key role in diagnosis and treatment Responsible for more than 42.5% of all malaria treatments in 2011

Program Results: Access, Coverage, Quality

Improved service delivery Increase in no. of health centers and community-based services Volunteers referred patients to hospitals, more follow-ups HIV/AIDS diagnosis and treatment now easier - reduced time for referral

and no payment required for treatment Increase in number of outpatient visits in 2011 – possibly because of

role of VHSG Community perspective

Increase in demand for services and caseload at health center Community participation through volunteer activities improved

quality of services Perception in the community of improved health

Declines in the case fatality rate and number of malaria deaths

“I have seen that since the community health activities began, there are more people use the blood testing and treating for AIDS at health center because the health volunteers provide community people with health education every month, especially to the patients.”

—Home-based Care Volunteer

Limitations Short timeframe Hard to extrapolate to other ODs Community members may not be a representative sample but

are expected to provide information for program improvements

Hard to determine impact and efficiency of interventions Only associations between CSS interventions and outcomes can be

established