www.3ieimpact.orgPhilip Davies

Rapid Evidence Assessments and Evidence Gap Maps

Philip DaviesInternational Initiative for Impact Evaluation [3ie]

BCURE Evidence-Informed Decision-Making Capacity Building Workshop1st and 2nd June 2015Pretoria, South Africa

www.3ieimpact.orgPhilip Davies

• Scaled down systematic reviews of existing evidence

• Timed to meet the needs of policy makers/practitioners (1-3 months)

• Strategically using the ‘three arms’ of systematic searching, but less exhaustively

Rapid Evidence Assessments – What Are They?

• Critical appraisal of identified studies is included

• Summary of findings, with caveats and qualifications

Photo © Panos East Africa

www.3ieimpact.orgPhilip Davies

Rapid Evidence Assessments – How Scaled Down?

www.3ieimpact.orgPhilip Davies

When to use a Rapid Evidence Assessment

• When a policy decision is required within months, based on the best available evidence within that time.

• When there is uncertainty about the amount and relevance of the available evidence

• When a map of evidence is required to establish the existing evidence, and to direct future research needs.

• When evidence of the likely effects of an intervention is required.

• When evidence of implementation barriers and facilitating factors/mechanisms is required

• When evidence on the cost and cost-benefits is required

www.3ieimpact.orgPhilip Davies

Rapid Evidence Assessments - Advantages

• Timed to meet the needs of decision makers, not academics/researchers

• A sounder evidence than selective literature reviews

• Better basis for making policy than being evidence-free

• Provides a challenge function to received wisdoms

• Challenges and strengthens a policy’s theory of change

• Provides more precise estimates of likely outcomes/effects

• Provides valid and reliable evidence on implementation

• Transparent strengths/weaknesses of evidence-base

www.3ieimpact.orgPhilip Davies

Rapid Evidence Assessments - Limitations

• Not as comprehensive as systematic reviews

• Not as data-detailed as a full systematic review

• May involve some selection bias and publication bias

• Can misrepresent the totality of evidence

• Can lead to Type I and Type II errors

• Need to be continued to produce full systematic reviews

• Need to use with caution

www.3ieimpact.orgPhilip Davies

3ie Evidence Gap Maps

• Maps of the existing evidence base on a policy issue, topic or sector such as maternal health, HIV/AIDS, agriculture, extreme poverty

• Structured around a framework of interventions and outcomes (intermediate and final)

• A ways of identifying where there is evidence, and where there is not

• An indication of the quality of this evidence

• Links to user-friendly summaries in the 3ie database of systematic reviews.

www.3ieimpact.orgPhilip Davies

Objectives of 3ie evidence gap maps

• To provide user-friendly tools for accessing and exploring existing evidence quickly and efficiently

• To facilitate informed judgement and evidence-based decision making

• To identify key “gaps” in the available evidence

• To indicate where future research should be focused

• To facilitate strategic use of scarce research funding

www.3ieimpact.orgPhilip Davies

Evidence gap map of interventions HIV/AIDS (Early Days)

Interventions / Outcomes Awareness, Knowledge Attitudes and Beliefs Risk Behaviour / Skills HIV Transmission

Take-up, Retention, Adherence Morbidity Mortality

Quality of Life, Well-being

Household Well-being

Labour Participation, Productivity Empowerment Stigma

Access, Service Quality

Behaviour Change Interventions

Peer Education Prevention for heterosexual men Condom use for HIV positive women Challenges in HIV prevention research Prevention in occupational settings Reducing stigma

Prevention for heterosexual men Peer interventions Peer education

Peer interventions Behaviour change interventions for women

Prevention in Latin America Couples-focused interventions Behaviour interventions for prevention

Prevention in heterosexual men Prevention in occupational settings

Behaviour interventions for prevention School based interventions for youth

Influence of social agents

Peer interventions

Prevention in Latin America

Prevention in occupational settings

School based interventions for youth

Condom Promotion, Distribution

Prevention for youth in Africa

Information, Education and Communication

Educating traditional healers

Parent-child communication in Africa

Educating traditional healers Prevention for youth in Africa Increasing youth's use of health

services Peer interventions for HIV positive women Educating traditional healers

Peer-based interventions for HIV positive women

Peer interventions for HIV positive women

School-based sexual health in Africa Prevention in African youth

Peer interventions for HIV positive women

Reducing stigma

Life skills education for youth Parent-child communication in Africa Girls' education ICT for youth

Peer-led sexual health education for youth Prevention for youth in Africa

Peer interventions for HIV positive women

ICT for youth Prevention in African youth

Life skills education for youth

Parent-child communication in Africa Peer-led sexual health education for

youth

Girls' education

Peer interventions for HIV positive women

ICT for youth

Mass Communication

Mass communication programmes Increasing youth's use of health

services

Mass media for young people

Community Mobilisation Behaviour interventions for prevention Reducing stigma

HIV Testing and Counselling

Counselling for testing of pregnant women Family planning Home-based VCT Family planning

Routine vs. voluntary testing VCT Routine vs. voluntary testing

Integrating PMTCT with other health services

Counselling for testing of pregnant

women Family planning

Counselling for testing of pregnant women

Routine vs. voluntary testing

Prevention of Mother to

Child Transmission

Counselling for testing of pregnant women Integrating PMTCT with other health services

Counselling for testing of pregnant women

Biomedical Interventions

Population based STI control

Challenges in HIV prevention research

Male Circumcision Male circumcision for prevention of

homosexual acquisition

Antiretroviral Therapy

Impact of treatment on risk behaviour Children's adherence to ART

Non-clinical outcomes of ART

Nurses for ART management

Nurses for ART management

Treatment and Care

Family support Palliative care

Patient adherence Palliative care Psychosocial well-being of HIV affected children

Task shifting in Africa

Family-centred treatment for HIV positive children Palliative care

Task shifting in Africa

Self-management interventions

Self-management

interventions

Family-centred treatment for HIV positive children

Health Systems

Integrated HIV and TB Service delivery

Integrating PMTCT with other health services

Task shifting in Africa Task shifting in Africa

Increasing youth's use of health services

Integrated HIV and TB Service delivery Integrated HIV and TB Service delivery

Key populations

Peer-education

Harm reduction for involuntary detainees

Peer-education

Harm reduction for involuntary detainees

Prevention interventions for female sex workers

Male circumcision for prevention of homosexual acquisition

Harm reduction for involuntary detainees Prevention interventions for female sex

workers

Behaviour interventions for sex workers Behaviour interventions for prevention

Harm reduction for involuntary

detainees

Behaviour interventions for sex workers

Interventions to Reduce HIV/AIDS Stigma

Reducing stigma Reducing stigma Reducing stigma

Reducing stigma

Structural Interventions

Economic Interventions

Economic interventions for prevention Challenges in HIV prevention research Economic interventions for prevention

Economic interventions for prevention

Inte

rven

tions

Outcomes

www.3ieimpact.orgPhilip Davies

Evidence gap map of interventions HIV/AIDS

www.3ieimpact.orgPhilip Davies

www.3ieimpact.orgPhilip Davies

Interactive Evidence Gap Maps(Latest Hi-Tech Interactive Platform)

www.3ieimpact.orgPhilip Davies

www.3ieimpact.orgPhilip Davies

www.3ieimpact.orgPhilip Davies

www.3ieimpact.orgPhilip Davies

• EGMs continue to be in high demand

• As a stand-alone product

• And as an integral part of SRs

• A new high-tech, interactive platform for presenting EGMs

Evidence Gap Maps• Agricultural Innovation

• Climate Change Adaptation

• Climate Change Mitigation

• Conservation

• Education (Primary And Secondary)

• Evidence For Peace Prograsms

• HIV/AIDS

• Integration Of HIV Services

• Maternal Health

• Productive Safety Net Programs

• REDD+ Initiatives

• Social Protection

• Water, Sanitation And Hygiene

www.3ieimpact.orgPhilip Davies

Thank youPhilip Davies

Email: pdavies@3ieimpact.org+44 (0)207 958 8350

Visit www.3ieimpact.org