RESEARCH ARTICLE Open Access

A systematic review of multi-level stigmainterventions: state of the science andfuture directionsDeepa Rao1* , Ahmed Elshafei2, Minh Nguyen3, Mark L. Hatzenbuehler4, Sarah Frey2 and Vivian F. Go3

Abstract

Background: Researchers have long recognized that stigma is a global, multi-level phenomenon requiring interventionapproaches that target multiple levels including individual, interpersonal, community, and structural levels. While existinginterventions have produced modest reductions in stigma, their full reach and impact remain limited by anearly exclusive focus targeting only one level of analysis.

Methods: We conducted the first systematic review of original research on multi-level stigma-reduction interventions.We used the following eligibility criteria for inclusion: (1) peer-reviewed, (2) contained original research, (3) publishedprior to initiation of search on November 30, 2017, (4) evaluated interventions that operated on more than one level,and (5) examined stigma as an outcome. We stratified and analyzed articles by several domains, including whether theresearch was conducted in a low-, middle-, or high-income country.

Results: Twenty-four articles met the inclusion criteria. The articles included a range of countries (low, middle, and highincome), stigmatized conditions/populations (e.g., HIV, mental health, leprosy), intervention targets (e.g., people livingwith a stigmatized condition, health care workers, family, and community members), and stigma reduction strategies(e.g., contact, social marketing, counseling, faith, problem solving), with most using education-based approaches. Atotal of 12 (50%) articles examined community-level interventions alongside interpersonal and/or intrapersonal levels,but only 1 (4%) combined a structural-level intervention with another level. Of the 24 studies, only 6 (25%)were randomized controlled trials. While most studies (17 of 24) reported statistically significant declines inat least one measure of stigma, fewer than half reported measures of practical significance (i.e., effect size);those that were reported varied widely in magnitude and were typically in the small-to-moderate range.

Conclusions: While there has been progress over the past decade in the development and evaluation ofmulti-level stigma interventions, much work remains to strengthen and expand this approach. We highlightseveral opportunities for new research and program development.

Keywords: Stigma, Multi-level interventions, Low- and middle-income countries

BackgroundStigma can aggravate disease processes and add numer-ous socioeconomic, psychosocial, and health burdens onpeople who hold marginalized identities or statuses, in-cluding reduced educational attainment, exposure topsychosocial stressors, and challenges in accessinghealthcare [1]. Behavioral scientists have studied the

severe negative consequences of stigma for individualscoping with various health conditions and have learnedthat stigma can deter individuals from optimally en-gaging in treatment for their condition, which has ser-ious impacts on morbidity and mortality [2]. Strikingly,when disease morbidity and mortality are low but thecondition is highly stigmatized, the burden of stigmamay exceed the burden of the disease in its impact onsocial, emotional, and work functioning, thus negativelyaffecting the overall quality of life [3]. Researchers havelong recognized that stigma operates on intrapersonal,

* Correspondence: deeparao@uw.edu1Department of Global Health, Department of Psychiatry and BehavioralSciences, University of Washington, Campus Mailbox 357965, HarrisHydraulics Building, 1705 NE Pacific Street, Seattle, WA 98195-7175, USAFull list of author information is available at the end of the article

© The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Rao et al. BMC Medicine (2019) 17:41 https://doi.org/10.1186/s12916-018-1244-y

Collection on: Stigma Research and Global Health

http://crossmark.crossref.org/dialog/?doi=10.1186/s12916-018-1244-y&domain=pdfhttp://orcid.org/0000-0003-4076-0170mailto:deeparao@uw.eduhttp://creativecommons.org/licenses/by/4.0/http://creativecommons.org/publicdomain/zero/1.0/

interpersonal, organizational, and structural levels, andas such, stigma is conceptualized as an inherentlymulti-level phenomenon [1]. The multi-level nature ofstigma renders the development of stigma interventionsparticularly challenging, in part because addressing mul-tiple levels through research is more complex, requiresmore resources, and may be more burdensome to partic-ipants than single-level interventions. However, for re-search teams willing to take on the task of addressingmultiple levels, the impacts on stigma reduction effortscan be farther reaching, more synergistic, and more hol-istic than single-level interventions [4].Two previous papers have reviewed the literature

on stigma reduction interventions. In Heijnders andVan Der Meij’s 2006 review [5], consistent with themulti-level approach to stigma [1], the authors identi-fied five levels of examination and mapped strategiesand target populations directly onto each level. First,at the intrapersonal level, the focus of interventions ison characteristics of the individuals living with a stig-matized condition, and strategies involve self-help,counseling, and treatment. Second, at the interper-sonal level, the intervention is focused on the en-hancement of care and support in the stigmatizedpersons’ local environment. Third, at the communitylevel, the focus is on reducing stigmatizing attitudesand behaviors in (non-stigmatized) community groupsusing strategies such as education, contact, and advo-cacy. Heijnders and Van Der Meij define contact asany interactions between the public and the affectedperson for the purpose of reducing stigma [5]. Fourth,at the organizational/institutional level, interventionsfocus on reducing stigma in an organization or insti-tution, and strategies include training programs andinstitutional policies. Fifth, at the governmental/struc-tural level, interventions focus on establishing andenforcing legal, policy, and rights-based structures.In 2014, Cook and colleagues [6] conducted a narra-

tive review that similarly considered multiple levels inwhich stigma interventions can operate as part of anecological system [7], but focused on only three levels:intrapersonal, interpersonal, and structural. Cook et al.’sdefinitions of these levels differed slightly from those ofHeijnders and Van Der Meij’s and were more flexible, inthat one strategy, such as education, could operate onmultiple levels. The authors’ primary purpose was to de-scribe how each strategy operates on multiple levels,while targeting both stigmatized and non-stigmatizedpopulations.Although neither review was systematic, both chal-

lenged investigators to build and evaluate multi-levelstigma reduction interventions. In Heijnders and VanDer Meij’s review [5], while all of the strategiesreviewed had the potential to operate on multiple

levels, the authors reviewed studies that evaluatedstigma reduction strategies at a single level of analysis.In their conclusion, they called for researchers to com-bine multiple strategies to target multiple levels. Cookand colleagues [6] conducted an updated narrative re-view of stigma interventions and analyzed these studiesfor cascading impacts across multiple levels. The au-thors determined that studies examining cascading ef-fects across levels were rare, concluding that stigmareduction interventions that examine effects acrosslevels were urgently needed. While these two prior re-views pointed out important lacuna in the literature onstigma interventions, our study addresses anotherknowledge gap by conducting the first systematic re-view of multi-level stigma interventions. We describethe country of origin of research studies, depict the de-sign and participants of each multi-level stigma inter-vention, discuss the strategies and outcomes used bythese interventions, and highlight opportunities for newresearch and program development.

MethodsWe conducted this review in accordance with thePreferred Reporting Items for Systematic Reviews(PRISMA) guidelines [8]. We included studies that fo-cused on stigma reduction interventions operating onmultiple levels, both within and outside of the USA. Theoverall purpose of our review was descriptive, ratherthan evaluative. Thus, for each study, we provide a basicindication of effectiveness in reducing stigma, but a de-tailed evaluation of study effectiveness was beyond thescope of this review.We used Heijnders and Van Der Meij’s categories

for the levels of the ecological system (i.e., intraper-sonal, interpersonal, community, organizational/insti-tutional, governmental/structural) [5]. However, weexpanded our organization of these predefined strat-egies (e.g., education, contact) such that they couldmap onto multiple levels, which Heijnders and VanDer Meij did not do in their analysis. For example, ifone target of an intervention was to improve attitudesheld, whether by the stigmatized or the non-stigma-tized, we categorized this focus at the intrapersonallevel. If an intervention’s target was to improve inter-actions between people with stigmatized conditionsand other stakeholders (e.g., caregivers, healthcareworkers), we categorized this focus at the interper-sonal level. If the (non-stigmatized) public was tar-geted, we identified the community level as the focus.If an organization was targeted, we identified theorganizational/institutional level as the focus. If a pol-icy or administrative structure was targeted, we identi-fied governmental/structural level as the focus.

Rao et al. BMC Medicine (2019) 17:41 Page 2 of 11

Search termsFor our systematic review, we input search terms into sixelectronic database sources (PubMed, Embase, CINAHL,Global Health, Scopus, and PsychINFO). We looked forall papers with the term “stigma” plus at least one of thefollowing terms: “intervention,” “program,” “programme,”or “policy” in either the title or abstract. We used theCovidence database [9] to extract and organize informa-tion from articles. Because of our focus on health-relatedstigmas, we used primarily health-based databases in oursearch.

Inclusion criteriaWe used the following eligibility criteria for inclusion:(1) peer-reviewed, (2) contained original research, (3)published prior to initiation of search on November 30,2017, (4) evaluated interventions that operate on morethan one level, as defined above, and (5) examinedstigma as an outcome.

Exclusion criteriaWe excluded protocol papers, papers published in lan-guages other than English, abstracts without full textsavailable, non-peer reviewed articles, and solely qualita-tive studies.

Data extractionAfter identifying a list of all relevant records and remov-ing duplicates, 10,621 titles remained for title, abstract,and full-text screening. The abstract/title review andsubsequent full-text review of the selected studies wereconducted independently by two investigators (AE andMN), who had approximately 99% agreement, disagree-ing on only 39 of 10,621 articles. Discrepancies were re-solved over discussions with two additional investigators(DR and VG). The investigators retained 138 articlesafter abstract screening and 24 articles after full-textscreening based on the inclusion/exclusion criteriaabove. One hundred and fourteen articles were excludedduring full-text screening because we found the articlesmet exclusion criteria only after reviewing the full text.This process is depicted in Fig. 1.

Data analysisWe used content analysis [10] to organize the selectedqualifying studies. DR and SF independently coded eacharticle. We read through each article and systematicallycreated and collapsed categories. When SF and DR en-countered discrepancies, the codes were discussed andadjusted by consensus and the levels, as presented above.The themes identified from the articles included the fol-lowing: condition/population studied (e.g., HIV, mentalhealth, substance use, leprosy, diabetes, epilepsy, or-phaned and vulnerable children), intervention targets

(e.g., people living with a condition, health care workers,caregivers/family members, community members), levelof intervention targeted, country that served as the settingfor the study, and stigma reduction strategies used in theinterventions (e.g., education, contact, social marketing,counseling, faith, problem solving). We also coded articlesfor information on the intensity of the strategies used(e.g., duration, number of sessions) and whether thestudies used validated stigma measures. We examined ef-fectiveness using a simple, parsimonious approach, cat-egorizing findings in terms of statistical significance (atleast 1 measure of stigma used showed statistically signifi-cant reduction) or statistical non-significance (no statis-tical significance found or no inferential statistics used).We provided confidence intervals when given in the arti-cles, and effect sizes if given or if enough information wasgiven to calculate effect sizes in the articles.

ResultsOverall, six of the 24 studies were randomized controlledtrials (two used individual randomization [11, 12] and fourused cluster randomization [13–16]) (Table 1). Eighteenstudies did not randomize or use a control group and thuswere not considered randomized controlled trials (RCTs).Most studies reported on pilot trials of interventions. Ofthe studies reviewed that did use RCT designs, four usedcluster randomization. All studies used convenience sam-ples rather than population-based sampling.We found approximately equal numbers of studies ori-

ginating from low- and middle-income country (LMIC)and high-income country (HIC) settings, with 13 studiesconducted in HIC and 11 studies conducted in LMIC.Five studies were based in the US, three in the UK, twoin Canada, two in Indonesia, two in South Africa, andone study spanned five African countries (Lesotho,Malawi, South Africa, Swaziland, and Tanzania). Onestudy was conducted in each of the following countries:Kenya, Zambia, China, India, Vietnam, Israel, Haiti,Australia, and Japan.Twelve articles examined stigma associated with men-

tal illness, six HIV, two leprosy, one Moebius syndrome,and one each of epilepsy, orphans and vulnerable chil-dren, and substance use. Eighteen articles describedstudies targeting stigmatized participants, 12 includedparticipants who were community members (e.g., stu-dents, police), six articles included healthcare workers asparticipants, eight articles addressed stigma among care-givers/family members, and two articles examinedstigma among youth at risk for HIV. Of the articles tar-geting stigmatized populations, six studies targeted bothstigmatized and community populations, eight studiestargeted both stigmatized and caregiver populations, andsix studies targeted both stigmatized and healthcareworker populations.

Rao et al. BMC Medicine (2019) 17:41 Page 3 of 11

Five of the six articles examining HIV-related stigmaoriginated from LMICs. Conversely, articles examiningmental illness-related stigma predominantly came from aHIC (e.g., UK, US, Canada), with only one out of 12 arti-cles from a LMIC (India). Five studies were publishedprior to 2010, whereas 19 were published between 2010and 2017. Five of the six studies of HIV-related stigmawere published after 2010, and nine of 12 studies of men-tal health-related stigma were published in 2010 or after.The most common levels examined together were

intrapersonal and interpersonal, with 15 articles includ-ing these two levels together. Of the 24 articles, four ex-amined the community level alongside both theintrapersonal and interpersonal levels. Three articles ex-amined intrapersonal and community levels together,and another three examined interpersonal and commu-nity levels together. Thus, a total of 10 articles examinedcommunity levels alongside either interpersonal, intra-personal, or both levels. Only one article examined theinstitutional level (along with intrapersonal), and no arti-cles targeted the structural level. Figure 2 depicts thesefindings, separated by LMIC and HIC study location.The most common stigma reduction strategy studied

was education, with 16 studies using this strategy. Tenstudies examined contact, five counseling or copingskills acquisition, three social support, three drama, andtwo problem solving. Individual studies also examinedcommunication skills, voluntary counseling and testing,psychiatric treatment, and outdoor adventure as stigmareduction techniques. Of the 12 articles that examined

the community level alongside at least one other level,six used contact as a primary strategy. Eight studies usededucation and contact strategies together, and six ofthese eight studies originated from HIC.Seven of the 24 studies examined one stigma reduction

strategy across more than one level. For example, Patalayand colleagues [17] trained university medical students inthe UK to lead workshops with secondary school studentson mental health conditions and services (educationalstrategy). Investigators measured medical students’ levelsof stigma and impact on the interpersonal level to exam-ine the workshop leadership’s potential impact on medicalstudents’ future practice behaviors. The researchers alsomeasured attitudes towards mental illness of the recipientsof the intervention, (the secondary school students),thereby using the same educational strategy and thenassessing community-level stigma. Of note, one studyused an educational strategy with specialized areas of con-tent: Brown and colleagues [18] had nursing students de-liver a stigma reduction program in the US to communitymembers by providing information on mental health is-sues as well as on a faith-based framework for dealingwith mental health issues. In other words, this interven-tion provided education on a condition as well as educa-tion on using faith to cope with the condition (twoeducational strategies), without using faith-based counsel-ing techniques as the intervention itself.The majority (16 of 24) of studies reviewed used

stigma scales that were validated and used in multiplecountries. The remaining eight studies used scales that

Fig. 1 Flow of article inclusion and exclusion from review

Rao et al. BMC Medicine (2019) 17:41 Page 4 of 11

Table

1Multi-levelstig

mainterven

tions

Autho

rTitle

Cou

ntry

Con

ditio

nInterven

tion

targets

Strategy/in

tensity

Levels

Stigmameasures

Effectiven

ess

1.Batey,Whitfield,M

ulla,

Strin

ger,Durojaiye,

McCormick,Turan,Nyblade

,Kempf,Turan,2016[27]

AdaptationandIm

plem

entatio

nof

anInterven

tionto

Redu

ceHIV-Related

StigmaAmon

gHealth

care

WorkersintheUnited

States:Pilotingof

theFRESH

Worksho

p

US

HIV

17he

alth

care

workers(HCW)

19Peop

leliving

with

HIV

(PLW

H)

Education,contact,

coping

/cou

nseling;

1.5

days

Intrapersonal

Interpersonal

Multi-coun

tryValidated

MeasuresforHCW

and

PLWHat

pre-

andpo

st-

interven

tion

Non

-sign

ificant;

CIN

otGiven

;ES

Not

Given

2.Bh

ana,Mellins,Petersen

,Alicea,M

yeza,H

olst,A

bram

s,John

,Chh

agan,N

estadt,Leu,

McKay,2014[28]

TheVU

KAfamily

prog

ram:

Pilotin

gafamily-based

psycho

social

interventio

nto

prom

otehealth

and

mentalhealth

amon

gHIVinfected

early

adolescentsinSouthAfrica

SouthAfrica

HIV

65adolescents

andtheir

caregivers/fam

ily

Education,prob

lem

solving,

commun

ication;

10sessions

over

3mon

ths

Intrapersonal

Interpersonal

US-Validated

Measure

ofEpilepsyStigmafor

Ado

lescen

tsat

pre-,

2weeks

post-in

terven

tion,

3mon

thspo

st-in

terven

tion

Non

-sign

ificant;

CIn

otgiven;

ESno

tgiven

3.Bo

gart,H

emmesch

2016

[29]

Benefitsof

supp

ortconferencesfor

parentsof

andpeop

lewith

Moebius

synd

rome

US

Moe

bius

47Peop

lewith

(PW)M

oebius

and48

caregivers/fam

ily

Education,contact,social

supp

ort;3-dayconferen

ceIntrapersonal

Interpersonal

US-Validated

Measure

ofVisibleDifferen

cesat

pre-

andpo

st-conferen

ceattend

ance

Non

-sign

ificant;

CIn

otgiven;

ESd=0.5

and

r=−0.15

4.Brow

n2009

[18]

Faith

-based

men

talh

ealth

education:Aservice-learning

oppo

rtun

ityfornu

rsingstud

ents

US

Men

tal

illne

ss(M

I)55

nursing

stud

ents,38

commun

itymem

bers

Education;90-m

inworksho

pInterpersonal

Com

mun

ityMulti-Cou

ntry

Validated

Measure

ofMen

talIllness

Stigmaat

pre-

andpo

st-

worksho

p

Sign

ificant;

CIn

otgiven;

ESd=0.4

5.Chidraw

i,Greeff,Term

ane,

Doak2016

[30]

HIV

stigmaexpe

riences

and

stigmatisatio

nbefore

andafter

anintervention

SouthAfrica

HIV

18PLWH,60

caregivers/fam

ilyEducation,contact,

prob

lem

solving;

5-mon

thinterven

tionwith

worksho

psandgrou

pprojects

Intrapersonal,

Interpersonal

Multi-coun

tryValidated

HIV

StigmaMeasure

atpre-interven

tionand

quarterly

for1year

Sign

ificant;

CIn

otgiven;

ES d=0.11

(careg

ivers)

d=2.51

(PLW

H)

6.Dadun

,Van

Brakel,Peters,

Lusli,Zw

eekhorst,Bun

ders,

Irwanto

2017

[14]

Impact

ofsocio-econ

omic

developm

ent,contactandpe

ercoun

selling

onstigmaagainst

person

saffected

byleprosyin

Cirebo

n,Indo

nesia-a

rand

omized

controlledtrial

Indo

nesia

Leprosy

237PW

leprosy,

213and375

commun

ity

Con

tact

Socio-econ

omic

Cop

ing/coun

seling;

5coun

selingsessions;2

years

with

commun

ity

Intrapersonal

Interpersonal

Com

mun

ity

Multi-coun

tryValidated

LeprosyStigmaScaleat

pre-

andpo

st-in

terven

tion

Sign

ificant;

CIn

otgiven;

ES d=1.57

7.Haw

ke,M

ichalak,Maxwell,

Parikh2014

[31]

Redu

cing

stigmatowardpe

ople

with

bipo

lardisorder:impact

ofa

filmed

theatricalinterven

tionbased

onape

rson

alnarrative

Canada

MI

48PW

MIand

caregivers/fam

ily,

29commun

ity,

60HCW

Educationcontact(film

ed);

50min

Intrapersonal

Interpersonal

Com

mun

ity

Multi-coun

tryValidated

Stigma/SocialDistance

ofMIScalesat

pre-,p

ost-

and1-mon

thpo

st-

interven

tion

Sign

ificant;

CIn

otgiven;

ESEta

squared0.32

8.Hen

derson

,Corker,Lewis-

Holmes,H

amilton

,Flach,

Rose,W

illiams,Pinfold,

Thornicroft2012

[32]

England’sTimeto

Chang

eantistig

macampaign:One

-year

outcom

esof

serviceuser–rated

expe

riences

ofdiscrim

ination

UK

MI

1584

commun

itySocialmarketin

g;12

mon

ths

Interpersonal

Com

mun

ityMulti-coun

tryValidated

Stigmaof

MIScale

Sign

ificant;

CIn

otgiven;

ESno

tgiven

Rao et al. BMC Medicine (2019) 17:41 Page 5 of 11

Table

1Multi-levelstig

mainterven

tions

(Con

tinued)

Autho

rTitle

Cou

ntry

Con

ditio

nInterven

tion

targets

Strategy/in

tensity

Levels

Stigmameasures

Effectiven

ess

9.Jürgen

sen,Sand

oy,

Miche

lo,Fylkesnes

2013

[13]

Effectsof

home-basedVo

luntary

Cou

nsellingandTestingon

HIV-

relatedstigma:Find

ings

from

acluster-rand

omized

trialin

Zambia

Zambia

HIV

1694

commun

ityVo

luntarycoun

selingand

testingover

2mon

ths

Interpersonal

Com

mun

ityMulti-coun

tryValidated

HIV

StigmaMeasure

atpre-

and6mon

thspo

st-

interven

tion

Sign

ificant;

CI(−1.08,−

0.054);ES

R2=0.02

10.Li,Wu,Liang,

Lin,Zh

ang,

Guo

,Rou

,Li2013[15]

Aninterventio

ntargetingservice

providersandclientsfor

methado

nemaintenance

treatm

entinCh

ina:Acluster-

rand

omized

trial

China

Substance

abuse

41HCW,179

peop

leusing

heroin

onmethado

ne

Education(3

grou

psessions),motivational

interviewing(2

sessions)

Intrapersonal

Institu

tional

Chine

seValidated

PerceivedStigmaof

Add

ictio

nsCareat

pre-,

3,6,9mon

thspo

stinterven

tion.

Non

-sign

ificant;

CI3

mon

ths

(−3.36,3.01);

6mon

ths(−

6.96,1.21);

9mon

ths(−

6.39,2.66);

ESNot

Given

11.Lusli,Peters,van

Brakel,

Zweekhorst,Iancu,Bun

ders,

Irwanto,Reg

eer2016

[16]

Theim

pact

ofarig

hts-based

coun

selinginterven

tionto

redu

cestigmain

peop

leaffected

byleprosyin

Indo

nesia

Indo

nesia

Leprosy

124PW

leprosy

Cop

ing/coun

seling,

5sessions

Intrapersonal

Interpersonal

Adapted

from

aMulti-

Cou

ntry

Validated

Measure

ofHIVStigmaat

pre-

andpo

st-in

terven

tion

Sign

ificant;

CIn

otgiven;

ESno

tgiven

12.M

aulik,Kallakuri,

Devarapalli,Vadlam

ani,Jha,

Patel2017[33]

Increasing

useof

men

talh

ealth

services

inremoteareasusing

mob

iletechno

logy:A

pre-

post

evaluatio

nof

theSM

ART

Men

tal

Health

projectin

ruralInd

ia

India

MI

238commun

ity,

23HCW

Education,dram

a,psychiatric

treatm

ent,8

weeks

incommun

ity;3

mon

thswith

HCW

Intrapersonal

Interpersonal

Com

mun

ity

Multi-Cou

ntry

Validated

CareAccessandStigma

Scaleat

pre-

andpo

st-

interven

tion

Sign

ificant;

CIn

otgiven;

ESno

tgiven

13.M

ichaels,Corrig

an,

Buchho

lz,Brown,Arthu

r,Netter,MacDon

ald-Wilson

2014

[11]

Chang

ingstigmathroug

ha

consum

er-based

stigma

redu

ctionprog

ram

US

MI

127PW

MI,131

HCW

Education,contact,4

worksho

pseach

2–3hin

duratio

n

Intrapersonal

Interpersonal

Multi-Cou

ntry

Validated

ScaleforMIStig

maat

pre-

andpo

st-in

terven

tion

Sign

ificant;

CIn

otgiven;

ESEta

squared0.08

14.M

ichalak,Living

ston

,Maxwell,Hole,Haw

ke,

Parikh2014

[34]

Using

theatreto

addressmen

tal

illne

ssstigma:Aknow

ledg

etranslationstud

yin

bipo

lar

disorder

Canada

MI

80PW

MI,84

HCW

Con

tact,d

rama;50-m

inpe

rform

ance

and30-m

inqu

estio

nandansw

ersession

Intrapersonal

Com

mun

ityMulti-Cou

ntry

Validated

ScaleforMIStig

maat

pre-,

post-,and3–4mon

ths

afterinterven

tion

Sign

ificant;

CIn

otgiven;

ESEta

squared0.11

15.N

goc,Weiss,Trung

2016

[12]

Effectsof

thefamily

schizoph

reniapsycho

education

prog

ram

forindividu

alswith

recent

onsetschizoph

reniain

Viet

Nam

Vietnam

MI

59PW

MIand

caregivers/fam

ilyEducation;3,1.5-h

sessions

Intrapersonal

Interpersonal

Adapted

Validated

Scale

forMIStig

maat

pre-

and

6mon

thspo

st-intervention

Sign

ificant;

CIn

otgiven;

ESEta

squared0.13

and0.18

16.O

rkibi2014[35]

Theeffect

ofdram

a-basedgrou

ptherapyon

aspe

ctsof

men

tal

illne

ssstigma

Israel

MI

5PW

MI,7

commun

ityDrama(the

rapy);20

weekly,2-hsessions

Intrapersonal

Com

mun

ityMulti-Cou

ntry

Validated

MI

Stigmascales

ateach

of14

session2

Sign

ificant;

CIn

otgiven;

ESno

tgiven

17.Patalay,A

nnis,Sharpe,

New

man,M

ain,Ragu

nathan,

Parkes,C

larke2017

[17]

APre-Po

stEvaluatio

nof

Ope

nMinds:a

Sustainable,Peer-

LedMen

talH

ealth

Literacy

Prog

rammein

Universities

and

Second

aryScho

ols

UK

MI

234commun

ity,

40HCW

Education;2worksho

psover

3weeks

Interpersonal

Com

mun

itySocialDistanceMeasure

used

intheUKat

pre-and

post-in

terven

tion

Sign

ificant;

CIn

otgiven;

ESod

dsratio

:0.85

Rao et al. BMC Medicine (2019) 17:41 Page 6 of 11

Table

1Multi-levelstig

mainterven

tions

(Con

tinued)

Autho

rTitle

Cou

ntry

Con

ditio

nInterven

tion

targets

Strategy/in

tensity

Levels

Stigmameasures

Effectiven

ess

18.Pinfold,Tho

rnicroft,

Huxley,Farm

er2005

[36]

Activeingred

ientsin

anti-stigma

prog

ramsin

men

talh

ealth

UK

MI

PWMI,

commun

ity(109

police,78

adults,

472scho

olstud

ents)

Education,social

marketin

g,contact;2,2-h

sessions;2,50-min

scho

olsessions;

Intrapersonal

Interpersonal

Com

mun

ity

Multi-Cou

ntry

Validated

socialdistance

scales

atpre-

andpo

st-in

terven

tion

Sign

ificant;

CIn

otgiven;

ESno

tgiven

19.Smith

Fawzi,Eustache,

Osw

ald,

Louis,Surkan,

Scanlan,Hoo

k,Mancuso,

Mukhe

rjee2012

[37]

Psycho

socialsupp

ortintervention

forH

IV-affected

families

inHaiti:

implications

forp

rogram

sand

policiesforo

rphans

and

vulnerablechildren

Haiti

HIV

168PLWH,130

caregivers/fam

ilySocialsupp

ort,coping

/coun

selingof

14and15

caregiversupp

ortgrou

psessions

with

andwith

out

youth.

Intrapersonal

Interpersonal

Multi-Cou

ntry

Validated

HIV

StigmaScaleat

ore-

andpo

st-in

terven

tion

assessmen

t

Sign

ificant;

CIn

otgiven

forStigma

Scale;

ESno

tgiven

20.Sne

ad,A

ckerson,Bailey,

Schm

itt,M

adan-Swain,

Martin

2004

[38]

Taking

charge

ofep

ilepsy:the

developm

entof

astructured

psycho

educationalg

roup

interven

tionforadolescentswith

epilepsyandtheirparents

US

Epilepsy

7PW

epilepsy

andcaregivers/

family

Education;6-weekgrou

pinterven

tion

Intrapersonal

Interpersonal

Multi-Cou

ntry

Validated

ScaleforEpilepsyStigma

atpre-

andpo

st-

interven

tion

Non

-sign

ificant;

CIn

otgiven;

ESno

tgiven

21.Stuhlmiller

2003

[39]

Breaking

downthestigmaof

men

talillnessthroug

han

adventurecamp:

Acollabo

rative

educationinitiative

Australia

MI

100PW

MI,200

commun

ityEducation,ou

tdoo

radventureover

2days

Intrapersonal

Interpersonal

Unp

ublishe

dScaleof

MI

Stigmaat

pre-

andpo

st-

interven

tion

Not

sign

ificant

(no

inferential

statistics);

CIn

otgiven;

ESno

tgiven

22.Thu

rman,Jarabi,Rice

2012

[40]

Caringforthe

caregiver:Evaluation

ofsupp

ortg

roup

sforg

uardians

oforph

ansandvulnerablechildren

inKenya

Kenya

Orphans/

vulnerable

children

766caregivers/

family

and1028

orph

ans/

vulnerable

children

Socialsupp

ort;Supp

ort

grou

psprovided

inthe

commun

ity

Intrapersonal

Interpersonal

Careg

iver

marginalization

scaleused

inRw

anda,

ValidationInform

ation

Not

Given

Sign

ificant;

CIn

otgiven;

ES standardized

beta

−0.22

23.U

ys,C

hirw

a,Ko

hi,G

reeff,

Naido

o,Makoae,Dlamini,

Durrheim,C

uca,Holzemer

2009

[41]

Evaluatio

nof

ahe

alth

setting-

basedstigmainterven

tionin

five

Africancoun

tries

5African

coun

tries

HIV

41PLWH,177

HCW

Con

tact,cop

ing/

coun

seling,

education

ina2-dayworksho

p

Intrapersonal

Com

mun

ityMulti-Cou

ntry

HIV

Stigma

ScaleforHCW

at3mon

ths

pre-

andwith

in1mon

thpo

st-in

terven

tion.

Sign

ificant;

CIn

otgiven;

ESno

tgiven

24.Yotsumoto,Hiro

se,

Hashimoto2010

[42]

Anaw

aren

essprog

ram:the

sign

ificanceof

lectures

delivered

byindividu

alswith

men

tal

disabilities

Japan

MI

12PW

MI,844

commun

ityCon

tact,edu

catio

n;2–5

lectures

perpe

rson

Intrapersonal

Com

mun

ityMulti-Cou

ntry

Validated

StigmaScaleat

post-

interven

tion

Non

-sign

ificant;

CIn

otgiven;

ESno

tgiven

Notes:W

ecatego

rized

finding

sin

term

sof

statistical

sign

ificance(atleast1measure

ofstigmaused

show

edstatistically

sign

ificant

redu

ctionat

p<0.05

)an

dno

n-sign

ificance(nostatistical

sign

ificancefoun

dor

noinferentialstatisticsused

).Wecalculated

effect

size

(ES)

whe

nen

ough

inform

ationwas

prov

ided

tocalculateCoh

en’sdor

Etasqua

red.

Unstand

ardizedtest

statisticswerelabe

ledas

‘ESNot

Given

’Abb

reviations:C

Icon

fiden

ceinterval,ESeffect

size,H

CWhe

alth

care

workers,H

IChigh

-incomecoun

try,HIV

human

immun

odeficiencyvirus,LM

IClow-an

dmiddle-incomecoun

try,MIm

entalilln

ess,PLWHpe

ople

living

with

HIV,P

RISM

APreferredRe

portingIte

msforSystem

aticRe

view

s,PW

peop

lewith

,RCT

rand

omized

controlledtrial,USUnitedStates

Rao et al. BMC Medicine (2019) 17:41 Page 7 of 11

were adapted from validated scales, used in a neighbor-ing country, or validated by the study team for use inthe country where the study had taken place. Beyonduse of validated and adapted measures, the articles pro-vided little information on how well the instrumentsperformed across cultures and contexts.In terms of effectiveness, 17 studies reported that their

intervention reduced stigma scores (p < 0.05) on at leastone measure of stigma and seven studies reportednon-significant results. Of these seven studies that foundnon-significance, five were conducted in high-incomecountries and two were conducted in middle-incomecountries (China and South Africa). Only two of the 24articles provided information on confidence intervals. Interms of practical significance, only 11 out of the 24studies provided information to calculate effect sizes orthe effect sizes themselves. Cohen’s d values that werereported ranged from 0.4 to 2.51, Eta squared andR-squared values ranged from 0.02 to 0.32, indicatingsmall-to-moderate effects across studies.

DiscussionWe set out to review intervention studies thattargeted multiple levels of stigma reduction and

identified 24 studies. Notably, the majority of studiesidentified and reviewed were published after 2010,demonstrating an increasing urgency and movementin the research community towards developing andvalidating stigma reduction interventions. Articlesthat originated from HICs tended to examine mentalillness-related stigma, whereas those from LMICstended toward the examination of HIV-related stigma.This may be due to availability of funds, as globalhealth spending in LMICs has decreased over time ex-cept for HIV-related work [19].Most investigators used validated or adapted mea-

sures of stigma in their studies, but provided little in-formation on how well the measures performed indiverse settings. Contextual psychometric informationand sensitivity/specificity of measures are useful piecesof information to determine accurate interpretation ofintervention effectiveness. This is particularly relevantfor studies that used adapted measures or measures val-idated in languages or contexts that differed fromwhere the studies were conducted. More detailed exam-ination of measures used to evaluate intervention ef-fectiveness will be an important direction for futureresearch on multi-level interventions.

Fig. 2 Levels examined together, separated by high- and low/middle-income country

Rao et al. BMC Medicine (2019) 17:41 Page 8 of 11

Similarly, we found relatively few studies that used ran-domized controlled trial (RCT) designs. The lack of RCTdesigns may be due to the challenges of conducting RCTsacross multiple levels. Investigators in future studies ofmulti-level stigma interventions may consider use ofnon-traditional hybrid trial designs, quasi-experimentaldesigns, or other types of pragmatic designs used in com-plex real-world settings. Similarly, we also noted that justunder half of the reviewed articles provided effect sizes,and those that were reported varied widely in magnitude.Adding rigor to these designs may help to narrow infor-mation on the potential benefits of interventions that op-erate on multiple levels.The intrapersonal and interpersonal levels were most

often targeted by the multi-level stigma interventionsstudied, which may be due to several factors. The broaderstigma literature has focused almost exclusively on thesetwo levels of analysis [1]; thus, multi-level interventionshave a larger evidence base from which to draw at theselevels. Relatedly, research has accumulated a wealth ofstigma measures at the individual/interpersonal levels ofanalysis. In contrast, until recently, fewer measures ofstigma existed at community, organizational, and struc-tural levels, which likely hindered the assessment ofmulti-level stigma interventions that incorporatedcommunities and social structures [20]. This focus on theindividual/interpersonal levels in multi-level stigma inter-ventions may also be due to convenience—interventionstudies are often easier to implement in clinical settingswhere people with health-related stigmatizing conditionsseek care and where their family members (who areneeded for research at the interpersonal level) are moreeasily identified and assessed. More research is needed toincorporate community-, organizational-, and structural-level influences into multi-level stigma interventions.Approximately half the studies reviewed examined

community-level stigma reduction, with intrapersonaland/or interpersonal levels. Studies that targeted com-munity levels of stigma predominantly used methods ofinteraction, or contact, across populations studied,examining the impact of exchanging information andmaking use of bi-directional learning and includingpeople living with stigmatized conditions in the process(e.g., teaching, drama). In addition, these studies tendedto incorporate exchanges of support, particularly whenfamily members and health care workers were involved.Despite accumulating research indicating that struc-

tural forms of stigma contribute to adverse health out-comes among members of stigmatized groups [21, 22],only one study combined an institutional-level approach,and no studies combined the structural-level approach,alongside another level. Researchers may consider insti-tutional- and structural-level interventions challenging,since they require time and financial resources to

examine stigma in large samples. Despite these chal-lenges, single-level studies are beginning to emerge thatexamine stigma reduction as a result of policy changesat the structural level [23]. Thus, one important direc-tion for future development of multi-level interventionsis greater attention to, and incorporation of, policy-levelinterventions to address stigma at the institutional andstructural levels.With respect to stigma-reduction strategies used by

these multi-level stigma interventions, most focused oneducation, either alone or in combination with other strat-egies, such as contact. Corrigan and colleagues found overyears of research that stand-alone educational programscan lead to stereotype suppression, in which members ofthe public suppress—rather than reject—stereotyped be-liefs upon learning that such beliefs are socially undesir-able [24, 25]. Thus, educational programs alone are oftenineffective in reducing stigmatizing attitudes in membersof the public, and the little resulting stigma reduction thatoccurs may be short-lived and superficial [26]. Future re-search on multi-level stigma interventions is thereforeneeded to explore a wider range of stigma-reduction strat-egies and to utilize evidence-based strategies that prior re-search has shown to be effective in reducing stigma.This review has several limitations. First, although we

introduced independent secondary article reviewers andcoders, our process of article selection, non-inclusion ofgray literature, inclusion of studies reported in Englishonly, and content analysis may have introduced selectionbiases that restrict the generalizability of our findings toall multi-level stigma interventions. Second, the scope ofour study did not include detailed comments regardinga methodological appraisal of studies and we includedlimited information on intervention effectiveness. Thelack of rigor in these studies may have led to samplingbias and non-generalizable conclusions. Thus, additionalresearch will need to be done before recommendationson effectiveness can be made.

ConclusionsStigma is inherently a cross-sectoral phenomenon [1]and thus efforts to reduce stigma and its pernicious ef-fects require a multi-level approach. Despite progressover the past decade in the development of multi-levelstigma interventions, much work remains to strengthenand broaden this approach. In Table 2, we highlight sev-eral opportunities for new research and program devel-opment in multi-level stigma interventions, organizedaround several key domains (e.g., measurement, mecha-nisms of change, implementation). This list is not ex-haustive, but rather is meant to underscore some of themost important areas of inquiry that are needed to ad-vance the knowledge base in this incipient field. For in-stance, multi-level stigma interventions may not always

Rao et al. BMC Medicine (2019) 17:41 Page 9 of 11

be appropriate; future research is therefore needed tosystematically compare the efficacy of single-level vs.multi-level stigma interventions in order to determinethe conditions under which multi-level stigma interven-tions may be preferable to single-level interventions. Fu-ture research is also needed to evaluate how changes atone level of stigma (e.g., intrapersonal) impact otherlevels of stigma (e.g., community) in order to guide thedevelopment of more effective multi-level interven-tions, to identify mechanisms of change in multi-levelstigma interventions, and to explore the barriers andfacilitators to the dissemination of multi-level stigmainterventions across diverse contexts. Only after an-swering these questions will it be possible to fullyevaluate whether multi-level stigma interventions areeffective in addressing the predicament of stigma inthe lives of the stigmatized.

AbbreviationsCI: Confidence interval; ES: Effect size; HCW: Health care workers; HIC: High-income country; HIV: Human immunodeficiency virus; LMIC: Low- and

middle-income country; MI: Mental illness; PLWH: People living with HIV;PRISMA: Preferred Reporting Items for Systematic Reviews; PW: People with;RCT: Randomized controlled trial; US: United States

AcknowledgementsThis article is part of a collection that draws upon a 2017 workshop onstigma research andglobal health, which was organized by the FogartyInternational Center, National Institute of Health,United States. The articlewas supported by a generous contribution by the FogartyInternationalCenter.

FundingDR is supported by the University of Washington/Fred Hutch Center for AIDSResearch, an NIH-funded program under award number AI027757 which issupported by the following NIH Institutes and Centers: NIAID, NCI, NIMH,NIDA, NICHD, NHLBI, NIA, NIGMS, and NIDDK. Fogarty International Centersupported the publication of this manuscript through its conceptualizationas a topic, convening authors, and providing funds for its publication.Content is the responsibility of the authors and does not represent theviews of the National Institutes of Health, Fogarty International Center, orassociated funders.

Availability of data and materialsThe datasets used and/or analyzed during the current study are availablefrom the corresponding author on reasonable request.

Authors’ contributionsDR analyzed and interpreted data, completed the first draft of the paper, andincorporated co-author feedback. AE analyzed and interpreted data; draftedportions of the paper, table, and figures; and provided critical feedback ondrafts of the manuscript. MN designed the analysis, analyzed and interpreteddata, and provided critical feedback on drafts of the manuscript. MH contributedto framing, drafted portions of the paper, and provided critical feedback on draftsof the manuscript. SF analyzed and interpreted data and provided criticalfeedback on drafts of the manuscript. VG analyzed and interpreted dataand provided critical feedback on drafts of the manuscript. All authorshave read and approved the final manuscript.

Ethics approval and consent to participateNot applicable

Consent for publicationNot applicable

Competing interestsThe authors declare that they have no competing interests.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims in publishedmaps and institutional affiliations.

Author details1Department of Global Health, Department of Psychiatry and BehavioralSciences, University of Washington, Campus Mailbox 357965, HarrisHydraulics Building, 1705 NE Pacific Street, Seattle, WA 98195-7175, USA.2Department of Global Health, University of Washington, Seattle, WA, USA.3Department of Health Behavior, Gillings School of Public Health, Universityof North Carolina, Chapel Hill, NC, USA. 4Department of SociomedicalSciences, Mailman School of Public Health, Columbia University, New York,NY, USA.

Received: 30 May 2018 Accepted: 18 December 2018

References1. Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a fundamental cause of

population health inequalities. Am J Public Health. 2013;103:813–21.2. Treves-Kagan S, Steward WT, Ntswane L, et al. Why increasing availability of

ART is not enough: a rapid, community-based study on how HIV-relatedstigma impacts engagement to care in rural South Africa. BMC PublicHealth. 2016;16:87.

Table 2 Future directions for multi-level stigma interventions

Addressing research gaps

Most multi-level stigma intervention research has focused on theindividual/interpersonal level; thus, more research is needed toincorporate community-, organizational-, and structural-levelinfluences into such interventions.

Most multi-level stigma intervention research has utilized education-based strategies (either alone or in combination with other strategies,like contact) to reduce stigma. Thus, more research is needed across awider range of stigma-reducing strategies.

Only a handful of stigmatized groups have been the focus of multi-level stigma reduction interventions, with primary focus on HIV andmental health. Thus, more research is needed to expand the range ofgroups that are evaluated with these interventions.

Methods and measurement

More methodologically rigorous methods are needed to test theefficacy of multi-level stigma interventions, including randomized con-trolled trials and quasi-experiments.

New measurement approaches are needed to evaluate synergisticand reciprocal relations of stigma reduction interventions across levelsof analysis.

Multi-level stigma interventions need to more fully engage withseveral key areas in intervention science, such as implementationscience.

Research questions

How do changes at one level of stigma (e.g., intrapersonal) impactother levels of stigma (e.g., community)?

How do multi-level stigma interventions compare to stigma interventionsat a single level in terms of efficacy in reducing stigma and/or its negativeconsequences?

What are the mechanisms of change? That is, when multi-level stigmainterventions are effective, why are they effective?

How are multi-level stigma interventions that are found effectivetranslated or disseminated? What interpersonal-, community-, andstructural-level factors promote or undermine their effectivedissemination?

Rao et al. BMC Medicine (2019) 17:41 Page 10 of 11

3. Sirey JA, Bruce ML, Alexopoulos GS, Perlick DA, Friedman SJ, Meyers BS.Stigma as a barrier to recovery: perceived stigma and patient-rated severityof illness as predictors of antidepressant drug adherence. Psychiatr Serv.2001;52:1615–20.

4. Richman L, Hatzenbuehler ML. A multilevel analysis of stigma and health:implications for research and policy. Policy Instights Behav Brain Sci. 2014;1:213–21.

5. Heijnders M, Van Der Meij S. The fight against stigma: an overview ofstigma-reduction strategies and interventions. Psychol Health Med. 2006;11:353–63.

6. Cook JE, Purdie-Vaughns V, Meyer IH, Busch JT. Intervening within andacross levels: a multilevel approach to stigma and public health. Soc SciMed. 2014;103:101–9.

7. Bronfenbrenner U. Toward an experimental ecology of humandevelopment. Am Psychol. 1977;32:513–31.

8. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items forsystematic reviews and meta-analyses: the PRISMA statement. Open Med.2009;3:e123–30.

9. Covidence systematic review software, Veritas Health Innovation,Melbourne, Australia. Available at www.covidence.org.

10. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis:implications for conducting a qualitative descriptive study. Nurs Health Sci.2013;15(3):398–405.

11. Michaels P, Corrigan P, Buchholz B, et al. Changing stigma through aconsumer-based stigma reduction program. Community Ment Health J.2014;50:395–401.

12. Ngoc TN, Weiss B, Trung LT. Effects of the family schizophreniapsychoeducation program for individuals with recent onset schizophreniain Viet Nam. Asian J Psychiatr. 2016;22:162–6.

13. Jürgensen M, Sandøy IF, Michelo C, Fylkesnes K. Effects of home-basedvoluntary counselling and testing on HIV-related stigma: findings from acluster-randomized trial in Zambia. Soc Sci Med. 2013;81:18.

14. Dadun D, Van Brakel WH, Peters RMH, et al. Impact of socio-economicdevelopment, contact and peer counselling on stigma against personsaffected by leprosy in Cirebon, Indonesia--a randomised controlled trial.(Report). Lepr Rev. 2017;88:2.

15. Li L, Wu Z, Liang LJ, et al. An intervention targeting service providers andclients for methadone maintenance treatment in China: a cluster-randomized trial. Addiction. 2013;108:356–66.

16. Lusli M, Peters R, van Brakel W, et al. The impact of a rights-basedcounselling intervention to reduce stigma in people affected by leprosy inIndonesia.(Research Article)(Report). PLoS Negl Trop Dis. 2016;10:e0005088.

17. Patalay P, Annis J, Sharpe H, et al. A pre-post evaluation of OpenMinds: asustainable, peer-led mental health literacy programme in universities andsecondary schools. Prev Sci. 2017;18:995–1005.

18. Brown JF. Faith-based mental health education: a service-learning opportunityfor nursing students. J Psychiatr Ment Health Nurs. 2009;16:581–8.

19. Institute for Health Metrics and Evaluation. Despite economic slump, donorsgive generously to global health, though at a slower rate. 2010. (Accessed 4Apr 2018), at http://www.healthdata.org/news-release/despite-economic-slump-donors-give-generously-global-health-though-slower-rate.

20. Meyer I. Prejudice as stress: conceptual and measurement problems. Am JPublic Health. 2003;93:262–5.

21. Hatzenbuehler M. Structural stigma: research evidence and implications forpsychological science. Am Psychol. 2016;71:742–51.

22. Link B, Hatzenbuehler M. Stigma as an unrecognized determinant ofpopulation health: research and policy implications. J Health Polit PolicyLaw. 2016;41:653–73.

23. Pulerwitz J, Oanh KT, Akinwolemiwa D, Ashburn K, Nyblade L. Improvinghospital-based quality of care by reducing HIV-related stigma: evaluationresults from Vietnam. AIDS Behav. 2015;19:246–56.

24. Corrigan PW, Penn DL. Lessons from social psychology on discreditingpsychiatric stigma. Am Psychol. 1999;54:765–76.

25. Penn D, Corrigan P. The effects of stereotype suppression on psychiatricstigma. Schizophr Res. 2002;55:269–76.

26. Corrigan PW, River LP, Lundin R, et al. Stigmatizing attributions aboutmental illness. J Community Psychol. 2000;28(1):91–102 2000.

27. Batey DS, Whitfield S, Mulla M, et al. Adaptation and implementation of anintervention to reduce HIV-related stigma among healthcare workers in theUnited States: piloting of the FRESH workshop. AIDS Patient Care STDs.2016;30:519–27.

28. Bhana A, Mellins CA, Petersen I, et al. The VUKA family program: piloting afamily-based psychosocial intervention to promote health and mentalhealth among HIV infected early adolescents in South Africa. AIDS Care.2014;26:1–11.

29. Bogart KR, Hemmesch AR. Benefits of support conferences for parents ofand people with Moebius syndrome. Stigma Health. 2016;1:109–21.

30. Chidrawi HC, Greeff M, Temane QM, Doak CM. HIV stigma experiences andstigmatisation before and after an intervention. Health SA Gesondheid.2016;21:196–205.

31. Hawke LD, Michalak EE, Maxwell V, Parikh SV. Reducing stigma towardpeople with bipolar disorder: impact of a filmed theatrical interventionbased on a personal narrative. Int J Soc Psychiatry. 2014;60:741–50.

32. Henderson C, Corker E, Lewis-Holmes E, et al. England’s time to changeantistigma campaign: one-year outcomes of service user-rated experiencesof discrimination. Psychiatr Serv. 2012;63:451–7.

33. Maulik PK, Kallakuri S, Devarapalli S, Vadlamani VK, Jha V, Patel A. Increasinguse of mental health services in remote areas using mobile technology: apre-post evaluation of the SMART Mental Health project in rural India. JGlob Health. 2017;7(1).

34. Michalak E, Livingston J, Maxwell V, Hole R, Hawke L, Parikh S. Using theatreto address mental illness stigma: a knowledge translation study in bipolardisorder. Int J Bipolar Disorders. 2014;2(1):12.

35. Orkibi H, Bar N, Eliakim I. The effect of drama-based group therapy onaspects of mental illness stigma. Arts Psychother. 2014;41:458–66.

36. Pinfold V, Thornicroft G, Huxley P, Farmer P. Active ingredients in anti-stigma programmes in mental health. Int Rev Psychiatry. 2005;17:123–31.

37. Smith Fawzi MC, Eustache E, Oswald C, et al. Psychosocial support interventionfor HIV-affected families in Haiti: implications for programs and policies fororphans and vulnerable children. Soc Sci Med. 2012;74:1494–503.

38. Snead K, Ackerson J, Bailey K, Schmitt MM, Madan-Swain A, Martin RC.Taking charge of epilepsy: the development of a structuredpsychoeducational group intervention for adolescents with epilepsy andtheir parents. Epilepsy Behav. 2004;5:547–56.

39. Stuhlmiller CM. Breaking down the stigma of mental illness through anadventure camp: a collaborative education initiative. Australian e-JAdvancement Mental Health. 2003;2:90–8.

40. Thurman T, Jarabi B, Rice J. Caring for the caregiver: evaluation of supportgroups for guardians of orphans and vulnerable children in Kenya. AIDSCare. 2012;24:811–9.

41. Uys L, Chirwa M, Kohi T, et al. Evaluation of a health setting-based stigmaintervention in five African countries.(Author abstract). AIDS Patient CareSTDs. 2009;23:1059.

42. Yotsumoto K, Hirose T, Hashimoto T. An awareness program: thesignificance of lectures delivered by individuals with mental disabilities.World Federation Occup Ther Bull. 2010;62:40–5.

Rao et al. BMC Medicine (2019) 17:41 Page 11 of 11

http://www.covidence.orghttp://www.healthdata.org/news-release/despite-economic-slump-donors-give-generously-global-health-though-slower-ratehttp://www.healthdata.org/news-release/despite-economic-slump-donors-give-generously-global-health-though-slower-rate

AbstractBackgroundMethodsResultsConclusions

BackgroundMethodsSearch termsInclusion criteriaExclusion criteriaData extractionData analysis

ResultsDiscussionConclusionsAbbreviationsAcknowledgementsFundingAvailability of data and materialsAuthors’ contributionsEthics approval and consent to participateConsent for publicationCompeting interestsPublisher’s NoteAuthor detailsReferences