REVIEW ARTICLE Whole Systems Research Methods in Health Care: A Scoping Review Nadine Ijaz, PhD, 1 Jennifer Rioux, PhD, 2 Charles Elder, MD, MPH, 3 and John Weeks 4 Abstract Objectives: This scoping review evaluates two decades of methodological advances made by ‘‘whole systems research’’ (WSR) pioneers in the fields of traditional, complementary, and integrative medicine (TCIM). Rooted in critiques of the classical randomized controlled trial (RCT)’s suitability for evaluating holistic, complex TCIM interventions, WSR centralizes the principle of ‘‘model validity,’’ representing a ‘‘fit’’ between research design and therapeutic paradigm. Design: In consultation with field experts, 41 clinical research exemplars were selected for review from across 13 TCIM disciplines, with the aim of mapping the range and methodological characteristics of WSR studies. Using an analytic charting approach, these studies’ primary and secondary features are characterized with reference to three focal areas: research method, intervention design, and outcome assessment. Results: The reviewed WSR exemplars investigate a wide range of multimodal and multicomponent TCIM interventions, typified by wellness-geared, multitarget, and multimorbid therapeutic aims. Most studies include a behavioral focus, at times in multidisciplinary or team-based contexts. Treatments are variously individual- ized, often with reference to ‘‘dual’’ (biomedical and paradigm-specific) diagnoses. Prospective and retro- spective study designs substantially reflect established biomedical research methods. Pragmatic, randomized, open label comparative effectiveness designs with ‘‘usual care’’ comparators are most widely used, at times with factorial treatment arms. Only two studies adopt a double-blind, placebo-controlled RCT format. Some cohort-based controlled trials engage nonrandomized allocation strategies (e.g., matched controls, preference- based assignment, and minimization); other key designs include single-cohort pre–post studies, modified n-of-1 series, case series, case report, and ethnography. Mixed methods designs (i.e., qualitative research and eco- nomic evaluations) are evident in about one-third of exemplars. Primary and secondary outcomes are pre- dominantly assessed, at multiple intervals, through patient-reported measures for symptom severity, quality of life/wellness, and/or treatment satisfaction; some studies concurrently evaluate objective outcomes. Conclusions: Aligned with trends emphasizing ‘‘fit-for-purpose’’ research designs to study the ‘‘real-world’’ effectiveness of complex, personalized clinical interventions, WSR has emerged as a maturing scholarly dis- cipline. The field is distinguished by its patient-centered salutogenic focus and engagement with nonbiomedical diagnostic and treatment frameworks. The rigorous pursuit of model validity may be further advanced by emphasizing complex analytic models, paradigm-specific outcome assessment, inter-rater reliability, and eth- nographically informed designs. Policy makers and funders seeking to support best practices in TCIM research may refer to this review as a key resource. 1 Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Canada. 2 Integral Ayurveda and Yoga Therapy, Chapel Hill, NC. 3 Kaiser Permanente Center for Health Research, Portland, OR. 4 johnweeks-integrator.com, Editor-in-Chief, JACM, Seattle, WA. ª Nadine Ijaz et al., 2019; Published by Mary Ann Liebert, Inc. This Open Access article is distributed under the terms of the Creative Commons Attribution Noncommercial License (http://creative commons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original authors and the source are cited. THE JOURNAL OF ALTERNATIVE AND COMPLEMENTARY MEDICINE JACM Volume 25, Supplement 1, 2019, pp. S21–S51 Mary Ann Liebert, Inc. DOI: 10.1089/acm.2018.0499 S21
Transcript
REVIEW ARTICLE
Whole Systems Research Methods in Health CareA Scoping Review
Nadine Ijaz PhD1 Jennifer Rioux PhD2 Charles Elder MD MPH3 and John Weeks4
Abstract
Objectives This scoping review evaluates two decades of methodological advances made by lsquolsquowhole systemsresearchrsquorsquo (WSR) pioneers in the fields of traditional complementary and integrative medicine (TCIM) Rootedin critiques of the classical randomized controlled trial (RCT)rsquos suitability for evaluating holistic complexTCIM interventions WSR centralizes the principle of lsquolsquomodel validityrsquorsquo representing a lsquolsquofitrsquorsquo between researchdesign and therapeutic paradigm
Design In consultation with field experts 41 clinical research exemplars were selected for review fromacross 13 TCIM disciplines with the aim of mapping the range and methodological characteristics of WSRstudies Using an analytic charting approach these studiesrsquo primary and secondary features are characterizedwith reference to three focal areas research method intervention design and outcome assessment
Results The reviewed WSR exemplars investigate a wide range of multimodal and multicomponent TCIMinterventions typified by wellness-geared multitarget and multimorbid therapeutic aims Most studies includea behavioral focus at times in multidisciplinary or team-based contexts Treatments are variously individual-ized often with reference to lsquolsquodualrsquorsquo (biomedical and paradigm-specific) diagnoses Prospective and retro-spective study designs substantially reflect established biomedical research methods Pragmatic randomizedopen label comparative effectiveness designs with lsquolsquousual carersquorsquo comparators are most widely used at timeswith factorial treatment arms Only two studies adopt a double-blind placebo-controlled RCT format Somecohort-based controlled trials engage nonrandomized allocation strategies (eg matched controls preference-based assignment and minimization) other key designs include single-cohort prendashpost studies modified n-of-1series case series case report and ethnography Mixed methods designs (ie qualitative research and eco-nomic evaluations) are evident in about one-third of exemplars Primary and secondary outcomes are pre-dominantly assessed at multiple intervals through patient-reported measures for symptom severity quality oflifewellness andor treatment satisfaction some studies concurrently evaluate objective outcomes
Conclusions Aligned with trends emphasizing lsquolsquofit-for-purposersquorsquo research designs to study the lsquolsquoreal-worldrsquorsquoeffectiveness of complex personalized clinical interventions WSR has emerged as a maturing scholarly dis-cipline The field is distinguished by its patient-centered salutogenic focus and engagement with nonbiomedicaldiagnostic and treatment frameworks The rigorous pursuit of model validity may be further advanced byemphasizing complex analytic models paradigm-specific outcome assessment inter-rater reliability and eth-nographically informed designs Policy makers and funders seeking to support best practices in TCIM researchmay refer to this review as a key resource
1Leslie Dan Faculty of Pharmacy University of Toronto Toronto Canada2Integral Ayurveda and Yoga Therapy Chapel Hill NC3Kaiser Permanente Center for Health Research Portland OR4johnweeks-integratorcom Editor-in-Chief JACM Seattle WA
ordf Nadine Ijaz et al 2019 Published by Mary Ann Liebert Inc This Open Access article is distributed under the terms of theCreative Commons Attribution Noncommercial License (httpcreative commonsorglicensesby-nc40) which permits anynoncommercial use distribution and reproduction in any medium provided the original authors and the source are cited
THE JOURNAL OF ALTERNATIVE AND COMPLEMENTARY MEDICINE JACMVolume 25 Supplement 1 2019 pp S21ndashS51Mary Ann Liebert IncDOI 101089acm20180499
S21
Keywords whole systems research complementary therapies integrative medicine clinical trials as topic pragmatic
trials complex interventions
Introduction
The adoption of lsquolsquofit-for-purposersquorsquo clinical researchdesigns has emerged in recent decades as a significant
trend in health care Policy makers increasingly formulatesystem-wide decisions informed by the combined resultsof lsquolsquopragmaticrsquorsquo controlled trials which rigorously investigatethe real-world effectiveness of health care interventions(compared to their idealized lsquolsquoexplanatoryrsquorsquo efficacy)1 Morefunders now commit to reducing health care costs by under-writing studies of complex interventions focused on preventivemultidisciplinary care2 Researchers in turn widely augmentmeasurements of objective biomarkers by evaluating patient-reported outcomes directly meaningful to those suffering illhealth3 Finally patients continue to demand evidence-informed care that reflects their values and priorities4
Few would argue that the double-blind placebo-controlledrandomized controlled trial (RCT) continues to occupy prideof position at the top of evidence based medicine (EBM)rsquosmethodological hierarchy of clinical trial designs That saidresearchers from multiple fieldsmdashincluding traditionalcomplementary and integrative medicine (TCIM)mdashhavecritiqued the RCTrsquos limitations and its disproportionate evi-dentiary dominance The present work a scoping reviewrepresents a first retrospective analysis of almost two decadesof research design advances made by scholars committed torigorous holistic clinical research designs that accuratelyrepresent the unique paradigmatic features of TCIM lsquolsquowholesystemsrsquorsquo interventions
Background
In 2003 Ritenbaugh et almdashresearchers in the TCIMfieldmdashpublished a seminal article proposing a new branch ofscientific inquiry which they termed lsquolsquowhole systems re-searchrsquorsquo (WSR)5 WSR pioneers proposed to innovate clinicalresearch designs to address the theoretical-methodologicaldissonance that may arise in using classical RCT designsmdashrevered as the lsquolsquogold standardrsquorsquo in biomedical researchmdashtoappropriately study TCIM care TCIM lsquolsquowhole systemsrsquorsquoparadigms (eg Chinese medicine and naturopathic medi-cine) they argued exemplify several central features (de-tailed below) that distinguish them from conventionalbiomedicine At the heart of WSR is the model validityprinciple defined here as the lsquolsquofitrsquorsquo between a studyrsquos designand the conceptual and clinical features of the studied inter-ventionrsquos underlying or originating paradigm6 WSR advo-cates envisioned the pursuit of model validity as a way torigorously supplement (and reprioritize) existing approachesto achieving external and internal validity in clinical research
The dominant RCT design as critics had observed over thetwo decades prior7ndash9 seeks to study singular isolated thera-peutic components to lsquolsquodetermine the single best treatment forall patientsrsquorsquo5 TCIM treatments however are typically com-plex (involving multiple synergistic treatment modalities orcomponents) and individually tailored to the specific patient69
Classical RCTs were purpose developed to assess the causal
effects of pharmaceutic treatments on particular physiologicpathways under double-blinded placebo-controlled condi-tions1011 However many TCIM interventions are behavior-ally focused (with a lsquolsquosalutogenicrsquorsquo emphasis on lifestyle anddisease prevention) rendering clinician and participant blind-ing difficult Constructing credible inert placebo controls formany TCIM treatments (eg acupuncture chiropractic andmassage) had moreover proved notoriously challenging9 Fi-nally scholars working in the relatively-marginal TCIM fieldhave characterized the high cost of conducting classical RCTsas a prohibitive barrier to research feasibility12
WSR proponents in the TCIM field were certainly notalone in advocating for revisions to methodological con-ventions in clinical research investigators in some bio-medical fields (eg psychotherapy surgery and dietetics)had at the time articulated parallel concerns around theRCTrsquos universal applicability613 However WSR propo-nents additionally pointed to a unique set of research chal-lenges arising from paradigmatic features of TCIM lsquolsquowholesystemsrsquorsquo in relation to which these differ substantivelyfrom conventional biomedical approaches56
As detailed in Table 1 many whole TCIM systems rely onconceptual models and diagnostic approaches distinct from orin addition to biomedical science Alongside an integrated(lsquolsquowhole personrsquorsquo) assessment of a patientrsquos physical mentalemotional and psychosocial well-being many TCIM occu-pations foundationally attend to patient preferences priori-ties and values in their treatment designs514 Classical RCTsengage objective measures at discrete endpoints to evaluatepredetermined primary treatment outcomes related to a nar-rowly defined disease or dysfunction1516 Conversely TCIMprovidersmdashwhose interventions are often multitarget ormultimorbid in their aimsmdashtypically rely on subjective as-sessment modes to track progressive (and often long term)improvements in patient well-being alongside a range ofinter-relating symptoms1516 Finally while RCTs classicallyevaluate an interventionrsquos effects before it is being deployedin mainstream care TCIM therapies are often in widespreadusage before being formally trialed17
For those advocating a WSR approach the evaluation ofsingular standardized TCIM modalities within classicalRCT frameworks did not suffice as a means by which toevaluate these therapiesrsquo effects Rather they insisted thatmodel validity must be sought6 Mirroring a growing chorusof biomedical researchers WSR advocates heralded theascent of lsquolsquopragmaticrsquorsquo RCT designs whichmdashthey notedmdashmight rigorously compare the real-world effectiveness ofcomplex individualized interventions with lsquolsquousualrsquorsquo bio-medical care with reference to diverse rather than homo-genous populations618ndash20 They called for engagement withmodified RCT designs (eg patient preference factorial andn-of-1 trials matched or waiting list controls)619 and re-commended adoption of more efficient and equally-rigorousdesign-adaptive allocation alternatives to randomization(eg minimization)21 Advocating for mixed methodsstudy designs they argued that qualitative methods couldnot only lsquolsquoassist in the development of appropriate outcome
S22 IJAZ ET AL
Ta
ble
1
Ch
ara
cteristics
of
Clin
ica
lW
ho
le
Sy
stem
sP
ara
dig
ms
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Ayurv
edic
med
icin
e293
0T
ypolo
gic
alas
sess
men
tof
const
ituti
on
(pra
kruti
)an
ddis
equil
ibri
um
(vik
ruti
)in
rela
tion
tow
hole
per
son
apar
amet
ers
(thre
edosh
as
kapha
va
ta
pit
ta)
met
aboli
cfu
nct
ion
(agni)
to
xin
load
(am
a)
bodil
yes
sence
s(t
ejas
oja
spra
na)
qual
itie
sdis
ease
stag
es
and
loca
tions
Nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
pal
pat
ion
puls
ean
dto
ngue
asse
ssm
ent
det
oxifi
cati
on
and
reju
ven
atio
nth
erap
ies
Die
tan
dli
fest
yle
counse
ling
her
bal
med
icin
em
anual
ther
apie
sen
emas
and
purg
atio
n
nas
altr
eatm
ents
yoga
and
med
itat
ion
bre
athin
gex
erci
ses
musi
can
dm
antr
aan
dse
lf-a
war
enes
sac
tivit
ies
Anth
roposo
phic
med
icin
e31
Bio
med
ical
asse
ssm
ent
+ty
polo
gic
alas
sess
men
tof
whole
per
son
aco
nst
ituti
on
and
dis
equil
ibri
um
inre
lati
on
tofo
ur
level
sof
form
ativ
efo
rces
(physi
cal
ether
ic
astr
al
ego)
and
thre
efold
stru
ctura
lfu
nct
ional
syst
ems
(ner
ve-
sense
m
oto
r-m
etab
oli
crh
yth
mic
)
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
Anth
roposo
phic
med
icat
ion
(hom
eopat
hic
her
bal
)die
tan
dli
fest
yle
counse
ling
art
ther
apy
rhyth
mic
alm
assa
ge
ther
apy
Eury
thm
ym
ovem
ent
ther
apy
bio
gra
phic
alco
unse
ling
psy
choth
erap
yndash
usu
albio
med
ical
care
Chin
ese
med
icin
e32
Typolo
gic
alas
sess
men
tof
whole
per
son
a
const
ituti
on
(vit
alsu
bst
ance
s)an
ddis
equil
ibri
um
(pat
hogen
icfa
ctors
st
agnat
ions)
inre
lati
on
tosi
xdiv
isio
ns
of
yin
and
yang
syst
emfu
nct
ioni
nte
ract
ion
(five
elem
ents
)st
ages
and
level
sof
dis
ease
Nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
pal
pat
ion
puls
ean
dto
ngue
asse
ssm
ent
Acu
punct
ure
m
oxib
ust
ion
her
bal
med
icin
etu
ina
mas
sage
cuppin
g
guash
asc
rapin
g
trsquoai
chi
qi
gong
die
tary
and
life
style
counse
ling
Chir
opra
ctic
med
icin
e333
4A
sses
smen
tof
bio
mec
han
ical
dis
ord
ers
bas
edon
bio
med
ical
conce
pts
of
musc
ulo
skel
etal
ner
vous
syst
ems
contr
over
sial
lyh
isto
rica
lly
conce
ptu
aliz
edin
nonbio
med
ical
term
sas
lsquolsquover
tebra
lsu
blu
xat
ionrsquo
rsquo
Bio
med
ical
dia
gnosi
s+
physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
dia
gnost
icim
agin
g
labora
tory
test
ing
Spin
alan
dso
ftti
ssue
man
ipula
tion
physi
cal
modal
itie
shom
eca
re
and
counse
ling
on
die
tex
erci
se
and
stre
ssre
duct
ion
Com
ple
men
tary
in
tegra
tive
med
icin
e35
Incl
usi
on
of
trea
tmen
tsori
gin
atin
gfr
om
ara
nge
of
whole
syst
emw
hole
pra
ctic
epar
adig
ms
wit
hin
the
ausp
ices
of
conven
tional
pre
ven
tive
bio
med
ical
hea
lth
care
del
iver
y
Bio
med
ical
dia
gnost
ics
+opti
onal
syst
em
modal
ity-s
pec
ific
dia
gnost
ics
Usu
albio
med
ical
care
plu
ssi
ngle
or
mult
iple
trea
tmen
tap
pro
aches
from
one
or
more
whole
syst
ems
whole
pra
ctic
epar
adig
ms
(incl
udin
gap
pro
aches
not
list
edher
e)
Ener
gy
med
icin
e36
Ass
essm
ent
of
whole
per
son
ben
erget
icfi
eld
Intu
itiv
een
erget
icobse
rvat
ions
Ara
nge
of
on-b
ody
(eg
hea
ling
touch
R
eiki)
and
off
-body
trea
tmen
ts
Hom
eopat
hic
med
icin
e373
8T
ypolo
gic
alas
sess
men
tof
rem
edy
signat
ure
(sim
illi
mum
)of
whole
per
son
aco
nst
ituti
on
and
dis
equil
ibri
um
poss
ibly
inre
lati
on
todis
ease
mia
sm(e
g
pso
ric
syco
tic
syphil
itic
)an
do
rkin
gdom
(pla
nt
anim
al
min
eral
)
Nar
rati
ve
case
-tak
ing
Hom
eopat
hic
dil
uti
ons
of
ara
nge
of
pla
nt
anim
al
and
min
eral
subst
ance
s
Mid
wif
ery
39
Wom
an-c
ente
red
per
inat
alca
rein
whic
hbir
this
norm
aliz
edas
ahea
lthy
even
tan
dth
em
idw
ifersquo
sro
leis
toholi
stic
ally
support
and
faci
lita
teth
ein
div
idual
wom
anrsquos
bir
thin
gch
oic
es
Bio
med
ical
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
Cas
e-lo
adbas
ed(c
onti
nuous)
as
wel
las
mid
wif
e-le
dt
eam
-bas
ed(n
onco
nti
nuous)
pre
-in
tra-
an
dpost
par
tum
bir
th-r
elat
edca
re(i
ncl
udin
gco
unse
ling
rela
ted
todie
tli
fest
yle
an
din
fant
feed
ingc
are)
wit
hopti
on
of
hom
e-or
hosp
ital
bir
th
(conti
nued
)
S23
Ta
ble
1
(Co
ntin
ued
)
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Nat
uro
pat
hic
med
icin
e40
Bio
med
ical
asse
ssm
ent
rein
terp
rete
dth
rough
aw
hole
per
son
ale
ns
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
asse
ssm
ents
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
labora
tory
and
bio
elec
tric
alte
stin
g
pal
pat
ion
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
dia
gnost
ics
Die
tphysi
cal
acti
vit
y
stre
ssm
anag
emen
tco
unse
ling
her
bal
med
icin
enutr
itio
nal
supple
men
tati
on
acupunct
ure
hom
eopat
hy
hydro
ther
apy
man
ual
ther
apie
sphysi
cal
modal
itie
san
din
stru
ctio
nin
min
db
ody
tech
niq
ues
Pre
ven
tive
rest
ora
tive
bio
med
icin
ecB
iom
edic
alas
sess
men
tw
ith
pre
ven
tive
rest
ora
tive
and
physi
olo
gic
psy
choso
cial
lens
Bio
med
ical
In
div
idual
gro
up-b
ased
beh
avio
ral
inte
rven
tions
gea
red
topre
ven
tingr
ehab
ilit
atin
gch
ronic
dis
ease
in
cludin
gdie
tnutr
ients
ex
erci
se
slee
p
stre
ssm
anag
emen
tpsy
choso
cial
support
sm
indb
ody
tech
niq
ues
ndashu
sual
bio
med
ical
care
Sw
edis
hm
assa
ge
ther
apy
41
Bio
med
ical
asse
ssm
ent
wit
han
emphas
ison
soft
tiss
ue
and
musc
ulo
skel
etal
dis
ord
ers
Bio
med
ical
dia
gnost
ics
+physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
Man
ual
ther
apy
incl
udin
gfr
icti
on
effleu
rage
pet
riss
age
vib
rati
on
tapote
men
tan
dsk
in-
roll
ing
trsquoai
chi4
2S
eeC
hin
ese
med
icin
eab
ove
See
Chin
ese
med
icin
eab
ove
Rit
ual
ized
movem
ent
sequen
cein
corp
ora
ting
bre
athw
ork
m
indfu
lnes
sim
ager
y
physi
cal
touch
an
dso
cial
inte
ract
ion
Yoga
ther
apy
434
4M
anag
emen
tre
duct
ion
or
elim
inat
ion
of
sym
pto
ms
that
pro
duce
suff
erin
g
enhan
cem
ent
of
funct
ion
illn
ess
pre
ven
tion
and
salu
togen
esis
Ass
essm
ent
of
per
son
aslsquolsquo
mult
idim
ensi
onal
syst
emrsquorsquo
that
incl
udes
inte
rconnec
tions
of
body
bre
ath
inte
llec
tm
ind
and
emoti
ons
Ther
apeu
tic
movem
ent
(asa
na)
and
bre
athw
ork
(pra
naya
ma)
wit
hdie
tary
m
edit
atio
n
man
tra
mudra
ch
anti
ng
ritu
al
and
self
-aw
aren
ess
life
style
pra
ctic
es
aT
his
table
pro
vid
esan
over
vie
wof
sele
cted
whole
syst
ems
par
adig
ms
studie
sfr
om
whic
har
eev
aluat
edin
this
revie
w
Itis
not
mea
nt
tobe
anex
hau
stiv
ere
pre
senta
tion
of
all
clin
ical
whole
syst
emsmdash
ther
ear
em
any
oth
ers
bW
hole
per
son
par
amet
ers
concu
rren
tly
addre
ssphysi
olo
gic
psy
cholo
gic
men
tal
emoti
onal
sp
irit
ual
so
cial
in
terg
ener
atio
nal
an
den
vir
onm
enta
lfa
ctors
aspar
tof
aholi
stic
conce
ptu
alpar
adig
m
Inoth
erw
ord
sth
ese
fact
ors
are
under
stood
asfu
ndam
enta
lly
inte
rconnec
ted
and
mutu
ally
gen
erat
ive
inre
lati
on
tohea
lth
and
wel
l-bei
ng
cT
he
term
lsquolsquopre
ven
tive
rest
ora
tive
bio
med
icin
ersquorsquo
ispro
vis
ional
lyuse
dher
eto
char
acte
rize
studie
sw
ith
ase
tof
uniq
ue
par
adig
mat
icfe
ature
sle
dby
conven
tional
med
ical
doct
ors
S24
measuresrsquorsquo before a clinical trial but also gather lsquolsquouniquephysical and psychosocial contextrsquorsquo within it and subse-quently help to lsquolsquoexplain the trial resultsrsquorsquo22
Going further WSR proponents argued that diverse researchmodesmdashprospective and retrospective experimental quasi-experimental and observational qualitative and quantitativeand holistic and reductivemdashbe equally valued for their distinctcontributions and rigorously applied as contextually appro-priate623 Asserting that EBMrsquos lsquolsquoprescriptive evidence hier-archies of research methodsrsquorsquo should be supplanted20 TCIMscholars variously conceptualized evidentiary frameworks(eg lsquolsquoevidence matrixrsquorsquo24 lsquolsquoevidence housersquorsquo25 and lsquolsquocircularmodelrsquorsquo19) in which a range of research designs might syn-thetically contribute to assessing a particular interventionrsquosefficacy effectiveness and other contextual dimensions
Taking on model validity with respect to intervention se-lection and design WSR advocates favored the evaluation oflsquolsquowhole systems or lsquobundlesrsquo of therapiesrsquorsquo rather thanlsquolsquosinglemodalitiesrsquorsquo alone6 They envisioned studies inwhich patients would undergo lsquolsquodouble classificationrsquorsquo usingbiomedical diagnostics as well as diagnosis from within therelevant TCIM paradigm and receive care that was individ-ualized on this basis6 Research teams they advised shouldinclude insiders from within the paradigms in which the in-terventions originated2627 and study recruitment strategiesshould address persons with complex multifactorial healthconditions28 as well as patient treatment preferences14
WSR leaders equally envisioned study outcome assessmentin relation to the model validity principle6 At a time whenvalidated patient-reported outcome measures (PROMs) werejust beginning to be widely used in conventional researchWSR proponents characterized subjective and paradigm-adherent quantitative outcome measures1522 as key evaluativetools alongside qualitative methods22 Measurables theyproposed should be multiple (addressing the therapeutictechniques applied patientndashpractitioner relationship and rangeof healthwellness impacts1545) and at more frequent intervalsand over a longer period than in conventional trials1623 lsquolsquoIn-novative statistical methodologyrsquorsquo6mdashincluding lsquolsquoparticipant-centeredrsquorsquo approaches46mdashwould be needed to synthesize thevoluminous data generated6 They called for lsquolsquocomplex con-ceptual modelsrsquorsquo16 to evaluate a whole systemrsquos combinedeffects lsquolsquoover and above its componentsrsquorsquo6 and variously pro-posed methodological engagement with network sciencecomplexity science and nonlinear dynamical systems2347ndash49
action research716 and program theory16 to this endSince 2003 the dominant landscape of clinical research
has transformed significantly Although the classical RCTcontinues to be prioritized in EBMrsquos evidentiary hierarchypragmatically designed comparative effectiveness studiesand lsquolsquofit-for-purposersquorsquo research designs1 have become morewidely accepted as important clinical and policy-makingresources50 Usage of PROMs clinical trial guidelines andquality assessment tools has become more widespread andlsquolsquofollowing considerable development in the fieldrsquorsquo theMedical Research Councilrsquos framework for trialing complexinterventions will once again be renewed in 20192 Withinthe TCIM world WSR principles have been increasinglytaken up4351ndash55 although more conventional research de-signs still predominate56 To date however no compre-hensive retrospective analysis of WSR advances has beenundertaken that is thus the present workrsquos aim
Methods
This article is a scoping review of the methodologicalfeatures of WSR studies with reference to the model validityprinciple Scoping reviews lsquolsquomap the literature on a particulartopic or research area and provide an opportunity to identifykey concepts gaps in the research and types and sources ofevidence to inform practice policymaking and researchrsquorsquo57
Scoping reviews lsquolsquodiffer from systematic reviews as authorsdo not typically assess the quality ofrsquorsquo58 nor lsquolsquoseek to lsquosyn-thesizersquo evidence or to aggregate findings from differentstudiesrsquorsquo59 They also diverge from lsquolsquonarrative or literaturereviews in that the scoping process requires analytical rein-terpretation of the literaturersquorsquo58 lsquolsquoNot linear but iterativersquorsquo incharacter scoping reviews primarily take a qualitative ana-lytic approach supported by numerical representation of thelsquolsquoextent nature and distributionrsquorsquo of key findings59
The present review adopts Arksey and OrsquoMalleyrsquos six-step scoping study framework involving (1) researchquestion identification (2) study identification (3) studyselection (4) data charting (5) result collation summaryand reporting and (6) (optional) consultation with areaexperts to validate findings59
Research question identification
The primary question driving this review is twofold in-terrogating (1) the range and characteristics of WSR clin-ical studies and (2) the ways in which these studies engagethe model validity principle
Study identification
WSR-type studies have been undertaken in multiple healthcare paradigms and the methodological terminology usedacross them varies Thus a broad initial keyword-based lit-erature search (eg lsquolsquowhole systems researchrsquorsquo lsquolsquocomplexrsquorsquolsquolsquoindividualizedrsquorsquo lsquolsquocomplementary medicinersquorsquo and lsquolsquomodelvalidityrsquorsquo) helped to locate many relevant methodologicalpublications but proved insufficient to identify a represen-tative set of clinical WSR exemplars A group of field experts(listed in the study acknowledgments) was therefore as-sembled by one coauthor (JW) to share WSR exemplarcitations In addition to reviewing the relevant historical lit-eratures the primary review author (NI) reviewed each ofthese recommended studies and scrutinized their referencelists for additional candidate exemplars The other coauthors( JR and CE) as WSR field experts further supplementedthis initial list As the review process progressed studyidentification through additional literature searches continuediteratively with study selection and data charting (below)
Study selection
To be eligible for inclusion studies were required to di-rectly report clinical outcomes with respect to an interven-tion based in a defined therapeutic whole system marked by aconceptual andor diagnostic model distinct from conven-tional biomedical care Studies adopting complex individu-alized salutogenic andor multimorbidmultitarget modesof care were prioritized Only peer-reviewed studies (withone or more associated publications) were included andall demonstrated a strong emphasis on model validity in atleast one of the following adopted research method(s)
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S25
intervention selection or design and outcome assessmentStudies were not required to refer directly to the model va-lidity principle nor to use WSR terminology Pilotfeasibilitydesigns were included unfulfilled study protocols were notNo attempt was made to exhaustively assemble all publishedstudies meeting study inclusion criteria rather the emphasiswas on assembling a diverse subset of such studies
Addressing a long-standing debate in the WSR field20
the multicomponent stand-alone disciplines of yoga therapyand trsquoai chi were defined as distinct whole systems despitetheir respective historical and conceptual connections tothe Ayurvedic and Chinese medicine systems Studies frommidwifery (a discipline not always included under the TCIMlsquolsquoumbrellarsquorsquo) were determined eligible for inclusion based on(1) the professionrsquos uniquely holistic woman-centered para-digm distinct from conventional obstetrics and (2) itshistorical roots in traditionalindigenous health care Stu-dies from a field provisionally termed lsquolsquopreventiverestorativebiomedicinersquorsquo were also included recognizing that (1)such studies diverge paradigmatically from conventionaltherapeutic norms and (2) that the multimodal behavior-ally focused studies led in particular by Ornish et al in the1980s6061 provided early methodological inspiration forwhole systems researchers
Study selection (ie identification charting and culling)continued iteratively until (1) lsquolsquotheoretical saturationrsquorsquo62
was reached in that review of additional candidate publi-cations failed to reveal new WSR methodological featuresand (2) a wide range of clinical whole systems paradigmswere represented within the dataset Study results were nottaken into account during the selection process
About 90 of studies recommended by at least one subjectarea expert were included and approximately two-thirds of theincluded studies had been directly recommended by at leastone WSR expert (including coauthors JR and CE) the re-mainder was identified in literature searches undertaken by theprimary author (NI) Four expert-recommended studies wereexcluded because they (1) did not meet the study inclusioncriteria (n = 1) or (2) were methodologically very similar toother selected exemplars providing little added value tothe review (n = 3) The final selection of studies deliberatelyover-represents traditional (ie Ayurvedic and Chinese)medicine systems to thoroughly address the paradigm-specificdiagnostic intervention and outcome design considerationsthat arise in these contexts
Data charting
Focused around three primary analytic categoriesmdashStudyDesign Intervention Selection and Outcome Evaluationmdashthe primary author (NI) summarized and evaluated eachcandidate study using an emergent set of tables and chartsThrough a constant comparative approach that reviewedeach study in relation to all others63 a set of analytic sub-parameters and conceptual frames progressively emergedThis process permitted a finalized study selection and adetailing of each studyrsquos distinct and nondistinct methodo-logical features
Expert validation of findings
While analysis and reporting were undertaken primarilyby the primary author (NI) a subset of categorizations
related to lsquolsquodual diagnosticsrsquorsquo and paradigm-specific out-comes was independently corroborated by another coauthor( JR) All coauthors ( JR CE JW) contributed insightsas to the emerging conceptual categories as the projectprogressed and provided input on the final analyses beforethis workrsquos peer review by other WSR field experts
Result collation summary and reporting
Results are synthetically presented and discussed in whatfollows using both narrative and graphical reporting Tofacilitate reading ease in-text WSR exemplar referencesname first authors only full citations may be found in thereference list To provide context and language to facilitatenuanced reporting of the WSR fieldrsquos features two noveltheoretical frameworks are presented below
Theory
Model validity framework
The model validity principle has been conceptualized ascentral to WSR and as noted earlier on various scholarshave suggested ways in which this principle may be enactedwithin clinical research contexts What remained implicit inmuch (although not all) of the early WSR methodologicalliterature is that WSR itself may be understood as part of anlsquolsquointegrative medicinersquorsquo movement64 geared to transformingdominant health care systems such that TCIM therapies maybe more broadly integrated alongside or as an adjunct toconventional biomedical care Clinical research is in thisscenario envisioned as a necessary but insufficient tool tohelp to dismantle barriers to integration17
Several scholars critiquing the integrative medicineproject have however suggested a potential for the distinctparadigmatic features of and practices with origins in non-biomedical therapeutic systems to be co-opted appro-priated or assimilated in such a process65 Model validityas a theoretical construct represents a commitment to ac-tively preserving these paradigms and practices in their ownright an approach aligned with the concept of a clearlylsquolsquoarticulatedrsquorsquo66 equitable medical pluralism67 rather thanan assimilative mode of integration65
What WSR pioneers were not able to fully apprise inadvance was how and to what degree future WSR methodsmight ultimately align or diverge with conventional researchstrategies in pursuit of model validity To facilitate analysisof these points in the present scoping review it is proposedthat the model validity principle be theoretically differenti-ated into three co-embedded categories as seen in Figure 1paradigm compatibility paradigm consistency and paradigmspecificity These categories are not mutually exclusive thatis a single study may concurrently include different aspects(eg method intervention design and outcome measures)marked by one or more of the identified characteristics
Paradigm compatibility model validityrsquos driving conceptis conceptualized as a category that includes two othersmdashparadigm consistency and paradigm specificitymdashthe secondof which is embedded within the first Paradigm compatibleresearch methods are those typically associated with domi-nant biomedical clinical research but which also readilylend themselves to the study of whole systems clinical in-terventions Paradigm-consistent methods differ in key
S26 IJAZ ET AL
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Keywords whole systems research complementary therapies integrative medicine clinical trials as topic pragmatic
trials complex interventions
Introduction
The adoption of lsquolsquofit-for-purposersquorsquo clinical researchdesigns has emerged in recent decades as a significant
trend in health care Policy makers increasingly formulatesystem-wide decisions informed by the combined resultsof lsquolsquopragmaticrsquorsquo controlled trials which rigorously investigatethe real-world effectiveness of health care interventions(compared to their idealized lsquolsquoexplanatoryrsquorsquo efficacy)1 Morefunders now commit to reducing health care costs by under-writing studies of complex interventions focused on preventivemultidisciplinary care2 Researchers in turn widely augmentmeasurements of objective biomarkers by evaluating patient-reported outcomes directly meaningful to those suffering illhealth3 Finally patients continue to demand evidence-informed care that reflects their values and priorities4
Few would argue that the double-blind placebo-controlledrandomized controlled trial (RCT) continues to occupy prideof position at the top of evidence based medicine (EBM)rsquosmethodological hierarchy of clinical trial designs That saidresearchers from multiple fieldsmdashincluding traditionalcomplementary and integrative medicine (TCIM)mdashhavecritiqued the RCTrsquos limitations and its disproportionate evi-dentiary dominance The present work a scoping reviewrepresents a first retrospective analysis of almost two decadesof research design advances made by scholars committed torigorous holistic clinical research designs that accuratelyrepresent the unique paradigmatic features of TCIM lsquolsquowholesystemsrsquorsquo interventions
Background
In 2003 Ritenbaugh et almdashresearchers in the TCIMfieldmdashpublished a seminal article proposing a new branch ofscientific inquiry which they termed lsquolsquowhole systems re-searchrsquorsquo (WSR)5 WSR pioneers proposed to innovate clinicalresearch designs to address the theoretical-methodologicaldissonance that may arise in using classical RCT designsmdashrevered as the lsquolsquogold standardrsquorsquo in biomedical researchmdashtoappropriately study TCIM care TCIM lsquolsquowhole systemsrsquorsquoparadigms (eg Chinese medicine and naturopathic medi-cine) they argued exemplify several central features (de-tailed below) that distinguish them from conventionalbiomedicine At the heart of WSR is the model validityprinciple defined here as the lsquolsquofitrsquorsquo between a studyrsquos designand the conceptual and clinical features of the studied inter-ventionrsquos underlying or originating paradigm6 WSR advo-cates envisioned the pursuit of model validity as a way torigorously supplement (and reprioritize) existing approachesto achieving external and internal validity in clinical research
The dominant RCT design as critics had observed over thetwo decades prior7ndash9 seeks to study singular isolated thera-peutic components to lsquolsquodetermine the single best treatment forall patientsrsquorsquo5 TCIM treatments however are typically com-plex (involving multiple synergistic treatment modalities orcomponents) and individually tailored to the specific patient69
Classical RCTs were purpose developed to assess the causal
effects of pharmaceutic treatments on particular physiologicpathways under double-blinded placebo-controlled condi-tions1011 However many TCIM interventions are behavior-ally focused (with a lsquolsquosalutogenicrsquorsquo emphasis on lifestyle anddisease prevention) rendering clinician and participant blind-ing difficult Constructing credible inert placebo controls formany TCIM treatments (eg acupuncture chiropractic andmassage) had moreover proved notoriously challenging9 Fi-nally scholars working in the relatively-marginal TCIM fieldhave characterized the high cost of conducting classical RCTsas a prohibitive barrier to research feasibility12
WSR proponents in the TCIM field were certainly notalone in advocating for revisions to methodological con-ventions in clinical research investigators in some bio-medical fields (eg psychotherapy surgery and dietetics)had at the time articulated parallel concerns around theRCTrsquos universal applicability613 However WSR propo-nents additionally pointed to a unique set of research chal-lenges arising from paradigmatic features of TCIM lsquolsquowholesystemsrsquorsquo in relation to which these differ substantivelyfrom conventional biomedical approaches56
As detailed in Table 1 many whole TCIM systems rely onconceptual models and diagnostic approaches distinct from orin addition to biomedical science Alongside an integrated(lsquolsquowhole personrsquorsquo) assessment of a patientrsquos physical mentalemotional and psychosocial well-being many TCIM occu-pations foundationally attend to patient preferences priori-ties and values in their treatment designs514 Classical RCTsengage objective measures at discrete endpoints to evaluatepredetermined primary treatment outcomes related to a nar-rowly defined disease or dysfunction1516 Conversely TCIMprovidersmdashwhose interventions are often multitarget ormultimorbid in their aimsmdashtypically rely on subjective as-sessment modes to track progressive (and often long term)improvements in patient well-being alongside a range ofinter-relating symptoms1516 Finally while RCTs classicallyevaluate an interventionrsquos effects before it is being deployedin mainstream care TCIM therapies are often in widespreadusage before being formally trialed17
For those advocating a WSR approach the evaluation ofsingular standardized TCIM modalities within classicalRCT frameworks did not suffice as a means by which toevaluate these therapiesrsquo effects Rather they insisted thatmodel validity must be sought6 Mirroring a growing chorusof biomedical researchers WSR advocates heralded theascent of lsquolsquopragmaticrsquorsquo RCT designs whichmdashthey notedmdashmight rigorously compare the real-world effectiveness ofcomplex individualized interventions with lsquolsquousualrsquorsquo bio-medical care with reference to diverse rather than homo-genous populations618ndash20 They called for engagement withmodified RCT designs (eg patient preference factorial andn-of-1 trials matched or waiting list controls)619 and re-commended adoption of more efficient and equally-rigorousdesign-adaptive allocation alternatives to randomization(eg minimization)21 Advocating for mixed methodsstudy designs they argued that qualitative methods couldnot only lsquolsquoassist in the development of appropriate outcome
S22 IJAZ ET AL
Ta
ble
1
Ch
ara
cteristics
of
Clin
ica
lW
ho
le
Sy
stem
sP
ara
dig
ms
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Ayurv
edic
med
icin
e293
0T
ypolo
gic
alas
sess
men
tof
const
ituti
on
(pra
kruti
)an
ddis
equil
ibri
um
(vik
ruti
)in
rela
tion
tow
hole
per
son
apar
amet
ers
(thre
edosh
as
kapha
va
ta
pit
ta)
met
aboli
cfu
nct
ion
(agni)
to
xin
load
(am
a)
bodil
yes
sence
s(t
ejas
oja
spra
na)
qual
itie
sdis
ease
stag
es
and
loca
tions
Nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
pal
pat
ion
puls
ean
dto
ngue
asse
ssm
ent
det
oxifi
cati
on
and
reju
ven
atio
nth
erap
ies
Die
tan
dli
fest
yle
counse
ling
her
bal
med
icin
em
anual
ther
apie
sen
emas
and
purg
atio
n
nas
altr
eatm
ents
yoga
and
med
itat
ion
bre
athin
gex
erci
ses
musi
can
dm
antr
aan
dse
lf-a
war
enes
sac
tivit
ies
Anth
roposo
phic
med
icin
e31
Bio
med
ical
asse
ssm
ent
+ty
polo
gic
alas
sess
men
tof
whole
per
son
aco
nst
ituti
on
and
dis
equil
ibri
um
inre
lati
on
tofo
ur
level
sof
form
ativ
efo
rces
(physi
cal
ether
ic
astr
al
ego)
and
thre
efold
stru
ctura
lfu
nct
ional
syst
ems
(ner
ve-
sense
m
oto
r-m
etab
oli
crh
yth
mic
)
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
Anth
roposo
phic
med
icat
ion
(hom
eopat
hic
her
bal
)die
tan
dli
fest
yle
counse
ling
art
ther
apy
rhyth
mic
alm
assa
ge
ther
apy
Eury
thm
ym
ovem
ent
ther
apy
bio
gra
phic
alco
unse
ling
psy
choth
erap
yndash
usu
albio
med
ical
care
Chin
ese
med
icin
e32
Typolo
gic
alas
sess
men
tof
whole
per
son
a
const
ituti
on
(vit
alsu
bst
ance
s)an
ddis
equil
ibri
um
(pat
hogen
icfa
ctors
st
agnat
ions)
inre
lati
on
tosi
xdiv
isio
ns
of
yin
and
yang
syst
emfu
nct
ioni
nte
ract
ion
(five
elem
ents
)st
ages
and
level
sof
dis
ease
Nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
pal
pat
ion
puls
ean
dto
ngue
asse
ssm
ent
Acu
punct
ure
m
oxib
ust
ion
her
bal
med
icin
etu
ina
mas
sage
cuppin
g
guash
asc
rapin
g
trsquoai
chi
qi
gong
die
tary
and
life
style
counse
ling
Chir
opra
ctic
med
icin
e333
4A
sses
smen
tof
bio
mec
han
ical
dis
ord
ers
bas
edon
bio
med
ical
conce
pts
of
musc
ulo
skel
etal
ner
vous
syst
ems
contr
over
sial
lyh
isto
rica
lly
conce
ptu
aliz
edin
nonbio
med
ical
term
sas
lsquolsquover
tebra
lsu
blu
xat
ionrsquo
rsquo
Bio
med
ical
dia
gnosi
s+
physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
dia
gnost
icim
agin
g
labora
tory
test
ing
Spin
alan
dso
ftti
ssue
man
ipula
tion
physi
cal
modal
itie
shom
eca
re
and
counse
ling
on
die
tex
erci
se
and
stre
ssre
duct
ion
Com
ple
men
tary
in
tegra
tive
med
icin
e35
Incl
usi
on
of
trea
tmen
tsori
gin
atin
gfr
om
ara
nge
of
whole
syst
emw
hole
pra
ctic
epar
adig
ms
wit
hin
the
ausp
ices
of
conven
tional
pre
ven
tive
bio
med
ical
hea
lth
care
del
iver
y
Bio
med
ical
dia
gnost
ics
+opti
onal
syst
em
modal
ity-s
pec
ific
dia
gnost
ics
Usu
albio
med
ical
care
plu
ssi
ngle
or
mult
iple
trea
tmen
tap
pro
aches
from
one
or
more
whole
syst
ems
whole
pra
ctic
epar
adig
ms
(incl
udin
gap
pro
aches
not
list
edher
e)
Ener
gy
med
icin
e36
Ass
essm
ent
of
whole
per
son
ben
erget
icfi
eld
Intu
itiv
een
erget
icobse
rvat
ions
Ara
nge
of
on-b
ody
(eg
hea
ling
touch
R
eiki)
and
off
-body
trea
tmen
ts
Hom
eopat
hic
med
icin
e373
8T
ypolo
gic
alas
sess
men
tof
rem
edy
signat
ure
(sim
illi
mum
)of
whole
per
son
aco
nst
ituti
on
and
dis
equil
ibri
um
poss
ibly
inre
lati
on
todis
ease
mia
sm(e
g
pso
ric
syco
tic
syphil
itic
)an
do
rkin
gdom
(pla
nt
anim
al
min
eral
)
Nar
rati
ve
case
-tak
ing
Hom
eopat
hic
dil
uti
ons
of
ara
nge
of
pla
nt
anim
al
and
min
eral
subst
ance
s
Mid
wif
ery
39
Wom
an-c
ente
red
per
inat
alca
rein
whic
hbir
this
norm
aliz
edas
ahea
lthy
even
tan
dth
em
idw
ifersquo
sro
leis
toholi
stic
ally
support
and
faci
lita
teth
ein
div
idual
wom
anrsquos
bir
thin
gch
oic
es
Bio
med
ical
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
Cas
e-lo
adbas
ed(c
onti
nuous)
as
wel
las
mid
wif
e-le
dt
eam
-bas
ed(n
onco
nti
nuous)
pre
-in
tra-
an
dpost
par
tum
bir
th-r
elat
edca
re(i
ncl
udin
gco
unse
ling
rela
ted
todie
tli
fest
yle
an
din
fant
feed
ingc
are)
wit
hopti
on
of
hom
e-or
hosp
ital
bir
th
(conti
nued
)
S23
Ta
ble
1
(Co
ntin
ued
)
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Nat
uro
pat
hic
med
icin
e40
Bio
med
ical
asse
ssm
ent
rein
terp
rete
dth
rough
aw
hole
per
son
ale
ns
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
asse
ssm
ents
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
labora
tory
and
bio
elec
tric
alte
stin
g
pal
pat
ion
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
dia
gnost
ics
Die
tphysi
cal
acti
vit
y
stre
ssm
anag
emen
tco
unse
ling
her
bal
med
icin
enutr
itio
nal
supple
men
tati
on
acupunct
ure
hom
eopat
hy
hydro
ther
apy
man
ual
ther
apie
sphysi
cal
modal
itie
san
din
stru
ctio
nin
min
db
ody
tech
niq
ues
Pre
ven
tive
rest
ora
tive
bio
med
icin
ecB
iom
edic
alas
sess
men
tw
ith
pre
ven
tive
rest
ora
tive
and
physi
olo
gic
psy
choso
cial
lens
Bio
med
ical
In
div
idual
gro
up-b
ased
beh
avio
ral
inte
rven
tions
gea
red
topre
ven
tingr
ehab
ilit
atin
gch
ronic
dis
ease
in
cludin
gdie
tnutr
ients
ex
erci
se
slee
p
stre
ssm
anag
emen
tpsy
choso
cial
support
sm
indb
ody
tech
niq
ues
ndashu
sual
bio
med
ical
care
Sw
edis
hm
assa
ge
ther
apy
41
Bio
med
ical
asse
ssm
ent
wit
han
emphas
ison
soft
tiss
ue
and
musc
ulo
skel
etal
dis
ord
ers
Bio
med
ical
dia
gnost
ics
+physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
Man
ual
ther
apy
incl
udin
gfr
icti
on
effleu
rage
pet
riss
age
vib
rati
on
tapote
men
tan
dsk
in-
roll
ing
trsquoai
chi4
2S
eeC
hin
ese
med
icin
eab
ove
See
Chin
ese
med
icin
eab
ove
Rit
ual
ized
movem
ent
sequen
cein
corp
ora
ting
bre
athw
ork
m
indfu
lnes
sim
ager
y
physi
cal
touch
an
dso
cial
inte
ract
ion
Yoga
ther
apy
434
4M
anag
emen
tre
duct
ion
or
elim
inat
ion
of
sym
pto
ms
that
pro
duce
suff
erin
g
enhan
cem
ent
of
funct
ion
illn
ess
pre
ven
tion
and
salu
togen
esis
Ass
essm
ent
of
per
son
aslsquolsquo
mult
idim
ensi
onal
syst
emrsquorsquo
that
incl
udes
inte
rconnec
tions
of
body
bre
ath
inte
llec
tm
ind
and
emoti
ons
Ther
apeu
tic
movem
ent
(asa
na)
and
bre
athw
ork
(pra
naya
ma)
wit
hdie
tary
m
edit
atio
n
man
tra
mudra
ch
anti
ng
ritu
al
and
self
-aw
aren
ess
life
style
pra
ctic
es
aT
his
table
pro
vid
esan
over
vie
wof
sele
cted
whole
syst
ems
par
adig
ms
studie
sfr
om
whic
har
eev
aluat
edin
this
revie
w
Itis
not
mea
nt
tobe
anex
hau
stiv
ere
pre
senta
tion
of
all
clin
ical
whole
syst
emsmdash
ther
ear
em
any
oth
ers
bW
hole
per
son
par
amet
ers
concu
rren
tly
addre
ssphysi
olo
gic
psy
cholo
gic
men
tal
emoti
onal
sp
irit
ual
so
cial
in
terg
ener
atio
nal
an
den
vir
onm
enta
lfa
ctors
aspar
tof
aholi
stic
conce
ptu
alpar
adig
m
Inoth
erw
ord
sth
ese
fact
ors
are
under
stood
asfu
ndam
enta
lly
inte
rconnec
ted
and
mutu
ally
gen
erat
ive
inre
lati
on
tohea
lth
and
wel
l-bei
ng
cT
he
term
lsquolsquopre
ven
tive
rest
ora
tive
bio
med
icin
ersquorsquo
ispro
vis
ional
lyuse
dher
eto
char
acte
rize
studie
sw
ith
ase
tof
uniq
ue
par
adig
mat
icfe
ature
sle
dby
conven
tional
med
ical
doct
ors
S24
measuresrsquorsquo before a clinical trial but also gather lsquolsquouniquephysical and psychosocial contextrsquorsquo within it and subse-quently help to lsquolsquoexplain the trial resultsrsquorsquo22
Going further WSR proponents argued that diverse researchmodesmdashprospective and retrospective experimental quasi-experimental and observational qualitative and quantitativeand holistic and reductivemdashbe equally valued for their distinctcontributions and rigorously applied as contextually appro-priate623 Asserting that EBMrsquos lsquolsquoprescriptive evidence hier-archies of research methodsrsquorsquo should be supplanted20 TCIMscholars variously conceptualized evidentiary frameworks(eg lsquolsquoevidence matrixrsquorsquo24 lsquolsquoevidence housersquorsquo25 and lsquolsquocircularmodelrsquorsquo19) in which a range of research designs might syn-thetically contribute to assessing a particular interventionrsquosefficacy effectiveness and other contextual dimensions
Taking on model validity with respect to intervention se-lection and design WSR advocates favored the evaluation oflsquolsquowhole systems or lsquobundlesrsquo of therapiesrsquorsquo rather thanlsquolsquosinglemodalitiesrsquorsquo alone6 They envisioned studies inwhich patients would undergo lsquolsquodouble classificationrsquorsquo usingbiomedical diagnostics as well as diagnosis from within therelevant TCIM paradigm and receive care that was individ-ualized on this basis6 Research teams they advised shouldinclude insiders from within the paradigms in which the in-terventions originated2627 and study recruitment strategiesshould address persons with complex multifactorial healthconditions28 as well as patient treatment preferences14
WSR leaders equally envisioned study outcome assessmentin relation to the model validity principle6 At a time whenvalidated patient-reported outcome measures (PROMs) werejust beginning to be widely used in conventional researchWSR proponents characterized subjective and paradigm-adherent quantitative outcome measures1522 as key evaluativetools alongside qualitative methods22 Measurables theyproposed should be multiple (addressing the therapeutictechniques applied patientndashpractitioner relationship and rangeof healthwellness impacts1545) and at more frequent intervalsand over a longer period than in conventional trials1623 lsquolsquoIn-novative statistical methodologyrsquorsquo6mdashincluding lsquolsquoparticipant-centeredrsquorsquo approaches46mdashwould be needed to synthesize thevoluminous data generated6 They called for lsquolsquocomplex con-ceptual modelsrsquorsquo16 to evaluate a whole systemrsquos combinedeffects lsquolsquoover and above its componentsrsquorsquo6 and variously pro-posed methodological engagement with network sciencecomplexity science and nonlinear dynamical systems2347ndash49
action research716 and program theory16 to this endSince 2003 the dominant landscape of clinical research
has transformed significantly Although the classical RCTcontinues to be prioritized in EBMrsquos evidentiary hierarchypragmatically designed comparative effectiveness studiesand lsquolsquofit-for-purposersquorsquo research designs1 have become morewidely accepted as important clinical and policy-makingresources50 Usage of PROMs clinical trial guidelines andquality assessment tools has become more widespread andlsquolsquofollowing considerable development in the fieldrsquorsquo theMedical Research Councilrsquos framework for trialing complexinterventions will once again be renewed in 20192 Withinthe TCIM world WSR principles have been increasinglytaken up4351ndash55 although more conventional research de-signs still predominate56 To date however no compre-hensive retrospective analysis of WSR advances has beenundertaken that is thus the present workrsquos aim
Methods
This article is a scoping review of the methodologicalfeatures of WSR studies with reference to the model validityprinciple Scoping reviews lsquolsquomap the literature on a particulartopic or research area and provide an opportunity to identifykey concepts gaps in the research and types and sources ofevidence to inform practice policymaking and researchrsquorsquo57
Scoping reviews lsquolsquodiffer from systematic reviews as authorsdo not typically assess the quality ofrsquorsquo58 nor lsquolsquoseek to lsquosyn-thesizersquo evidence or to aggregate findings from differentstudiesrsquorsquo59 They also diverge from lsquolsquonarrative or literaturereviews in that the scoping process requires analytical rein-terpretation of the literaturersquorsquo58 lsquolsquoNot linear but iterativersquorsquo incharacter scoping reviews primarily take a qualitative ana-lytic approach supported by numerical representation of thelsquolsquoextent nature and distributionrsquorsquo of key findings59
The present review adopts Arksey and OrsquoMalleyrsquos six-step scoping study framework involving (1) researchquestion identification (2) study identification (3) studyselection (4) data charting (5) result collation summaryand reporting and (6) (optional) consultation with areaexperts to validate findings59
Research question identification
The primary question driving this review is twofold in-terrogating (1) the range and characteristics of WSR clin-ical studies and (2) the ways in which these studies engagethe model validity principle
Study identification
WSR-type studies have been undertaken in multiple healthcare paradigms and the methodological terminology usedacross them varies Thus a broad initial keyword-based lit-erature search (eg lsquolsquowhole systems researchrsquorsquo lsquolsquocomplexrsquorsquolsquolsquoindividualizedrsquorsquo lsquolsquocomplementary medicinersquorsquo and lsquolsquomodelvalidityrsquorsquo) helped to locate many relevant methodologicalpublications but proved insufficient to identify a represen-tative set of clinical WSR exemplars A group of field experts(listed in the study acknowledgments) was therefore as-sembled by one coauthor (JW) to share WSR exemplarcitations In addition to reviewing the relevant historical lit-eratures the primary review author (NI) reviewed each ofthese recommended studies and scrutinized their referencelists for additional candidate exemplars The other coauthors( JR and CE) as WSR field experts further supplementedthis initial list As the review process progressed studyidentification through additional literature searches continuediteratively with study selection and data charting (below)
Study selection
To be eligible for inclusion studies were required to di-rectly report clinical outcomes with respect to an interven-tion based in a defined therapeutic whole system marked by aconceptual andor diagnostic model distinct from conven-tional biomedical care Studies adopting complex individu-alized salutogenic andor multimorbidmultitarget modesof care were prioritized Only peer-reviewed studies (withone or more associated publications) were included andall demonstrated a strong emphasis on model validity in atleast one of the following adopted research method(s)
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S25
intervention selection or design and outcome assessmentStudies were not required to refer directly to the model va-lidity principle nor to use WSR terminology Pilotfeasibilitydesigns were included unfulfilled study protocols were notNo attempt was made to exhaustively assemble all publishedstudies meeting study inclusion criteria rather the emphasiswas on assembling a diverse subset of such studies
Addressing a long-standing debate in the WSR field20
the multicomponent stand-alone disciplines of yoga therapyand trsquoai chi were defined as distinct whole systems despitetheir respective historical and conceptual connections tothe Ayurvedic and Chinese medicine systems Studies frommidwifery (a discipline not always included under the TCIMlsquolsquoumbrellarsquorsquo) were determined eligible for inclusion based on(1) the professionrsquos uniquely holistic woman-centered para-digm distinct from conventional obstetrics and (2) itshistorical roots in traditionalindigenous health care Stu-dies from a field provisionally termed lsquolsquopreventiverestorativebiomedicinersquorsquo were also included recognizing that (1)such studies diverge paradigmatically from conventionaltherapeutic norms and (2) that the multimodal behavior-ally focused studies led in particular by Ornish et al in the1980s6061 provided early methodological inspiration forwhole systems researchers
Study selection (ie identification charting and culling)continued iteratively until (1) lsquolsquotheoretical saturationrsquorsquo62
was reached in that review of additional candidate publi-cations failed to reveal new WSR methodological featuresand (2) a wide range of clinical whole systems paradigmswere represented within the dataset Study results were nottaken into account during the selection process
About 90 of studies recommended by at least one subjectarea expert were included and approximately two-thirds of theincluded studies had been directly recommended by at leastone WSR expert (including coauthors JR and CE) the re-mainder was identified in literature searches undertaken by theprimary author (NI) Four expert-recommended studies wereexcluded because they (1) did not meet the study inclusioncriteria (n = 1) or (2) were methodologically very similar toother selected exemplars providing little added value tothe review (n = 3) The final selection of studies deliberatelyover-represents traditional (ie Ayurvedic and Chinese)medicine systems to thoroughly address the paradigm-specificdiagnostic intervention and outcome design considerationsthat arise in these contexts
Data charting
Focused around three primary analytic categoriesmdashStudyDesign Intervention Selection and Outcome Evaluationmdashthe primary author (NI) summarized and evaluated eachcandidate study using an emergent set of tables and chartsThrough a constant comparative approach that reviewedeach study in relation to all others63 a set of analytic sub-parameters and conceptual frames progressively emergedThis process permitted a finalized study selection and adetailing of each studyrsquos distinct and nondistinct methodo-logical features
Expert validation of findings
While analysis and reporting were undertaken primarilyby the primary author (NI) a subset of categorizations
related to lsquolsquodual diagnosticsrsquorsquo and paradigm-specific out-comes was independently corroborated by another coauthor( JR) All coauthors ( JR CE JW) contributed insightsas to the emerging conceptual categories as the projectprogressed and provided input on the final analyses beforethis workrsquos peer review by other WSR field experts
Result collation summary and reporting
Results are synthetically presented and discussed in whatfollows using both narrative and graphical reporting Tofacilitate reading ease in-text WSR exemplar referencesname first authors only full citations may be found in thereference list To provide context and language to facilitatenuanced reporting of the WSR fieldrsquos features two noveltheoretical frameworks are presented below
Theory
Model validity framework
The model validity principle has been conceptualized ascentral to WSR and as noted earlier on various scholarshave suggested ways in which this principle may be enactedwithin clinical research contexts What remained implicit inmuch (although not all) of the early WSR methodologicalliterature is that WSR itself may be understood as part of anlsquolsquointegrative medicinersquorsquo movement64 geared to transformingdominant health care systems such that TCIM therapies maybe more broadly integrated alongside or as an adjunct toconventional biomedical care Clinical research is in thisscenario envisioned as a necessary but insufficient tool tohelp to dismantle barriers to integration17
Several scholars critiquing the integrative medicineproject have however suggested a potential for the distinctparadigmatic features of and practices with origins in non-biomedical therapeutic systems to be co-opted appro-priated or assimilated in such a process65 Model validityas a theoretical construct represents a commitment to ac-tively preserving these paradigms and practices in their ownright an approach aligned with the concept of a clearlylsquolsquoarticulatedrsquorsquo66 equitable medical pluralism67 rather thanan assimilative mode of integration65
What WSR pioneers were not able to fully apprise inadvance was how and to what degree future WSR methodsmight ultimately align or diverge with conventional researchstrategies in pursuit of model validity To facilitate analysisof these points in the present scoping review it is proposedthat the model validity principle be theoretically differenti-ated into three co-embedded categories as seen in Figure 1paradigm compatibility paradigm consistency and paradigmspecificity These categories are not mutually exclusive thatis a single study may concurrently include different aspects(eg method intervention design and outcome measures)marked by one or more of the identified characteristics
Paradigm compatibility model validityrsquos driving conceptis conceptualized as a category that includes two othersmdashparadigm consistency and paradigm specificitymdashthe secondof which is embedded within the first Paradigm compatibleresearch methods are those typically associated with domi-nant biomedical clinical research but which also readilylend themselves to the study of whole systems clinical in-terventions Paradigm-consistent methods differ in key
S26 IJAZ ET AL
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
1
Ch
ara
cteristics
of
Clin
ica
lW
ho
le
Sy
stem
sP
ara
dig
ms
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Ayurv
edic
med
icin
e293
0T
ypolo
gic
alas
sess
men
tof
const
ituti
on
(pra
kruti
)an
ddis
equil
ibri
um
(vik
ruti
)in
rela
tion
tow
hole
per
son
apar
amet
ers
(thre
edosh
as
kapha
va
ta
pit
ta)
met
aboli
cfu
nct
ion
(agni)
to
xin
load
(am
a)
bodil
yes
sence
s(t
ejas
oja
spra
na)
qual
itie
sdis
ease
stag
es
and
loca
tions
Nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
pal
pat
ion
puls
ean
dto
ngue
asse
ssm
ent
det
oxifi
cati
on
and
reju
ven
atio
nth
erap
ies
Die
tan
dli
fest
yle
counse
ling
her
bal
med
icin
em
anual
ther
apie
sen
emas
and
purg
atio
n
nas
altr
eatm
ents
yoga
and
med
itat
ion
bre
athin
gex
erci
ses
musi
can
dm
antr
aan
dse
lf-a
war
enes
sac
tivit
ies
Anth
roposo
phic
med
icin
e31
Bio
med
ical
asse
ssm
ent
+ty
polo
gic
alas
sess
men
tof
whole
per
son
aco
nst
ituti
on
and
dis
equil
ibri
um
inre
lati
on
tofo
ur
level
sof
form
ativ
efo
rces
(physi
cal
ether
ic
astr
al
ego)
and
thre
efold
stru
ctura
lfu
nct
ional
syst
ems
(ner
ve-
sense
m
oto
r-m
etab
oli
crh
yth
mic
)
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
Anth
roposo
phic
med
icat
ion
(hom
eopat
hic
her
bal
)die
tan
dli
fest
yle
counse
ling
art
ther
apy
rhyth
mic
alm
assa
ge
ther
apy
Eury
thm
ym
ovem
ent
ther
apy
bio
gra
phic
alco
unse
ling
psy
choth
erap
yndash
usu
albio
med
ical
care
Chin
ese
med
icin
e32
Typolo
gic
alas
sess
men
tof
whole
per
son
a
const
ituti
on
(vit
alsu
bst
ance
s)an
ddis
equil
ibri
um
(pat
hogen
icfa
ctors
st
agnat
ions)
inre
lati
on
tosi
xdiv
isio
ns
of
yin
and
yang
syst
emfu
nct
ioni
nte
ract
ion
(five
elem
ents
)st
ages
and
level
sof
dis
ease
Nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
pal
pat
ion
puls
ean
dto
ngue
asse
ssm
ent
Acu
punct
ure
m
oxib
ust
ion
her
bal
med
icin
etu
ina
mas
sage
cuppin
g
guash
asc
rapin
g
trsquoai
chi
qi
gong
die
tary
and
life
style
counse
ling
Chir
opra
ctic
med
icin
e333
4A
sses
smen
tof
bio
mec
han
ical
dis
ord
ers
bas
edon
bio
med
ical
conce
pts
of
musc
ulo
skel
etal
ner
vous
syst
ems
contr
over
sial
lyh
isto
rica
lly
conce
ptu
aliz
edin
nonbio
med
ical
term
sas
lsquolsquover
tebra
lsu
blu
xat
ionrsquo
rsquo
Bio
med
ical
dia
gnosi
s+
physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
dia
gnost
icim
agin
g
labora
tory
test
ing
Spin
alan
dso
ftti
ssue
man
ipula
tion
physi
cal
modal
itie
shom
eca
re
and
counse
ling
on
die
tex
erci
se
and
stre
ssre
duct
ion
Com
ple
men
tary
in
tegra
tive
med
icin
e35
Incl
usi
on
of
trea
tmen
tsori
gin
atin
gfr
om
ara
nge
of
whole
syst
emw
hole
pra
ctic
epar
adig
ms
wit
hin
the
ausp
ices
of
conven
tional
pre
ven
tive
bio
med
ical
hea
lth
care
del
iver
y
Bio
med
ical
dia
gnost
ics
+opti
onal
syst
em
modal
ity-s
pec
ific
dia
gnost
ics
Usu
albio
med
ical
care
plu
ssi
ngle
or
mult
iple
trea
tmen
tap
pro
aches
from
one
or
more
whole
syst
ems
whole
pra
ctic
epar
adig
ms
(incl
udin
gap
pro
aches
not
list
edher
e)
Ener
gy
med
icin
e36
Ass
essm
ent
of
whole
per
son
ben
erget
icfi
eld
Intu
itiv
een
erget
icobse
rvat
ions
Ara
nge
of
on-b
ody
(eg
hea
ling
touch
R
eiki)
and
off
-body
trea
tmen
ts
Hom
eopat
hic
med
icin
e373
8T
ypolo
gic
alas
sess
men
tof
rem
edy
signat
ure
(sim
illi
mum
)of
whole
per
son
aco
nst
ituti
on
and
dis
equil
ibri
um
poss
ibly
inre
lati
on
todis
ease
mia
sm(e
g
pso
ric
syco
tic
syphil
itic
)an
do
rkin
gdom
(pla
nt
anim
al
min
eral
)
Nar
rati
ve
case
-tak
ing
Hom
eopat
hic
dil
uti
ons
of
ara
nge
of
pla
nt
anim
al
and
min
eral
subst
ance
s
Mid
wif
ery
39
Wom
an-c
ente
red
per
inat
alca
rein
whic
hbir
this
norm
aliz
edas
ahea
lthy
even
tan
dth
em
idw
ifersquo
sro
leis
toholi
stic
ally
support
and
faci
lita
teth
ein
div
idual
wom
anrsquos
bir
thin
gch
oic
es
Bio
med
ical
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
Cas
e-lo
adbas
ed(c
onti
nuous)
as
wel
las
mid
wif
e-le
dt
eam
-bas
ed(n
onco
nti
nuous)
pre
-in
tra-
an
dpost
par
tum
bir
th-r
elat
edca
re(i
ncl
udin
gco
unse
ling
rela
ted
todie
tli
fest
yle
an
din
fant
feed
ingc
are)
wit
hopti
on
of
hom
e-or
hosp
ital
bir
th
(conti
nued
)
S23
Ta
ble
1
(Co
ntin
ued
)
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Nat
uro
pat
hic
med
icin
e40
Bio
med
ical
asse
ssm
ent
rein
terp
rete
dth
rough
aw
hole
per
son
ale
ns
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
asse
ssm
ents
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
labora
tory
and
bio
elec
tric
alte
stin
g
pal
pat
ion
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
dia
gnost
ics
Die
tphysi
cal
acti
vit
y
stre
ssm
anag
emen
tco
unse
ling
her
bal
med
icin
enutr
itio
nal
supple
men
tati
on
acupunct
ure
hom
eopat
hy
hydro
ther
apy
man
ual
ther
apie
sphysi
cal
modal
itie
san
din
stru
ctio
nin
min
db
ody
tech
niq
ues
Pre
ven
tive
rest
ora
tive
bio
med
icin
ecB
iom
edic
alas
sess
men
tw
ith
pre
ven
tive
rest
ora
tive
and
physi
olo
gic
psy
choso
cial
lens
Bio
med
ical
In
div
idual
gro
up-b
ased
beh
avio
ral
inte
rven
tions
gea
red
topre
ven
tingr
ehab
ilit
atin
gch
ronic
dis
ease
in
cludin
gdie
tnutr
ients
ex
erci
se
slee
p
stre
ssm
anag
emen
tpsy
choso
cial
support
sm
indb
ody
tech
niq
ues
ndashu
sual
bio
med
ical
care
Sw
edis
hm
assa
ge
ther
apy
41
Bio
med
ical
asse
ssm
ent
wit
han
emphas
ison
soft
tiss
ue
and
musc
ulo
skel
etal
dis
ord
ers
Bio
med
ical
dia
gnost
ics
+physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
Man
ual
ther
apy
incl
udin
gfr
icti
on
effleu
rage
pet
riss
age
vib
rati
on
tapote
men
tan
dsk
in-
roll
ing
trsquoai
chi4
2S
eeC
hin
ese
med
icin
eab
ove
See
Chin
ese
med
icin
eab
ove
Rit
ual
ized
movem
ent
sequen
cein
corp
ora
ting
bre
athw
ork
m
indfu
lnes
sim
ager
y
physi
cal
touch
an
dso
cial
inte
ract
ion
Yoga
ther
apy
434
4M
anag
emen
tre
duct
ion
or
elim
inat
ion
of
sym
pto
ms
that
pro
duce
suff
erin
g
enhan
cem
ent
of
funct
ion
illn
ess
pre
ven
tion
and
salu
togen
esis
Ass
essm
ent
of
per
son
aslsquolsquo
mult
idim
ensi
onal
syst
emrsquorsquo
that
incl
udes
inte
rconnec
tions
of
body
bre
ath
inte
llec
tm
ind
and
emoti
ons
Ther
apeu
tic
movem
ent
(asa
na)
and
bre
athw
ork
(pra
naya
ma)
wit
hdie
tary
m
edit
atio
n
man
tra
mudra
ch
anti
ng
ritu
al
and
self
-aw
aren
ess
life
style
pra
ctic
es
aT
his
table
pro
vid
esan
over
vie
wof
sele
cted
whole
syst
ems
par
adig
ms
studie
sfr
om
whic
har
eev
aluat
edin
this
revie
w
Itis
not
mea
nt
tobe
anex
hau
stiv
ere
pre
senta
tion
of
all
clin
ical
whole
syst
emsmdash
ther
ear
em
any
oth
ers
bW
hole
per
son
par
amet
ers
concu
rren
tly
addre
ssphysi
olo
gic
psy
cholo
gic
men
tal
emoti
onal
sp
irit
ual
so
cial
in
terg
ener
atio
nal
an
den
vir
onm
enta
lfa
ctors
aspar
tof
aholi
stic
conce
ptu
alpar
adig
m
Inoth
erw
ord
sth
ese
fact
ors
are
under
stood
asfu
ndam
enta
lly
inte
rconnec
ted
and
mutu
ally
gen
erat
ive
inre
lati
on
tohea
lth
and
wel
l-bei
ng
cT
he
term
lsquolsquopre
ven
tive
rest
ora
tive
bio
med
icin
ersquorsquo
ispro
vis
ional
lyuse
dher
eto
char
acte
rize
studie
sw
ith
ase
tof
uniq
ue
par
adig
mat
icfe
ature
sle
dby
conven
tional
med
ical
doct
ors
S24
measuresrsquorsquo before a clinical trial but also gather lsquolsquouniquephysical and psychosocial contextrsquorsquo within it and subse-quently help to lsquolsquoexplain the trial resultsrsquorsquo22
Going further WSR proponents argued that diverse researchmodesmdashprospective and retrospective experimental quasi-experimental and observational qualitative and quantitativeand holistic and reductivemdashbe equally valued for their distinctcontributions and rigorously applied as contextually appro-priate623 Asserting that EBMrsquos lsquolsquoprescriptive evidence hier-archies of research methodsrsquorsquo should be supplanted20 TCIMscholars variously conceptualized evidentiary frameworks(eg lsquolsquoevidence matrixrsquorsquo24 lsquolsquoevidence housersquorsquo25 and lsquolsquocircularmodelrsquorsquo19) in which a range of research designs might syn-thetically contribute to assessing a particular interventionrsquosefficacy effectiveness and other contextual dimensions
Taking on model validity with respect to intervention se-lection and design WSR advocates favored the evaluation oflsquolsquowhole systems or lsquobundlesrsquo of therapiesrsquorsquo rather thanlsquolsquosinglemodalitiesrsquorsquo alone6 They envisioned studies inwhich patients would undergo lsquolsquodouble classificationrsquorsquo usingbiomedical diagnostics as well as diagnosis from within therelevant TCIM paradigm and receive care that was individ-ualized on this basis6 Research teams they advised shouldinclude insiders from within the paradigms in which the in-terventions originated2627 and study recruitment strategiesshould address persons with complex multifactorial healthconditions28 as well as patient treatment preferences14
WSR leaders equally envisioned study outcome assessmentin relation to the model validity principle6 At a time whenvalidated patient-reported outcome measures (PROMs) werejust beginning to be widely used in conventional researchWSR proponents characterized subjective and paradigm-adherent quantitative outcome measures1522 as key evaluativetools alongside qualitative methods22 Measurables theyproposed should be multiple (addressing the therapeutictechniques applied patientndashpractitioner relationship and rangeof healthwellness impacts1545) and at more frequent intervalsand over a longer period than in conventional trials1623 lsquolsquoIn-novative statistical methodologyrsquorsquo6mdashincluding lsquolsquoparticipant-centeredrsquorsquo approaches46mdashwould be needed to synthesize thevoluminous data generated6 They called for lsquolsquocomplex con-ceptual modelsrsquorsquo16 to evaluate a whole systemrsquos combinedeffects lsquolsquoover and above its componentsrsquorsquo6 and variously pro-posed methodological engagement with network sciencecomplexity science and nonlinear dynamical systems2347ndash49
action research716 and program theory16 to this endSince 2003 the dominant landscape of clinical research
has transformed significantly Although the classical RCTcontinues to be prioritized in EBMrsquos evidentiary hierarchypragmatically designed comparative effectiveness studiesand lsquolsquofit-for-purposersquorsquo research designs1 have become morewidely accepted as important clinical and policy-makingresources50 Usage of PROMs clinical trial guidelines andquality assessment tools has become more widespread andlsquolsquofollowing considerable development in the fieldrsquorsquo theMedical Research Councilrsquos framework for trialing complexinterventions will once again be renewed in 20192 Withinthe TCIM world WSR principles have been increasinglytaken up4351ndash55 although more conventional research de-signs still predominate56 To date however no compre-hensive retrospective analysis of WSR advances has beenundertaken that is thus the present workrsquos aim
Methods
This article is a scoping review of the methodologicalfeatures of WSR studies with reference to the model validityprinciple Scoping reviews lsquolsquomap the literature on a particulartopic or research area and provide an opportunity to identifykey concepts gaps in the research and types and sources ofevidence to inform practice policymaking and researchrsquorsquo57
Scoping reviews lsquolsquodiffer from systematic reviews as authorsdo not typically assess the quality ofrsquorsquo58 nor lsquolsquoseek to lsquosyn-thesizersquo evidence or to aggregate findings from differentstudiesrsquorsquo59 They also diverge from lsquolsquonarrative or literaturereviews in that the scoping process requires analytical rein-terpretation of the literaturersquorsquo58 lsquolsquoNot linear but iterativersquorsquo incharacter scoping reviews primarily take a qualitative ana-lytic approach supported by numerical representation of thelsquolsquoextent nature and distributionrsquorsquo of key findings59
The present review adopts Arksey and OrsquoMalleyrsquos six-step scoping study framework involving (1) researchquestion identification (2) study identification (3) studyselection (4) data charting (5) result collation summaryand reporting and (6) (optional) consultation with areaexperts to validate findings59
Research question identification
The primary question driving this review is twofold in-terrogating (1) the range and characteristics of WSR clin-ical studies and (2) the ways in which these studies engagethe model validity principle
Study identification
WSR-type studies have been undertaken in multiple healthcare paradigms and the methodological terminology usedacross them varies Thus a broad initial keyword-based lit-erature search (eg lsquolsquowhole systems researchrsquorsquo lsquolsquocomplexrsquorsquolsquolsquoindividualizedrsquorsquo lsquolsquocomplementary medicinersquorsquo and lsquolsquomodelvalidityrsquorsquo) helped to locate many relevant methodologicalpublications but proved insufficient to identify a represen-tative set of clinical WSR exemplars A group of field experts(listed in the study acknowledgments) was therefore as-sembled by one coauthor (JW) to share WSR exemplarcitations In addition to reviewing the relevant historical lit-eratures the primary review author (NI) reviewed each ofthese recommended studies and scrutinized their referencelists for additional candidate exemplars The other coauthors( JR and CE) as WSR field experts further supplementedthis initial list As the review process progressed studyidentification through additional literature searches continuediteratively with study selection and data charting (below)
Study selection
To be eligible for inclusion studies were required to di-rectly report clinical outcomes with respect to an interven-tion based in a defined therapeutic whole system marked by aconceptual andor diagnostic model distinct from conven-tional biomedical care Studies adopting complex individu-alized salutogenic andor multimorbidmultitarget modesof care were prioritized Only peer-reviewed studies (withone or more associated publications) were included andall demonstrated a strong emphasis on model validity in atleast one of the following adopted research method(s)
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S25
intervention selection or design and outcome assessmentStudies were not required to refer directly to the model va-lidity principle nor to use WSR terminology Pilotfeasibilitydesigns were included unfulfilled study protocols were notNo attempt was made to exhaustively assemble all publishedstudies meeting study inclusion criteria rather the emphasiswas on assembling a diverse subset of such studies
Addressing a long-standing debate in the WSR field20
the multicomponent stand-alone disciplines of yoga therapyand trsquoai chi were defined as distinct whole systems despitetheir respective historical and conceptual connections tothe Ayurvedic and Chinese medicine systems Studies frommidwifery (a discipline not always included under the TCIMlsquolsquoumbrellarsquorsquo) were determined eligible for inclusion based on(1) the professionrsquos uniquely holistic woman-centered para-digm distinct from conventional obstetrics and (2) itshistorical roots in traditionalindigenous health care Stu-dies from a field provisionally termed lsquolsquopreventiverestorativebiomedicinersquorsquo were also included recognizing that (1)such studies diverge paradigmatically from conventionaltherapeutic norms and (2) that the multimodal behavior-ally focused studies led in particular by Ornish et al in the1980s6061 provided early methodological inspiration forwhole systems researchers
Study selection (ie identification charting and culling)continued iteratively until (1) lsquolsquotheoretical saturationrsquorsquo62
was reached in that review of additional candidate publi-cations failed to reveal new WSR methodological featuresand (2) a wide range of clinical whole systems paradigmswere represented within the dataset Study results were nottaken into account during the selection process
About 90 of studies recommended by at least one subjectarea expert were included and approximately two-thirds of theincluded studies had been directly recommended by at leastone WSR expert (including coauthors JR and CE) the re-mainder was identified in literature searches undertaken by theprimary author (NI) Four expert-recommended studies wereexcluded because they (1) did not meet the study inclusioncriteria (n = 1) or (2) were methodologically very similar toother selected exemplars providing little added value tothe review (n = 3) The final selection of studies deliberatelyover-represents traditional (ie Ayurvedic and Chinese)medicine systems to thoroughly address the paradigm-specificdiagnostic intervention and outcome design considerationsthat arise in these contexts
Data charting
Focused around three primary analytic categoriesmdashStudyDesign Intervention Selection and Outcome Evaluationmdashthe primary author (NI) summarized and evaluated eachcandidate study using an emergent set of tables and chartsThrough a constant comparative approach that reviewedeach study in relation to all others63 a set of analytic sub-parameters and conceptual frames progressively emergedThis process permitted a finalized study selection and adetailing of each studyrsquos distinct and nondistinct methodo-logical features
Expert validation of findings
While analysis and reporting were undertaken primarilyby the primary author (NI) a subset of categorizations
related to lsquolsquodual diagnosticsrsquorsquo and paradigm-specific out-comes was independently corroborated by another coauthor( JR) All coauthors ( JR CE JW) contributed insightsas to the emerging conceptual categories as the projectprogressed and provided input on the final analyses beforethis workrsquos peer review by other WSR field experts
Result collation summary and reporting
Results are synthetically presented and discussed in whatfollows using both narrative and graphical reporting Tofacilitate reading ease in-text WSR exemplar referencesname first authors only full citations may be found in thereference list To provide context and language to facilitatenuanced reporting of the WSR fieldrsquos features two noveltheoretical frameworks are presented below
Theory
Model validity framework
The model validity principle has been conceptualized ascentral to WSR and as noted earlier on various scholarshave suggested ways in which this principle may be enactedwithin clinical research contexts What remained implicit inmuch (although not all) of the early WSR methodologicalliterature is that WSR itself may be understood as part of anlsquolsquointegrative medicinersquorsquo movement64 geared to transformingdominant health care systems such that TCIM therapies maybe more broadly integrated alongside or as an adjunct toconventional biomedical care Clinical research is in thisscenario envisioned as a necessary but insufficient tool tohelp to dismantle barriers to integration17
Several scholars critiquing the integrative medicineproject have however suggested a potential for the distinctparadigmatic features of and practices with origins in non-biomedical therapeutic systems to be co-opted appro-priated or assimilated in such a process65 Model validityas a theoretical construct represents a commitment to ac-tively preserving these paradigms and practices in their ownright an approach aligned with the concept of a clearlylsquolsquoarticulatedrsquorsquo66 equitable medical pluralism67 rather thanan assimilative mode of integration65
What WSR pioneers were not able to fully apprise inadvance was how and to what degree future WSR methodsmight ultimately align or diverge with conventional researchstrategies in pursuit of model validity To facilitate analysisof these points in the present scoping review it is proposedthat the model validity principle be theoretically differenti-ated into three co-embedded categories as seen in Figure 1paradigm compatibility paradigm consistency and paradigmspecificity These categories are not mutually exclusive thatis a single study may concurrently include different aspects(eg method intervention design and outcome measures)marked by one or more of the identified characteristics
Paradigm compatibility model validityrsquos driving conceptis conceptualized as a category that includes two othersmdashparadigm consistency and paradigm specificitymdashthe secondof which is embedded within the first Paradigm compatibleresearch methods are those typically associated with domi-nant biomedical clinical research but which also readilylend themselves to the study of whole systems clinical in-terventions Paradigm-consistent methods differ in key
S26 IJAZ ET AL
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
1
(Co
ntin
ued
)
Para
dig
mC
once
ptu
al
model
Dia
gnost
ics
Tre
atm
ent
modes
Nat
uro
pat
hic
med
icin
e40
Bio
med
ical
asse
ssm
ent
rein
terp
rete
dth
rough
aw
hole
per
son
ale
ns
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
asse
ssm
ents
Bio
med
ical
dia
gnost
ics
+ad
dit
ional
nar
rati
ve
case
-tak
ing
physi
cal
exam
inat
ion
labora
tory
and
bio
elec
tric
alte
stin
g
pal
pat
ion
+opti
onal
Chin
ese
med
icin
ehom
eopat
hic
dia
gnost
ics
Die
tphysi
cal
acti
vit
y
stre
ssm
anag
emen
tco
unse
ling
her
bal
med
icin
enutr
itio
nal
supple
men
tati
on
acupunct
ure
hom
eopat
hy
hydro
ther
apy
man
ual
ther
apie
sphysi
cal
modal
itie
san
din
stru
ctio
nin
min
db
ody
tech
niq
ues
Pre
ven
tive
rest
ora
tive
bio
med
icin
ecB
iom
edic
alas
sess
men
tw
ith
pre
ven
tive
rest
ora
tive
and
physi
olo
gic
psy
choso
cial
lens
Bio
med
ical
In
div
idual
gro
up-b
ased
beh
avio
ral
inte
rven
tions
gea
red
topre
ven
tingr
ehab
ilit
atin
gch
ronic
dis
ease
in
cludin
gdie
tnutr
ients
ex
erci
se
slee
p
stre
ssm
anag
emen
tpsy
choso
cial
support
sm
indb
ody
tech
niq
ues
ndashu
sual
bio
med
ical
care
Sw
edis
hm
assa
ge
ther
apy
41
Bio
med
ical
asse
ssm
ent
wit
han
emphas
ison
soft
tiss
ue
and
musc
ulo
skel
etal
dis
ord
ers
Bio
med
ical
dia
gnost
ics
+physi
cal
exam
inat
ion
pal
pat
ion
funct
ional
asse
ssm
ent
Man
ual
ther
apy
incl
udin
gfr
icti
on
effleu
rage
pet
riss
age
vib
rati
on
tapote
men
tan
dsk
in-
roll
ing
trsquoai
chi4
2S
eeC
hin
ese
med
icin
eab
ove
See
Chin
ese
med
icin
eab
ove
Rit
ual
ized
movem
ent
sequen
cein
corp
ora
ting
bre
athw
ork
m
indfu
lnes
sim
ager
y
physi
cal
touch
an
dso
cial
inte
ract
ion
Yoga
ther
apy
434
4M
anag
emen
tre
duct
ion
or
elim
inat
ion
of
sym
pto
ms
that
pro
duce
suff
erin
g
enhan
cem
ent
of
funct
ion
illn
ess
pre
ven
tion
and
salu
togen
esis
Ass
essm
ent
of
per
son
aslsquolsquo
mult
idim
ensi
onal
syst
emrsquorsquo
that
incl
udes
inte
rconnec
tions
of
body
bre
ath
inte
llec
tm
ind
and
emoti
ons
Ther
apeu
tic
movem
ent
(asa
na)
and
bre
athw
ork
(pra
naya
ma)
wit
hdie
tary
m
edit
atio
n
man
tra
mudra
ch
anti
ng
ritu
al
and
self
-aw
aren
ess
life
style
pra
ctic
es
aT
his
table
pro
vid
esan
over
vie
wof
sele
cted
whole
syst
ems
par
adig
ms
studie
sfr
om
whic
har
eev
aluat
edin
this
revie
w
Itis
not
mea
nt
tobe
anex
hau
stiv
ere
pre
senta
tion
of
all
clin
ical
whole
syst
emsmdash
ther
ear
em
any
oth
ers
bW
hole
per
son
par
amet
ers
concu
rren
tly
addre
ssphysi
olo
gic
psy
cholo
gic
men
tal
emoti
onal
sp
irit
ual
so
cial
in
terg
ener
atio
nal
an
den
vir
onm
enta
lfa
ctors
aspar
tof
aholi
stic
conce
ptu
alpar
adig
m
Inoth
erw
ord
sth
ese
fact
ors
are
under
stood
asfu
ndam
enta
lly
inte
rconnec
ted
and
mutu
ally
gen
erat
ive
inre
lati
on
tohea
lth
and
wel
l-bei
ng
cT
he
term
lsquolsquopre
ven
tive
rest
ora
tive
bio
med
icin
ersquorsquo
ispro
vis
ional
lyuse
dher
eto
char
acte
rize
studie
sw
ith
ase
tof
uniq
ue
par
adig
mat
icfe
ature
sle
dby
conven
tional
med
ical
doct
ors
S24
measuresrsquorsquo before a clinical trial but also gather lsquolsquouniquephysical and psychosocial contextrsquorsquo within it and subse-quently help to lsquolsquoexplain the trial resultsrsquorsquo22
Going further WSR proponents argued that diverse researchmodesmdashprospective and retrospective experimental quasi-experimental and observational qualitative and quantitativeand holistic and reductivemdashbe equally valued for their distinctcontributions and rigorously applied as contextually appro-priate623 Asserting that EBMrsquos lsquolsquoprescriptive evidence hier-archies of research methodsrsquorsquo should be supplanted20 TCIMscholars variously conceptualized evidentiary frameworks(eg lsquolsquoevidence matrixrsquorsquo24 lsquolsquoevidence housersquorsquo25 and lsquolsquocircularmodelrsquorsquo19) in which a range of research designs might syn-thetically contribute to assessing a particular interventionrsquosefficacy effectiveness and other contextual dimensions
Taking on model validity with respect to intervention se-lection and design WSR advocates favored the evaluation oflsquolsquowhole systems or lsquobundlesrsquo of therapiesrsquorsquo rather thanlsquolsquosinglemodalitiesrsquorsquo alone6 They envisioned studies inwhich patients would undergo lsquolsquodouble classificationrsquorsquo usingbiomedical diagnostics as well as diagnosis from within therelevant TCIM paradigm and receive care that was individ-ualized on this basis6 Research teams they advised shouldinclude insiders from within the paradigms in which the in-terventions originated2627 and study recruitment strategiesshould address persons with complex multifactorial healthconditions28 as well as patient treatment preferences14
WSR leaders equally envisioned study outcome assessmentin relation to the model validity principle6 At a time whenvalidated patient-reported outcome measures (PROMs) werejust beginning to be widely used in conventional researchWSR proponents characterized subjective and paradigm-adherent quantitative outcome measures1522 as key evaluativetools alongside qualitative methods22 Measurables theyproposed should be multiple (addressing the therapeutictechniques applied patientndashpractitioner relationship and rangeof healthwellness impacts1545) and at more frequent intervalsand over a longer period than in conventional trials1623 lsquolsquoIn-novative statistical methodologyrsquorsquo6mdashincluding lsquolsquoparticipant-centeredrsquorsquo approaches46mdashwould be needed to synthesize thevoluminous data generated6 They called for lsquolsquocomplex con-ceptual modelsrsquorsquo16 to evaluate a whole systemrsquos combinedeffects lsquolsquoover and above its componentsrsquorsquo6 and variously pro-posed methodological engagement with network sciencecomplexity science and nonlinear dynamical systems2347ndash49
action research716 and program theory16 to this endSince 2003 the dominant landscape of clinical research
has transformed significantly Although the classical RCTcontinues to be prioritized in EBMrsquos evidentiary hierarchypragmatically designed comparative effectiveness studiesand lsquolsquofit-for-purposersquorsquo research designs1 have become morewidely accepted as important clinical and policy-makingresources50 Usage of PROMs clinical trial guidelines andquality assessment tools has become more widespread andlsquolsquofollowing considerable development in the fieldrsquorsquo theMedical Research Councilrsquos framework for trialing complexinterventions will once again be renewed in 20192 Withinthe TCIM world WSR principles have been increasinglytaken up4351ndash55 although more conventional research de-signs still predominate56 To date however no compre-hensive retrospective analysis of WSR advances has beenundertaken that is thus the present workrsquos aim
Methods
This article is a scoping review of the methodologicalfeatures of WSR studies with reference to the model validityprinciple Scoping reviews lsquolsquomap the literature on a particulartopic or research area and provide an opportunity to identifykey concepts gaps in the research and types and sources ofevidence to inform practice policymaking and researchrsquorsquo57
Scoping reviews lsquolsquodiffer from systematic reviews as authorsdo not typically assess the quality ofrsquorsquo58 nor lsquolsquoseek to lsquosyn-thesizersquo evidence or to aggregate findings from differentstudiesrsquorsquo59 They also diverge from lsquolsquonarrative or literaturereviews in that the scoping process requires analytical rein-terpretation of the literaturersquorsquo58 lsquolsquoNot linear but iterativersquorsquo incharacter scoping reviews primarily take a qualitative ana-lytic approach supported by numerical representation of thelsquolsquoextent nature and distributionrsquorsquo of key findings59
The present review adopts Arksey and OrsquoMalleyrsquos six-step scoping study framework involving (1) researchquestion identification (2) study identification (3) studyselection (4) data charting (5) result collation summaryand reporting and (6) (optional) consultation with areaexperts to validate findings59
Research question identification
The primary question driving this review is twofold in-terrogating (1) the range and characteristics of WSR clin-ical studies and (2) the ways in which these studies engagethe model validity principle
Study identification
WSR-type studies have been undertaken in multiple healthcare paradigms and the methodological terminology usedacross them varies Thus a broad initial keyword-based lit-erature search (eg lsquolsquowhole systems researchrsquorsquo lsquolsquocomplexrsquorsquolsquolsquoindividualizedrsquorsquo lsquolsquocomplementary medicinersquorsquo and lsquolsquomodelvalidityrsquorsquo) helped to locate many relevant methodologicalpublications but proved insufficient to identify a represen-tative set of clinical WSR exemplars A group of field experts(listed in the study acknowledgments) was therefore as-sembled by one coauthor (JW) to share WSR exemplarcitations In addition to reviewing the relevant historical lit-eratures the primary review author (NI) reviewed each ofthese recommended studies and scrutinized their referencelists for additional candidate exemplars The other coauthors( JR and CE) as WSR field experts further supplementedthis initial list As the review process progressed studyidentification through additional literature searches continuediteratively with study selection and data charting (below)
Study selection
To be eligible for inclusion studies were required to di-rectly report clinical outcomes with respect to an interven-tion based in a defined therapeutic whole system marked by aconceptual andor diagnostic model distinct from conven-tional biomedical care Studies adopting complex individu-alized salutogenic andor multimorbidmultitarget modesof care were prioritized Only peer-reviewed studies (withone or more associated publications) were included andall demonstrated a strong emphasis on model validity in atleast one of the following adopted research method(s)
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S25
intervention selection or design and outcome assessmentStudies were not required to refer directly to the model va-lidity principle nor to use WSR terminology Pilotfeasibilitydesigns were included unfulfilled study protocols were notNo attempt was made to exhaustively assemble all publishedstudies meeting study inclusion criteria rather the emphasiswas on assembling a diverse subset of such studies
Addressing a long-standing debate in the WSR field20
the multicomponent stand-alone disciplines of yoga therapyand trsquoai chi were defined as distinct whole systems despitetheir respective historical and conceptual connections tothe Ayurvedic and Chinese medicine systems Studies frommidwifery (a discipline not always included under the TCIMlsquolsquoumbrellarsquorsquo) were determined eligible for inclusion based on(1) the professionrsquos uniquely holistic woman-centered para-digm distinct from conventional obstetrics and (2) itshistorical roots in traditionalindigenous health care Stu-dies from a field provisionally termed lsquolsquopreventiverestorativebiomedicinersquorsquo were also included recognizing that (1)such studies diverge paradigmatically from conventionaltherapeutic norms and (2) that the multimodal behavior-ally focused studies led in particular by Ornish et al in the1980s6061 provided early methodological inspiration forwhole systems researchers
Study selection (ie identification charting and culling)continued iteratively until (1) lsquolsquotheoretical saturationrsquorsquo62
was reached in that review of additional candidate publi-cations failed to reveal new WSR methodological featuresand (2) a wide range of clinical whole systems paradigmswere represented within the dataset Study results were nottaken into account during the selection process
About 90 of studies recommended by at least one subjectarea expert were included and approximately two-thirds of theincluded studies had been directly recommended by at leastone WSR expert (including coauthors JR and CE) the re-mainder was identified in literature searches undertaken by theprimary author (NI) Four expert-recommended studies wereexcluded because they (1) did not meet the study inclusioncriteria (n = 1) or (2) were methodologically very similar toother selected exemplars providing little added value tothe review (n = 3) The final selection of studies deliberatelyover-represents traditional (ie Ayurvedic and Chinese)medicine systems to thoroughly address the paradigm-specificdiagnostic intervention and outcome design considerationsthat arise in these contexts
Data charting
Focused around three primary analytic categoriesmdashStudyDesign Intervention Selection and Outcome Evaluationmdashthe primary author (NI) summarized and evaluated eachcandidate study using an emergent set of tables and chartsThrough a constant comparative approach that reviewedeach study in relation to all others63 a set of analytic sub-parameters and conceptual frames progressively emergedThis process permitted a finalized study selection and adetailing of each studyrsquos distinct and nondistinct methodo-logical features
Expert validation of findings
While analysis and reporting were undertaken primarilyby the primary author (NI) a subset of categorizations
related to lsquolsquodual diagnosticsrsquorsquo and paradigm-specific out-comes was independently corroborated by another coauthor( JR) All coauthors ( JR CE JW) contributed insightsas to the emerging conceptual categories as the projectprogressed and provided input on the final analyses beforethis workrsquos peer review by other WSR field experts
Result collation summary and reporting
Results are synthetically presented and discussed in whatfollows using both narrative and graphical reporting Tofacilitate reading ease in-text WSR exemplar referencesname first authors only full citations may be found in thereference list To provide context and language to facilitatenuanced reporting of the WSR fieldrsquos features two noveltheoretical frameworks are presented below
Theory
Model validity framework
The model validity principle has been conceptualized ascentral to WSR and as noted earlier on various scholarshave suggested ways in which this principle may be enactedwithin clinical research contexts What remained implicit inmuch (although not all) of the early WSR methodologicalliterature is that WSR itself may be understood as part of anlsquolsquointegrative medicinersquorsquo movement64 geared to transformingdominant health care systems such that TCIM therapies maybe more broadly integrated alongside or as an adjunct toconventional biomedical care Clinical research is in thisscenario envisioned as a necessary but insufficient tool tohelp to dismantle barriers to integration17
Several scholars critiquing the integrative medicineproject have however suggested a potential for the distinctparadigmatic features of and practices with origins in non-biomedical therapeutic systems to be co-opted appro-priated or assimilated in such a process65 Model validityas a theoretical construct represents a commitment to ac-tively preserving these paradigms and practices in their ownright an approach aligned with the concept of a clearlylsquolsquoarticulatedrsquorsquo66 equitable medical pluralism67 rather thanan assimilative mode of integration65
What WSR pioneers were not able to fully apprise inadvance was how and to what degree future WSR methodsmight ultimately align or diverge with conventional researchstrategies in pursuit of model validity To facilitate analysisof these points in the present scoping review it is proposedthat the model validity principle be theoretically differenti-ated into three co-embedded categories as seen in Figure 1paradigm compatibility paradigm consistency and paradigmspecificity These categories are not mutually exclusive thatis a single study may concurrently include different aspects(eg method intervention design and outcome measures)marked by one or more of the identified characteristics
Paradigm compatibility model validityrsquos driving conceptis conceptualized as a category that includes two othersmdashparadigm consistency and paradigm specificitymdashthe secondof which is embedded within the first Paradigm compatibleresearch methods are those typically associated with domi-nant biomedical clinical research but which also readilylend themselves to the study of whole systems clinical in-terventions Paradigm-consistent methods differ in key
S26 IJAZ ET AL
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
measuresrsquorsquo before a clinical trial but also gather lsquolsquouniquephysical and psychosocial contextrsquorsquo within it and subse-quently help to lsquolsquoexplain the trial resultsrsquorsquo22
Going further WSR proponents argued that diverse researchmodesmdashprospective and retrospective experimental quasi-experimental and observational qualitative and quantitativeand holistic and reductivemdashbe equally valued for their distinctcontributions and rigorously applied as contextually appro-priate623 Asserting that EBMrsquos lsquolsquoprescriptive evidence hier-archies of research methodsrsquorsquo should be supplanted20 TCIMscholars variously conceptualized evidentiary frameworks(eg lsquolsquoevidence matrixrsquorsquo24 lsquolsquoevidence housersquorsquo25 and lsquolsquocircularmodelrsquorsquo19) in which a range of research designs might syn-thetically contribute to assessing a particular interventionrsquosefficacy effectiveness and other contextual dimensions
Taking on model validity with respect to intervention se-lection and design WSR advocates favored the evaluation oflsquolsquowhole systems or lsquobundlesrsquo of therapiesrsquorsquo rather thanlsquolsquosinglemodalitiesrsquorsquo alone6 They envisioned studies inwhich patients would undergo lsquolsquodouble classificationrsquorsquo usingbiomedical diagnostics as well as diagnosis from within therelevant TCIM paradigm and receive care that was individ-ualized on this basis6 Research teams they advised shouldinclude insiders from within the paradigms in which the in-terventions originated2627 and study recruitment strategiesshould address persons with complex multifactorial healthconditions28 as well as patient treatment preferences14
WSR leaders equally envisioned study outcome assessmentin relation to the model validity principle6 At a time whenvalidated patient-reported outcome measures (PROMs) werejust beginning to be widely used in conventional researchWSR proponents characterized subjective and paradigm-adherent quantitative outcome measures1522 as key evaluativetools alongside qualitative methods22 Measurables theyproposed should be multiple (addressing the therapeutictechniques applied patientndashpractitioner relationship and rangeof healthwellness impacts1545) and at more frequent intervalsand over a longer period than in conventional trials1623 lsquolsquoIn-novative statistical methodologyrsquorsquo6mdashincluding lsquolsquoparticipant-centeredrsquorsquo approaches46mdashwould be needed to synthesize thevoluminous data generated6 They called for lsquolsquocomplex con-ceptual modelsrsquorsquo16 to evaluate a whole systemrsquos combinedeffects lsquolsquoover and above its componentsrsquorsquo6 and variously pro-posed methodological engagement with network sciencecomplexity science and nonlinear dynamical systems2347ndash49
action research716 and program theory16 to this endSince 2003 the dominant landscape of clinical research
has transformed significantly Although the classical RCTcontinues to be prioritized in EBMrsquos evidentiary hierarchypragmatically designed comparative effectiveness studiesand lsquolsquofit-for-purposersquorsquo research designs1 have become morewidely accepted as important clinical and policy-makingresources50 Usage of PROMs clinical trial guidelines andquality assessment tools has become more widespread andlsquolsquofollowing considerable development in the fieldrsquorsquo theMedical Research Councilrsquos framework for trialing complexinterventions will once again be renewed in 20192 Withinthe TCIM world WSR principles have been increasinglytaken up4351ndash55 although more conventional research de-signs still predominate56 To date however no compre-hensive retrospective analysis of WSR advances has beenundertaken that is thus the present workrsquos aim
Methods
This article is a scoping review of the methodologicalfeatures of WSR studies with reference to the model validityprinciple Scoping reviews lsquolsquomap the literature on a particulartopic or research area and provide an opportunity to identifykey concepts gaps in the research and types and sources ofevidence to inform practice policymaking and researchrsquorsquo57
Scoping reviews lsquolsquodiffer from systematic reviews as authorsdo not typically assess the quality ofrsquorsquo58 nor lsquolsquoseek to lsquosyn-thesizersquo evidence or to aggregate findings from differentstudiesrsquorsquo59 They also diverge from lsquolsquonarrative or literaturereviews in that the scoping process requires analytical rein-terpretation of the literaturersquorsquo58 lsquolsquoNot linear but iterativersquorsquo incharacter scoping reviews primarily take a qualitative ana-lytic approach supported by numerical representation of thelsquolsquoextent nature and distributionrsquorsquo of key findings59
The present review adopts Arksey and OrsquoMalleyrsquos six-step scoping study framework involving (1) researchquestion identification (2) study identification (3) studyselection (4) data charting (5) result collation summaryand reporting and (6) (optional) consultation with areaexperts to validate findings59
Research question identification
The primary question driving this review is twofold in-terrogating (1) the range and characteristics of WSR clin-ical studies and (2) the ways in which these studies engagethe model validity principle
Study identification
WSR-type studies have been undertaken in multiple healthcare paradigms and the methodological terminology usedacross them varies Thus a broad initial keyword-based lit-erature search (eg lsquolsquowhole systems researchrsquorsquo lsquolsquocomplexrsquorsquolsquolsquoindividualizedrsquorsquo lsquolsquocomplementary medicinersquorsquo and lsquolsquomodelvalidityrsquorsquo) helped to locate many relevant methodologicalpublications but proved insufficient to identify a represen-tative set of clinical WSR exemplars A group of field experts(listed in the study acknowledgments) was therefore as-sembled by one coauthor (JW) to share WSR exemplarcitations In addition to reviewing the relevant historical lit-eratures the primary review author (NI) reviewed each ofthese recommended studies and scrutinized their referencelists for additional candidate exemplars The other coauthors( JR and CE) as WSR field experts further supplementedthis initial list As the review process progressed studyidentification through additional literature searches continuediteratively with study selection and data charting (below)
Study selection
To be eligible for inclusion studies were required to di-rectly report clinical outcomes with respect to an interven-tion based in a defined therapeutic whole system marked by aconceptual andor diagnostic model distinct from conven-tional biomedical care Studies adopting complex individu-alized salutogenic andor multimorbidmultitarget modesof care were prioritized Only peer-reviewed studies (withone or more associated publications) were included andall demonstrated a strong emphasis on model validity in atleast one of the following adopted research method(s)
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S25
intervention selection or design and outcome assessmentStudies were not required to refer directly to the model va-lidity principle nor to use WSR terminology Pilotfeasibilitydesigns were included unfulfilled study protocols were notNo attempt was made to exhaustively assemble all publishedstudies meeting study inclusion criteria rather the emphasiswas on assembling a diverse subset of such studies
Addressing a long-standing debate in the WSR field20
the multicomponent stand-alone disciplines of yoga therapyand trsquoai chi were defined as distinct whole systems despitetheir respective historical and conceptual connections tothe Ayurvedic and Chinese medicine systems Studies frommidwifery (a discipline not always included under the TCIMlsquolsquoumbrellarsquorsquo) were determined eligible for inclusion based on(1) the professionrsquos uniquely holistic woman-centered para-digm distinct from conventional obstetrics and (2) itshistorical roots in traditionalindigenous health care Stu-dies from a field provisionally termed lsquolsquopreventiverestorativebiomedicinersquorsquo were also included recognizing that (1)such studies diverge paradigmatically from conventionaltherapeutic norms and (2) that the multimodal behavior-ally focused studies led in particular by Ornish et al in the1980s6061 provided early methodological inspiration forwhole systems researchers
Study selection (ie identification charting and culling)continued iteratively until (1) lsquolsquotheoretical saturationrsquorsquo62
was reached in that review of additional candidate publi-cations failed to reveal new WSR methodological featuresand (2) a wide range of clinical whole systems paradigmswere represented within the dataset Study results were nottaken into account during the selection process
About 90 of studies recommended by at least one subjectarea expert were included and approximately two-thirds of theincluded studies had been directly recommended by at leastone WSR expert (including coauthors JR and CE) the re-mainder was identified in literature searches undertaken by theprimary author (NI) Four expert-recommended studies wereexcluded because they (1) did not meet the study inclusioncriteria (n = 1) or (2) were methodologically very similar toother selected exemplars providing little added value tothe review (n = 3) The final selection of studies deliberatelyover-represents traditional (ie Ayurvedic and Chinese)medicine systems to thoroughly address the paradigm-specificdiagnostic intervention and outcome design considerationsthat arise in these contexts
Data charting
Focused around three primary analytic categoriesmdashStudyDesign Intervention Selection and Outcome Evaluationmdashthe primary author (NI) summarized and evaluated eachcandidate study using an emergent set of tables and chartsThrough a constant comparative approach that reviewedeach study in relation to all others63 a set of analytic sub-parameters and conceptual frames progressively emergedThis process permitted a finalized study selection and adetailing of each studyrsquos distinct and nondistinct methodo-logical features
Expert validation of findings
While analysis and reporting were undertaken primarilyby the primary author (NI) a subset of categorizations
related to lsquolsquodual diagnosticsrsquorsquo and paradigm-specific out-comes was independently corroborated by another coauthor( JR) All coauthors ( JR CE JW) contributed insightsas to the emerging conceptual categories as the projectprogressed and provided input on the final analyses beforethis workrsquos peer review by other WSR field experts
Result collation summary and reporting
Results are synthetically presented and discussed in whatfollows using both narrative and graphical reporting Tofacilitate reading ease in-text WSR exemplar referencesname first authors only full citations may be found in thereference list To provide context and language to facilitatenuanced reporting of the WSR fieldrsquos features two noveltheoretical frameworks are presented below
Theory
Model validity framework
The model validity principle has been conceptualized ascentral to WSR and as noted earlier on various scholarshave suggested ways in which this principle may be enactedwithin clinical research contexts What remained implicit inmuch (although not all) of the early WSR methodologicalliterature is that WSR itself may be understood as part of anlsquolsquointegrative medicinersquorsquo movement64 geared to transformingdominant health care systems such that TCIM therapies maybe more broadly integrated alongside or as an adjunct toconventional biomedical care Clinical research is in thisscenario envisioned as a necessary but insufficient tool tohelp to dismantle barriers to integration17
Several scholars critiquing the integrative medicineproject have however suggested a potential for the distinctparadigmatic features of and practices with origins in non-biomedical therapeutic systems to be co-opted appro-priated or assimilated in such a process65 Model validityas a theoretical construct represents a commitment to ac-tively preserving these paradigms and practices in their ownright an approach aligned with the concept of a clearlylsquolsquoarticulatedrsquorsquo66 equitable medical pluralism67 rather thanan assimilative mode of integration65
What WSR pioneers were not able to fully apprise inadvance was how and to what degree future WSR methodsmight ultimately align or diverge with conventional researchstrategies in pursuit of model validity To facilitate analysisof these points in the present scoping review it is proposedthat the model validity principle be theoretically differenti-ated into three co-embedded categories as seen in Figure 1paradigm compatibility paradigm consistency and paradigmspecificity These categories are not mutually exclusive thatis a single study may concurrently include different aspects(eg method intervention design and outcome measures)marked by one or more of the identified characteristics
Paradigm compatibility model validityrsquos driving conceptis conceptualized as a category that includes two othersmdashparadigm consistency and paradigm specificitymdashthe secondof which is embedded within the first Paradigm compatibleresearch methods are those typically associated with domi-nant biomedical clinical research but which also readilylend themselves to the study of whole systems clinical in-terventions Paradigm-consistent methods differ in key
S26 IJAZ ET AL
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
intervention selection or design and outcome assessmentStudies were not required to refer directly to the model va-lidity principle nor to use WSR terminology Pilotfeasibilitydesigns were included unfulfilled study protocols were notNo attempt was made to exhaustively assemble all publishedstudies meeting study inclusion criteria rather the emphasiswas on assembling a diverse subset of such studies
Addressing a long-standing debate in the WSR field20
the multicomponent stand-alone disciplines of yoga therapyand trsquoai chi were defined as distinct whole systems despitetheir respective historical and conceptual connections tothe Ayurvedic and Chinese medicine systems Studies frommidwifery (a discipline not always included under the TCIMlsquolsquoumbrellarsquorsquo) were determined eligible for inclusion based on(1) the professionrsquos uniquely holistic woman-centered para-digm distinct from conventional obstetrics and (2) itshistorical roots in traditionalindigenous health care Stu-dies from a field provisionally termed lsquolsquopreventiverestorativebiomedicinersquorsquo were also included recognizing that (1)such studies diverge paradigmatically from conventionaltherapeutic norms and (2) that the multimodal behavior-ally focused studies led in particular by Ornish et al in the1980s6061 provided early methodological inspiration forwhole systems researchers
Study selection (ie identification charting and culling)continued iteratively until (1) lsquolsquotheoretical saturationrsquorsquo62
was reached in that review of additional candidate publi-cations failed to reveal new WSR methodological featuresand (2) a wide range of clinical whole systems paradigmswere represented within the dataset Study results were nottaken into account during the selection process
About 90 of studies recommended by at least one subjectarea expert were included and approximately two-thirds of theincluded studies had been directly recommended by at leastone WSR expert (including coauthors JR and CE) the re-mainder was identified in literature searches undertaken by theprimary author (NI) Four expert-recommended studies wereexcluded because they (1) did not meet the study inclusioncriteria (n = 1) or (2) were methodologically very similar toother selected exemplars providing little added value tothe review (n = 3) The final selection of studies deliberatelyover-represents traditional (ie Ayurvedic and Chinese)medicine systems to thoroughly address the paradigm-specificdiagnostic intervention and outcome design considerationsthat arise in these contexts
Data charting
Focused around three primary analytic categoriesmdashStudyDesign Intervention Selection and Outcome Evaluationmdashthe primary author (NI) summarized and evaluated eachcandidate study using an emergent set of tables and chartsThrough a constant comparative approach that reviewedeach study in relation to all others63 a set of analytic sub-parameters and conceptual frames progressively emergedThis process permitted a finalized study selection and adetailing of each studyrsquos distinct and nondistinct methodo-logical features
Expert validation of findings
While analysis and reporting were undertaken primarilyby the primary author (NI) a subset of categorizations
related to lsquolsquodual diagnosticsrsquorsquo and paradigm-specific out-comes was independently corroborated by another coauthor( JR) All coauthors ( JR CE JW) contributed insightsas to the emerging conceptual categories as the projectprogressed and provided input on the final analyses beforethis workrsquos peer review by other WSR field experts
Result collation summary and reporting
Results are synthetically presented and discussed in whatfollows using both narrative and graphical reporting Tofacilitate reading ease in-text WSR exemplar referencesname first authors only full citations may be found in thereference list To provide context and language to facilitatenuanced reporting of the WSR fieldrsquos features two noveltheoretical frameworks are presented below
Theory
Model validity framework
The model validity principle has been conceptualized ascentral to WSR and as noted earlier on various scholarshave suggested ways in which this principle may be enactedwithin clinical research contexts What remained implicit inmuch (although not all) of the early WSR methodologicalliterature is that WSR itself may be understood as part of anlsquolsquointegrative medicinersquorsquo movement64 geared to transformingdominant health care systems such that TCIM therapies maybe more broadly integrated alongside or as an adjunct toconventional biomedical care Clinical research is in thisscenario envisioned as a necessary but insufficient tool tohelp to dismantle barriers to integration17
Several scholars critiquing the integrative medicineproject have however suggested a potential for the distinctparadigmatic features of and practices with origins in non-biomedical therapeutic systems to be co-opted appro-priated or assimilated in such a process65 Model validityas a theoretical construct represents a commitment to ac-tively preserving these paradigms and practices in their ownright an approach aligned with the concept of a clearlylsquolsquoarticulatedrsquorsquo66 equitable medical pluralism67 rather thanan assimilative mode of integration65
What WSR pioneers were not able to fully apprise inadvance was how and to what degree future WSR methodsmight ultimately align or diverge with conventional researchstrategies in pursuit of model validity To facilitate analysisof these points in the present scoping review it is proposedthat the model validity principle be theoretically differenti-ated into three co-embedded categories as seen in Figure 1paradigm compatibility paradigm consistency and paradigmspecificity These categories are not mutually exclusive thatis a single study may concurrently include different aspects(eg method intervention design and outcome measures)marked by one or more of the identified characteristics
Paradigm compatibility model validityrsquos driving conceptis conceptualized as a category that includes two othersmdashparadigm consistency and paradigm specificitymdashthe secondof which is embedded within the first Paradigm compatibleresearch methods are those typically associated with domi-nant biomedical clinical research but which also readilylend themselves to the study of whole systems clinical in-terventions Paradigm-consistent methods differ in key
S26 IJAZ ET AL
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
ways from conventional research approaches but are dis-tinctly suited to evaluating a wide range of whole systemsinterventions Paradigm-specific research methods differ ordiverge from conventional research approaches and arefurthermore uniquely tailored to one specific clinical wholesystem or paradigm
Individualization spectrum
Individualized care represents a core therapeutic principleacross TCIM whole systems and is thus a key considerationwith regards to WSR model validity Strategies for indi-vidualizing care in clinical research contexts have beenexplored over the last two decades in particular in the fieldof biomedical psychotherapy Researchers in that field haveunfolded what has come to be known as lsquolsquomanualizationrsquorsquoin which formal treatment manuals specify a predeterminedset of intervention parameters within which study cliniciansare granted scope to individually tailor treatments68 Itshould be similarly noted that Chinese Ayurvedic and othertraditional medicine systems have for many centuries usedsemistandardized treatment protocolization as a structurewithin which to personalize patient treatments3032 In suchtraditional systems generalized treatment parameters (egdietary recommendations herbal formulations and acu-puncture point combinations) are detailed in relation toparticular primary diagnostic or constitutional patterns pro-viding clinicians with a framework within which to furthertailor care However in other TCIM paradigms (eg natu-ropathic medicine and chiropractic) individual clinicianscommonly individualize treatments with fewer definedconstraints
To facilitate a nuanced representation of the range of ap-proaches to intervention individualization evident in the WSR
exemplars reviewed a theoretically-novel individualizationspectrum is presented in Figure 2 This spectrum dif-ferentiates the broad range of approaches to treatmentpersonalization under three broad categories generalstandardization manualization with tailoring and un-constrained individualization Toward the left of thespectrummdashlsquolsquogeneral standardizationrsquorsquomdashare interventionsinvolving predefined inflexible interventions uniformlydelivered to all participants At the spectrumrsquos right aretreatments characterized by their lsquolsquounconstrained individ-ualizationrsquorsquo in which providers have discretion to uniquelytreat each patient within the breadth of their clinical scopeAt the spectrumrsquos center are lsquolsquomanualization with tailoringrsquorsquoapproaches in which clinicians have autonomy to personalizetreatments in adherence to prespecified intervention parame-ters As seen in Figure 2 the spectrumrsquos three base categoriesare neither rigid nor mutually exclusive rather features of oneapproach may be evident in an intervention or study dominatedby another
Results Overview
This scoping review evaluates a total of 41 WSR stud-ies from across the paradigms of anthroposophic69ndash71
Ayurvedic2972ndash76 Chinese77ndash86 chiropractic33 complemen-taryintegrative87ndash91 energy36 homeopathic92 naturopath-ic8593ndash96 and preventiverestorative6097ndash99 medicines as wellas midwifery100 Swedish massage101102 trsquoai chi103 and yogatherapy29104 The whole systems interventions reported acrossthese studies range in size from one74 to almost three thou-sand98 patients and in duration from 1 day87 to severalyears6971 Conducted across several continents these studiesaddress many areas of clinical focus including acute87 andchronic94 anxiety adjunct oncology care83888991 acute79 aswell as chronic71 illness (including headache36 rheuma-toid arthritis69 heart disease6072959899 and diabetes93105)insomnia92 obesity29 and tinnitus82 musculoskeletalpain33707585869096101103 reproductive6074778198 and re-spiratory737880 conditions and medically unexplainedsymptoms84 Rather than treating lsquolsquodiseasersquorsquo conditions per sea number of studies focus primarily on well-being76 quality oflife (QoL)8891 social and emotional skills104 prevention andrehabilitation6097ndash99 clinical care dynamics102 and patientsatisfaction with clinical care100
Several of the reviewed studies have secondary associ-ated publications detailing qualitative research338493101 oreconomic outcomes9596106 Other secondary publicationtypes include stand-alone study protocols3375 earlier pilotfeasibility studies80101 methodological works3386101 andarticles detailing additionalfollow-up outcomes6070839298100
FIG 2 Spectrum of clinical individualization strategies
FIG 1 Model validity framework
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S27
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
2
Meth
od
olo
gica
lO
verv
iew
of
Wh
ole
Sy
stem
sR
esea
rch
Stu
dies
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Att
ias
2016
Isra
el87
Com
ple
men
tary
in
tegra
tive
med
icin
eP
reoper
ativ
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=360
1day
Arm
sIndash
V
Usu
alca
re(p
har
mac
euti
cal)
+I
stan
dar
diz
edguid
edim
ager
y
II
indiv
idual
ized
guid
edim
ager
y
III
indiv
idual
ized
refl
exolo
gy
IV
indiv
idual
ized
guid
edim
ager
y+
indiv
idual
ized
refl
exolo
gy
V
indiv
idual
ized
acupunct
ure
A
rmV
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(anxie
tyV
AS
)
Azi
zi2011
Chin
a77
Ch
ines
em
edic
ine
Men
opau
se-r
elat
edsy
mpto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
m
ult
iarm
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
A
rmII
S
tandar
diz
edher
bal
mix
ture
+st
andar
diz
edac
upunct
ure
A
rmII
IU
sual
care
(phar
mac
euti
cal)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Kupper
man
index
)S
econ
dary
H
orm
onal
blo
odw
ork
(est
radio
l)
num
ber
of
sym
pto
ms
Bel
l2011
Unit
edS
tate
sof
Am
eric
a921
081
09
Hom
eop
ath
icm
edic
ine
Coff
ee-i
nduce
din
som
nia
n-o
f-1
Ser
ies
Dynam
ical
lyal
loca
ted
pat
ient-
bli
nded
pla
cebo-c
ontr
oll
ed
two-
per
iod
com
par
ativ
eef
fect
iven
ess
(AB
1A
B2)
des
ign
N=
54
4middot
1w
eek
phas
es
Inte
rven
tion
A
Hom
eopat
hic
pla
cebo
Inte
rven
tion
B1
Hom
eopat
hic
rem
edy
IIn
terv
enti
on
B2
Hom
eopat
hic
rem
edy
II
Pri
mary
F
unct
ional
slee
pbio
mea
sure
s(P
oly
som
nogra
phy)
Sym
pto
mse
ver
ity
PR
OM
M
enta
lhea
lth
PR
OM
s
Ben
-Ary
e2018
Isra
el88
Com
ple
men
tary
in
tegra
tive
med
icin
eC
hem
o-i
nduce
dta
ste
dis
ord
er
Pre
ndashp
ost
coh
ort
stu
dy
Sin
gle
arm
ch
art
revie
wdes
ign
N=
34
pound12
wee
ks
Sin
gle
arm
In
div
idual
ized
com
ple
men
tary
inte
gra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
(MY
CaW
)S
ym
pto
mse
ver
ity
PR
OM
Bra
dle
y2012
Unit
edS
tate
sof
Am
eric
a931
10
Natu
rop
ath
icm
edic
ine
Type
IIdia
bet
es
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
93
usu
alca
reco
mpar
ator
Qual
itat
ive
subst
udy
110
N=
40
+N
=329
(contr
ol)
6ndash12
month
sIn
-dep
thin
terv
iew
sN
=5
Arm
IIn
div
idual
ized
whole
syst
emnat
uro
pat
hic
+usu
alca
re
Arm
II
Usu
albio
med
ical
care
dat
afr
om
elec
tronic
hea
lth
reco
rds
Qu
ali
tati
ve
aim
sE
xplo
rati
on
of
pat
ient-
report
edex
per
ience
sre
ceiv
ing
firs
t-ti
me
nat
uro
pat
hic
care
Pri
mary
A
dher
ence
PR
OM
m
enta
lhea
lth
PR
OM
s(P
HQ
-8)
self
-ef
fica
cyP
RO
M
emoti
onal
wel
lnes
sP
RO
M
blo
od
lipid
sblo
od
pre
ssure
S
econ
dary
T
reat
men
tsa
tisf
acti
on
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
Bre
des
en2016
Unit
edS
tate
sof
Am
eric
a971
11
Pre
ven
tive
rest
ora
tive
bio
med
icin
eA
lzhei
mer
rsquosdis
ease
Ret
rosp
ecti
ve
case
seri
esN
=10
5ndash24
month
spound3
5yea
rfo
llow
-up
Man
ual
ized
tai
lore
dpre
ven
tive
die
tli
fest
yle
pro
toco
lP
rim
ary
F
unct
ional
dis
ease
pro
gre
ssio
nte
stin
g(q
uan
tita
tive
MR
Ineu
ropsy
cholo
gic
test
ing)
nar
rati
ve
case
report
ing
Bri
nkhau
s2004
Ger
man
y78
Ch
ines
em
edic
ine
Sea
sonal
rhin
itis
Ran
dom
ized
con
troll
edtr
ial
Pla
cebo-c
ontr
oll
ed
pat
ient-
bli
nded
des
ign
intr
apar
adig
mat
icco
mpar
ator
N=
57
2m
onth
s
Arm
IS
tandar
diz
edtr
adit
ional
her
bal
mix
ture
+m
anual
ized
tai
lore
dher
bal
mix
ture
+m
anual
ized
ta
ilore
dac
upunct
ure
A
rmII
H
erbal
pla
cebo
+S
ham
acupunct
ure
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(VA
S)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Q
oL
PR
OM
s(S
F-3
6)
pla
cebo
cred
ibil
ity
scal
em
edic
atio
nusa
ge
blo
odw
ork
for
adver
seef
fect
test
ing
(conti
nued
)
S28
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Coole
y2009
Can
ada9
4N
atu
rop
ath
icm
edic
ine
Moder
ate
sever
ean
xie
ty
Ran
dom
ized
con
troll
edtr
ial
Open
label
pla
cebo-c
ontr
oll
ed
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
81
Dagger8w
eeks
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
+st
andar
diz
edm
ult
ivit
amin
and
her
b+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
Arm
II
Pla
cebo
(mult
ivit
amin
)+
stan
dar
diz
eddee
pbre
athin
ged
uca
tion
+usu
alca
re(p
sych
oth
erap
y)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
(SF
-36)
Fat
igue
PR
OM
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)ad
her
ence
PR
OM
an
dpat
ient
sati
sfac
tion
PR
OM
Dubro
ff2015
Can
ada7
2A
yu
rved
icm
edic
ine
Coro
nar
yhea
rtdis
ease
Pre
ndashp
ost
coh
ort
stu
dy
N=
19
3m
onth
sM
anual
ized
tai
lore
dA
yurv
edic
die
tary
counse
ling
her
bal
form
ula
tion
+st
andar
diz
edyoga
med
itat
ion
and
bre
athw
ork
Pri
mary
A
rter
ial
puls
ew
ave
vel
oci
ty
Sec
on
dary
A
nth
ropom
etri
cs(B
MI)
blo
od
pre
ssure
blo
od
lipid
sm
edic
atio
nusa
ge
Eld
er2006
Unit
edS
tate
sof
Am
eric
a105
Ayu
rved
icm
edic
ine
Type
IIdia
bet
esR
an
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
60
18
wee
ks
6-m
onth
foll
ow
-up
Arm
IS
tandar
diz
edA
yurv
edic
inte
rven
tion
(die
tary
counse
ling
med
itat
ion
her
bal
supple
men
t)+
indiv
idual
ized
exer
cise
A
rmII
S
tandar
ddia
bet
esed
uca
tion
clas
ses
+usu
alca
re
Pri
mary
B
lood
glu
cose
S
econ
dary
B
lood
lipid
sblo
od
pre
ssure
puls
ean
thro
pom
etri
cs(w
eight)
ad
her
ence
qual
itat
ive
adher
ence
bar
rier
sfa
cili
tato
rs
per
ceiv
edben
efits
Eld
er2018
Unit
edS
tate
sof
Am
eric
a331
12ndash115
Ch
irop
ract
icm
edic
ine
Chro
nic
nec
kb
ack
pai
nC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hpro
pen
sity
score
mat
ched
contr
ols
A
ssoci
ated
cross
-sec
tional
surv
eyan
del
ectr
onic
med
ical
reco
rdre
vie
w114
qual
itat
ive
subst
udy
115
N=
70
N=
139
(contr
ol)
6
month
sIn
terv
iew
sfo
cus
gro
ups
N=
90
(pat
ients
)n
=25
+n
=14
(hea
lth
care
pro
vid
ers)
Arm
IIn
div
idual
ized
whole
syst
emch
iropra
ctic
care
A
rmII
U
sual
care
(bio
med
ical
)Q
uali
tati
ve
Aim
sIn
itia
lmdashT
oex
plo
repat
ient
pra
ctit
ioner
exper
ience
sw
ith
and
dec
isio
n-m
akin
gar
ound
AC
use
for
chro
nic
MS
Kpai
n
Iter
ativ
emdashT
och
arac
teri
zepra
ctic
alis
sues
face
dby
pat
ients
seek
ing
alte
rnat
ives
toopio
idch
ronic
MS
Kpai
nm
anag
emen
t
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
S
leep
qual
ity
PR
OM
M
enta
lhea
lth
PR
OM
sQ
oL
PR
OM
dir
ect
hea
lth
cost
s
Flo
wer
2012
Unit
edS
tate
sof
Am
eric
a79
Ch
ines
em
edic
ine
Uri
nar
ytr
act
infe
ctio
nP
rosp
ecti
ve
case
seri
esF
easi
bil
ityp
ilot
des
ign
N=
14
6m
onth
s
Indiv
idual
ized
her
bal
mix
ture
+st
andar
diz
edlsquolsquo
acute
rsquorsquoher
bal
mix
ture
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
S
ym
pto
mse
ver
ity
and
wel
lnes
sP
RO
Ms
med
icat
ion
usa
ge
Fors
ter
2016
Aust
rali
a1001
16
Mid
wif
ery
Pat
ient
sati
sfac
tionC
-se
ctio
nra
tes
Ran
dom
ized
con
troll
edtr
ial
Com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
2314
per
inat
alca
re+2
month
spost
par
tum
Arm
IM
anual
ized
case
load
lsquolsquoco
nti
nuousrsquo
rsquom
idw
ifer
yper
inat
alca
re
Arm
II
Usu
alca
re(n
onca
selo
adm
idw
ifer
y
junio
robst
etri
cor
gen
eral
pra
ctit
ioner
)
Pri
mary
P
atie
nt
sati
sfac
tion
PR
OM
(unval
idat
ed)
cesa
rean
bir
th
Sec
on
dary
M
edic
atio
nusa
ge
inst
rum
enta
lin
duce
dbir
ths
mat
ernal
per
inea
ltr
aum
ain
fant
anth
ropom
etri
cs
(conti
nued
)
S29
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Ham
re2007
Ger
man
y701
17
An
thro
poso
ph
icm
edic
ine
Low
bac
kpai
n
Con
troll
edP
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
62
12
month
s2-y
ear
foll
ow
-up
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
care
A
rmII
U
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(sym
pto
msc
ore
)Q
oL
PR
OM
(SF
-36)
Ham
re2013
Ger
man
y711
07
An
thro
poso
ph
icm
edic
ine
Chro
nic
dis
ease
s
Pre
ndashP
ost
Coh
ort
Stu
dy
N=
1510
4yea
rs
Indiv
idual
ized
whole
syst
eman
thro
poso
phic
care
P
rim
ary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
(Sym
pto
mS
core
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
Men
tal
hea
lth
PR
OM
P
atie
nt
sati
sfac
tion
Ham
re2018
Ger
man
y69
An
thro
poso
ph
icm
edic
ine
Rheu
mat
oid
arth
riti
s
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
enhan
ced
usu
alca
reco
mpar
ator
Ass
oci
ated
synth
etic
over
vie
wof
21
rela
ted
publi
cati
ons
107
N=
251
4yea
rs
Arm
IIn
div
idual
ized
whole
syst
eman
thro
poso
phic
med
icin
e+
Usu
alca
re(c
ort
icost
eroid
and
NS
AID
s)A
rmII
E
nhan
ced
usu
alca
re(c
ort
icost
eroid
sN
SA
IDs
+D
MA
RD
s)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
C
-rea
ctiv
eblo
od
pro
tein
dis
ease
pro
gre
ssio
nte
st(R
atin
gen
Sco
re)
Huan
g2018
Chin
a801
18
Ch
ines
em
edic
ine
Bro
nch
iect
asis
n-o
f-1
Ser
ies
Ran
dom
ized
double
-bli
nd
six-p
erio
dcr
oss
over
com
par
ativ
eef
fect
iven
ess
(AB
AB
AB
)des
ign
N=
17
3phas
es(2
middot4
wee
ks)
3
wee
kw
ashouts
Inte
rven
tion
A
Man
ual
ized
tai
lore
dher
bal
mix
ture
In
terv
enti
on
B
Sta
ndar
diz
edher
bal
mix
ture
Pri
mary
P
atie
nt-
gen
erat
edsy
mpto
mse
ver
ity
PR
OM
Sec
on
dary
S
putu
mvolu
me
blo
odw
ork
for
adver
seev
ent
asse
ssm
ent
Hull
ender
Rubin
2015
Unit
edS
tate
sof
Am
eric
a81
Ch
ines
em
edic
ine
IVF
outc
om
esR
etro
spec
tive
post
-on
lyst
ud
yM
ult
iarm
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
rean
din
trap
arad
igm
atic
com
par
ators
N
=1231
Arm
IIn
div
idual
ized
whole
syst
emC
hin
ese
med
icin
e+
IVF
Arm
II
Sta
ndar
diz
edac
upunct
ure
+IV
FA
rmII
IU
sual
care
(IV
Fal
one)
Pri
mary
H
ealt
hev
ent
(Liv
ebir
th)
Sec
on
dary
H
ealt
hev
ents
(bio
chem
ical
pre
gnan
cy
single
ton
twin
tri
ple
tec
topic
abort
ed
ges
tati
onal
age)
Jack
son
2006
Unit
edS
tate
sof
Am
eric
a82
Ch
ines
em
edic
ine
Tin
nit
us
n-o
f-1
Ser
ies
Open
label
tw
o-p
erio
d(A
B)
des
ign
N=
6
14
day
str
eatm
ent
two
(pre
ndashpost
)14
day
eval
uat
ion
phas
es
Per
iod
A
Indiv
idual
ized
trad
itio
nal
acupunct
ure
10
trea
tmen
ts
Per
iod
B
Post
-tre
atm
ent
contr
ol
per
iod
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(unval
idat
ed)
Sec
on
dary
pat
ient-
gen
erat
edP
RO
M(M
YM
OP
)
Josh
i2017
India
73
Ayu
rved
icm
edic
ine
Ast
hm
aC
on
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
pro
of-
of-
conce
pt
des
ign
intr
apar
adig
mat
ican
dhea
lthy
com
par
ators
N
=115
+n
=69
(contr
ol)
6
month
s
Arm
sI
1II
M
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
e(I
vat
a-dom
inan
tas
thm
aII
kap
ha-
dom
inan
tas
thm
a)
Arm
III
Hea
lthy
contr
ol
gro
up
com
par
ator
Pri
mary
B
lood
IgE
level
seo
sinophil
counts
sp
irom
etry
cy
tokin
es
Lung
funct
ion
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
Kes
sler
2015
Ger
man
y74
Ayu
rved
icm
edic
ine
Infe
rtil
ity
Sin
gle
Case
Rep
ort
N=
1
12
month
sIn
div
idual
ized
whole
syst
emA
yurv
edic
med
icin
eP
rim
ary
H
ealt
hev
ent
(liv
ebir
th)
nar
rati
ve
case
report
ing
(conti
nued
)
S30
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Kes
sler
2018
Ger
man
y751
19
Ayu
rved
icm
edic
ine
Knee
ost
eoar
thri
tis
Ran
dom
ized
Con
troll
edT
rial
Open
-lab
el
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
151
12
wee
ks
6-
and
12-w
eek
foll
ow
-up
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emA
yurv
edic
med
icin
eA
rmII
U
sual
care
in
cludin
gocc
upat
ional
man
ual
ther
apy
hom
eex
erci
se
die
tary
counse
ling
and
pai
nm
edic
atio
n
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
ood
pai
n
slee
pP
RO
MS
(SF
-36)
med
icat
ion
usa
ge
Lit
chke
2018
Unit
edS
tate
sof
Am
eric
a104
Yoga
ther
ap
yS
oci
ale
moti
onal
skil
lsin
auti
sm
Pre
ndashP
ost
Coh
ort
Stu
dy
Explo
rato
rydes
ign
N=
5
4w
eeks
Sta
ndar
diz
edbiw
eekly
yoga
ther
apy
inst
ruct
ion
Pri
mary
P
sych
oso
cial
scal
esobse
rver
report
ed
Nar
rati
ve
note
s
McC
ull
och
2011
Unit
edS
tate
sof
Am
eric
a831
20
Ch
ines
em
edic
ine
an
dV
itam
inth
erap
yL
ungc
olo
nca
nce
rsu
rviv
al
Ret
rosp
ecti
ve
Con
troll
edP
ost
-On
lyS
tud
yM
atch
edco
ntr
ol
com
par
ativ
eoutc
om
esdes
ign
wit
hpro
pen
sity
score
and
mar
gin
alst
ruct
ura
lan
alyti
cm
ethods
usu
alca
reco
mpar
ator
Lung
cance
rN
=239
+n
=127
54
(contr
ol)
C
olo
nca
nce
rN
=193
+n
=136
65
(contr
ol)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
emC
hin
ese
med
icin
e+
Vit
amin
ther
apy
+U
sual
care
(bio
med
ical
onco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
H
ealt
hev
ent
(Surv
ival
)
Mil
ls2016
Unit
edS
tate
sof
Am
eric
a76
Ayu
rved
icm
edic
ine
Wel
l-bei
ng
Qu
asi
-Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
69
6day
s
Arm
IS
tandar
diz
edA
yurv
edic
min
dndash
body
resi
den
tial
gro
up
pro
gra
mA
rmII
R
esid
enti
alvac
atio
nat
sam
esi
te
Pri
mary
W
elln
ess
PR
OM
sfo
rsp
irit
ual
ity
gra
titu
de
self
-co
mpas
sion
psy
cholo
gic
wel
l-bei
ng
men
tal
hea
lth
Sec
on
dary
B
lood
pre
ssure
an
thro
pom
etri
cs(h
eight
wei
ght)
Orn
ish
1998
Unit
edS
tate
sof
Am
eric
a601
21
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Ran
dom
ized
Con
troll
edT
rial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
48
1yea
rn
=35
5-y
ear
foll
ow
-up
Arm
IS
tandar
diz
edli
fest
yle
pro
gra
m(d
iet
exer
cise
sm
okin
gce
ssat
ion
stre
ssm
anag
emen
t)w
ith
biw
eekly
support
gro
up
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
A
ngio
gra
phy
(coro
nar
yar
tery
lesi
on
char
acte
rist
ics)
blo
od
lipid
s
Pat
erso
n2011
Unit
edK
ingdom
841
22
Ch
ines
em
edic
ine
Med
ical
lyunex
pla
ined
sym
pto
ms
Ran
dom
ized
con
troll
edtr
ial
Open
label
des
ign
wit
hw
aiti
ng
list
cross
over
contr
ol
Qual
itat
ive
subst
udy
122
N=
80
6m
onth
s1-y
ear
foll
ow
-up
N=
20
pre
-an
dpost
inte
rvie
ws
Arm
IIn
div
idual
ized
trad
itio
nal
acupunct
ure
+usu
alca
re
Arm
II
Wai
tli
st+
usu
alca
re
Pri
mary
P
atie
nt-
gen
erat
edP
RO
M(M
YM
OP
)S
econ
dary
W
ellb
eing
and
QoL
PR
OM
sm
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
Per
lman
2016
Unit
edS
tate
sof
Am
eric
a411
011
231
24
Sw
edis
hm
ass
age
ther
ap
yK
nee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
pro
toco
ldev
elopm
ent4
1
dose
-findin
g1
24
and
qual
itat
ive1
23
subst
udie
sN
=222
8w
eeks
52-w
eek
foll
ow
-up
N=
18
inte
rvie
ws
Arm
IM
anual
ized
tai
lore
dS
wed
ish
mas
sage
Arm
II
Sta
ndar
diz
edli
ght
touch
bodyw
ork
A
rmII
IU
sual
care
(bio
med
ical
)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
F
unct
ional
test
s(w
alk
test
ra
nge
of
moti
on)
hea
lth
expen
dit
ure
m
edic
atio
nusa
ge
(conti
nued
)
S31
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Rio
ux
2014
Unit
edS
tate
sof
Am
eric
a29
Ayu
rved
icm
edic
ine
Obes
ity
Pre
ndashp
ost
coh
ort
stu
dy
Fea
sibil
ityp
ilot
des
ign
N=
17
3m
onth
s6-
and
9-m
onth
foll
ow
-ups
Man
ual
ized
tai
lore
dA
yurv
edic
die
tli
fest
yle
counse
ling
+st
andar
diz
edyoga
ther
apy
inst
ruct
ion
and
hom
epra
ctic
e
Pri
mary
A
nth
ropom
etri
c(w
eight
BM
Ibody
fat
wai
sth
ipci
rcum
fere
nce
rat
io)
Par
adig
m-
spec
ific
PR
OM
s(u
nval
idat
ed)
Psy
choso
cial
PR
OM
s(B
andura
)A
dher
ence
Rit
enbau
gh
2008
Unit
edS
tate
sof
Am
eric
a85
Ch
ines
ean
dN
atu
rop
ath
icm
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Ran
dom
ized
con
troll
edtr
ial
Mult
iarm
open
label
com
par
ativ
eef
fect
iven
ess
des
ign
inte
rpar
adig
mat
ican
dusu
alca
reco
mpar
ators
N
=160
6m
onth
s(C
hin
ese
med
icin
e)
8m
onth
s(n
aturo
pat
hic
med
icin
e)
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
e+
stan
dar
diz
edre
laxat
ion
tapes
A
rmII
M
anual
ized
tai
lore
dnat
uro
pat
hic
med
icin
e+
stan
dar
diz
ednutr
itio
nal
supple
men
tA
rmII
IS
pec
ialt
yden
tal
care
physi
cal
ther
apyp
sych
olo
gic
refe
rral
s
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
Rit
enbau
gh
2012
Unit
edS
tate
sof
Am
eric
a861
251
26
Ch
ines
em
edic
ine
Tem
poro
man
dib
ula
rdis
ord
ers
Dyn
am
icall
y-a
lloca
ted
con
troll
edtr
ial
Open
label
st
epped
care
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
168
1yea
r
Arm
IM
anual
ized
tai
lore
dw
hole
syst
ems
Chin
ese
med
icin
eA
rmII
P
sych
oso
cial
self
-car
eed
uca
tion
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
m
enta
lhea
lth
wel
lbei
ng
PR
OM
S
Med
icat
ion
usa
ge
See
ly2013
Can
ada9
51
27
Natu
rop
ath
icm
edic
ine
Car
dio
vas
cula
rdis
ease
pre
ven
tion
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
27
N=
246
1yea
r
Arm
IM
anual
ized
tai
lore
ddie
tex
erci
seco
unse
ling
nutr
itio
nal
supple
men
tati
on
+U
sual
care
(bio
med
ical
)A
rmII
U
sual
care
(bio
med
ical
)E
con
om
icE
valu
ati
on
D
irec
t+
indir
ect
cost
sto
emplo
yer
so
ciet
y
Pri
mary
C
ardio
vas
cula
rri
sk(F
ram
ingham
algori
thm
)bas
edon
blo
od
lipid
san
dglu
cose
blo
od
pre
ssure
an
dan
thro
pom
etri
cs(w
aist
circ
um
fere
nce
)S
econ
dary
Q
oL
PR
OM
(SF
-36)
pat
ient
gen
erat
edP
RO
M(M
YM
OP
)
Shal
om
-Shar
abi
2017
Isra
el89
Com
ple
men
tary
in
tegra
tive
med
icin
eC
ance
rgas
troin
test
inal
Q
oL
Con
troll
edp
rendashp
ost
stu
dy
Com
par
ativ
eef
fect
iven
ess
des
ign
pra
gm
atic
ally
-as
signed
contr
ol
usu
alca
reco
mpar
ator
N=
175
6w
eeks
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
e+
usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
sin
cludin
gpat
ient-
gen
erat
edP
RO
M(M
YC
aW)
Sil
ber
man
2010
Unit
edS
tate
sof
Am
eric
a981
28ndash130
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Pre
ndashp
ost
coh
ort
stu
dy
Tim
ese
ries
des
ign
N=
2974
1yea
r
Sta
ndar
diz
eddie
tli
fest
yle
and
stre
ssm
anag
emen
tpro
gra
mw
ith
psy
choso
cial
gro
up
support
Pri
mary
F
unct
ional
test
ing
(tre
adm
ill)
blo
od
lipid
san
dsu
gar
m
enta
lhea
lth
PR
OM
sad
her
ence
Suth
erla
nd
2009
Unit
edS
tate
sof
Am
eric
a36
En
ergy
med
icin
eC
hro
nic
hea
dac
hes
Pre
ndashp
ost
coh
ort
stu
dy
Qual
itat
ive
des
ign
N=
13
inte
rvie
ws
3ndash7
wee
ks
3-m
onth
foll
ow
-up
Indiv
idual
ized
hea
ling
touch
sess
ions
Qu
ali
tati
ve
ou
tcom
es
Inte
rvie
wfo
rS
ym
pto
mse
ver
ity
wel
lnes
sQ
oL
(conti
nued
)
S32
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Ta
ble
2
(Co
ntin
ued
)
Stu
dy
loca
tion
Para
dig
mf
ocu
sD
esig
nIn
terv
enti
ons
Outc
om
es
Szc
zurk
o2007
Can
ada9
61
31
Natu
rop
ath
icm
edic
ine
Chro
nic
low
bac
kpai
n
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
wit
hopti
onal
cross
-over
usu
alca
reco
mpar
ator
Eco
nom
icsu
bst
udy1
31
N=
75
Dagger8
wee
ks
Arm
IS
tandar
diz
edac
upunct
ure
dee
pbre
athin
g
die
tex
erci
seco
unse
ling
Arm
II
Educa
tional
physi
oth
erap
ybookle
t+
live
exer
cise
rel
axat
ion
inst
ruct
ion
Eco
nom
icev
alu
ati
on
D
irec
t+
indir
ect
cost
sto
indiv
idual
em
plo
yer
so
ciet
y
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
(Osw
estr
y)
QoL
PR
OM
(SF
-36)
Sec
on
dary
S
ym
pto
mse
ver
ity
PR
OM
an
thro
pom
etri
cs(B
MI
wei
ght)
fu
nct
ional
test
ing
(ran
ge
of
moti
on)
med
icat
ion
usa
ge
hea
lth
serv
ice
uti
liza
tion
adher
ence
w
ork
abse
nte
eism
Wan
g2016
Unit
edS
tate
sof
Am
eric
a103
trsquoai
chi
Knee
ost
eoar
thri
tis
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
204
12
wee
ks
Arm
IS
tandar
diz
edtrsquo
ai
chi
gro
up
inst
ruct
ion
+hom
epra
ctic
eA
rmII
U
sual
care
(physi
cal
ther
apy)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
M
enta
lhea
lth
QoL
an
dse
lf-e
ffica
cyP
RO
Ms
(SF
-36)
funct
ional
test
ing
(wal
k)
outc
om
eex
pec
tati
on
PR
OM
Way
ne
2018
Unit
edS
tate
sof
Am
eric
a90
Com
ple
men
tary
in
tegra
tive
med
icin
eL
ow
bac
kpai
n
Con
troll
edp
rendashp
ost
stu
dy
Open
label
pre
fere
nce
allo
cate
d
com
par
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
309
12
month
s
Arm
IIn
div
idual
ized
mult
idis
cipli
nar
yco
mple
men
tary
in
tegra
tive
med
icin
eplu
susu
alca
re(b
iom
edic
al)
Arm
II
Usu
alca
re(b
iom
edic
al)
Pri
mary
S
ym
pto
mse
ver
ity
PR
OM
S
econ
dary
Q
oL
PR
OM
hea
lth
serv
ice
uti
liza
tion
med
icat
ion
usa
ge
trea
tmen
tsa
tisf
acti
on
Wit
t2015
Ger
man
y91
Com
ple
men
tary
in
tegra
tive
med
icin
eB
reas
tca
nce
rQ
oL
Ran
dom
ized
con
troll
edtr
ial
Open
label
co
mpar
ativ
eef
fect
iven
ess
des
ign
usu
alca
reco
mpar
ator
N=
275
6m
onth
s
Arm
IIn
div
idual
ized
co
mple
xco
mple
men
tary
inte
gra
tive
med
icin
e+
Usu
alca
re(b
iom
edic
alonco
logy)
Arm
II
Usu
alca
re(b
iom
edic
alonco
logy)
Pri
mary
Q
oL
PR
OM
pat
ient-
gen
erat
edP
RO
M(u
nval
idat
ed)
hea
lth
even
t(s
urv
ival
)m
edic
atio
nusa
ge
hea
lth
serv
ice
uti
liza
tion
trea
tmen
tsa
tisf
acti
on
Wel
ch2013
Unit
edK
ingdom
102
Sw
edis
hm
ass
age
ther
ap
yIn
tegra
tive
care
dynam
ics
Eth
nogra
ph
yQ
ual
itat
ive-
dom
inan
tac
tion
rese
arch
des
ign
dra
win
gon
per
spec
tives
from
clin
icia
ns
pat
ients
an
dst
aff
inan
inte
gra
tive
care
clin
ic
N=
11
(physi
cian
s)
n=
33
(sta
ff)
n=
22
(pat
ients
)n
=1
(mas
sage
ther
apis
t)
12ndash13
wee
ks
Indiv
idual
ized
Sw
edis
hm
assa
ge
ther
apy
Qu
ali
tati
ve
aim
sT
oev
aluat
eco
nte
xtu
alfa
ctors
atpla
yin
anin
tegra
tive
care
sett
ing
wit
hat
tenti
on
toth
ein
terf
acin
gof
clin
icia
ns
from
mult
iple
dis
cipli
nes
Mix
ed-m
ethods
ques
tionnai
res
inte
rvie
ws
fiel
dnote
sre
flex
ive
journ
als
Zen
g2013
Unit
edS
tate
sof
Am
eric
a99
Pre
ven
tive
rest
ora
tive
bio
med
icin
eC
ardio
vas
cula
rre
hab
ilit
atio
n
Con
troll
edp
rendashp
ost
coh
ort
stu
dy
Mult
iarm
open
label
com
par
ativ
eoutc
om
esdes
ign
wit
hm
atch
edco
ntr
ol
com
par
ator
and
econom
icev
aluat
ion
N=
461
+1796
(contr
ol)
1
yea
r+3
yea
rfo
llow
-up
Arm
IS
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(O
rnis
h)
wit
hgro
up
com
ponen
tA
rmII
S
tandar
diz
eddie
tli
fest
yle
an
dst
ress
reduct
ion
pro
gra
m(B
enso
n-H
enry
)w
ith
gro
up
com
ponen
tA
rms
III
an
dIV
(matc
hed
pair
s)
Tra
dit
ional
card
iovas
cula
rre
hab
ilit
atio
n
No
card
iac
rehab
ilit
atio
n
Pri
mary
H
ealt
hse
rvic
euti
liza
tion
(hosp
ital
izat
ion)
hea
lth
even
t(m
ort
alit
y)
dir
ect
cost
s(i
nst
ituti
onal
)
Hea
din
gs
bold
edfo
rem
phas
is
BM
Ibody
mas
sin
dex
D
MA
RD
dis
ease
-modif
yin
gan
ti-r
heu
mat
icdru
g
IgE
im
munoglo
buli
nE
IV
F
invit
rofe
rtil
izat
ion
MR
Im
agnet
icre
sonan
ceim
agin
g
MY
CaW
M
easu
reY
ours
elf
Conce
rns
and
Wel
lbei
ng
MY
MO
P
Mea
sure
Yours
elf
Med
ical
Outc
om
eP
rofi
le
NS
AID
nonst
eroid
alan
ti-i
nfl
amm
atory
dru
g
PR
OM
pat
ient-
report
edoutc
om
em
easu
re
QoL
Q
ual
ity
of
life
S
F-3
6
Short
-Form
36
VA
S
vis
ual
anal
og
scal
e
S33
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Some studies337192101 feature multiple associated publi-cations a synthesis article107 related to one mixed methodsstudy in particular71 details 21 inter-related peer-reviewedpublications
What follows is a synthetic analytic report of the majormethodological features of the reviewed WSR studiespresented in three parts Part I (Study Design) addresses theprimary methodological modes selected by whole systemsresearchers Part II (Interventions) reviews the main char-acteristics of and strategies used in defining WSR inter-ventions across the reviewed exemplars Part III (OutcomeAssessment) elaborates the range of approaches to outcomeassessment adopted in each of the WSR exemplars andacross the field as a whole At the end of each of these threesections findings are discussed with reference to the modelvalidity principle and with a view to practical considerationsrelevant for researchers in the WSR field A subsequentDiscussionConclusion segment synthetically integrates find-ings from all three sections positioning them in a broaderhealth systems context
Table 2 provides a detailed overview of the 41 reviewedstudiesrsquo methodological features Additional Tables andFigures are used throughout this review to detail and sum-marize findings Where data are clearly represented withcitations in Tables andor Figures a note to this effect ismade in the review text direct in-text citations are providedfor more detailed findings not represented in graphical form
Part I study design
The reviewed WSR studies engage a cross-sectionof prospective and retrospective study types including
various controlled and uncontrolled experimental quasi-experimental and observational designs (Figure 3) Figure 4presents a detailed overview by study of major researchdesign features and will be repeatedly cited in the text toassist readers in identifying exemplars with particularcharacteristics As elaborated in what follows and is sum-marized in Figure 5 open label prospective comparativeeffectiveness designs with usual care comparators and ran-domized allocation represent the most common WSR ap-proach placebo controls and double blinding are rarelyapplied in the reviewed studies On the whole quantitativemethods dominate across almost all reviewed exemplarsThat said one-third are mixed methods studies most ofwhich incorporate qualitative methods (Fig 6) and a fewwith economic evaluations (Fig 7)
Comparativecontrolled trials Twenty-seven reviewedstudies including two with retrospective designs involve in-terventions whose clinical outcomes are contrasted head-to-head with at least one controlcomparator arm (most oftenlsquolsquousual carersquorsquo) Seven of these trials have three or more arms(Fig 4) Prendashpost designs are evident in all prospective studieswhereas the two retrospective studies evaluate postoutcomesonly As elaborated in what follows the reviewed comparativecontrolled studies implement various statistical and pragmaticapproaches to participant allocation use controlscomparatorsthat are largely activepositive andmdashwhile typically open la-belmdashapply assessor blinding methods in several cases
Statistical allocation Of the 27 evaluated controlledstudies 16 engage statistical approaches in allocating
FIG 3 Typology of whole systems research designs
S34 IJAZ ET AL
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
patients to particular treatment arms Randomization is thedominant approach although some studies use design-adaptive allocations (eg minimization) or matched controldesigns (Fig 3)
Simple randomization76ndash7884919496100 (n = 8 n = 4 withdemographic stratification789194100) and block randomiza-tion758795103 (n = 4 n = 2 with stratification7595) are at timesapplied alongside additional elements Szczurko et alrsquossimply randomized study for instance implements an op-
tional preference-based crossover96 Attias et alrsquos six-armedstudy uses block randomization to first allocate for individ-ualized versus standardized care subsequently assigningintervention-arm patients to receive a particular comple-mentary care approach based on the clinician type scheduledto work on the week day of their scheduled surgery87
Two randomized trials provide no additional details on theirallocation designs101105 although a few others use distinctiverandomization variants Ornish et al engage a randomized
FIG 4 Study designs in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S35
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
invitational design60 aimed at reducing disappointment-relatedattrition by asking participants to lsquolsquoagree to be testedrsquorsquo withoutbeing advance-apprised of the active interventionrsquos specif-ics132 After using simple randomization to assign the first 20(of 80) participants Paterson et al use minimizationmdasha designadaptive allocation strategymdashto allocate the remaining pa-tients84 Ritenbaugh et alrsquos study also applies a design adaptiverandomization approach with reference to several balancingfactors85
In a nonrandomized design-adaptive approach Ritenbaughet alrsquos study86 implements a stepped care (triaged) method
using minimization to dynamically allocate those with themost severe symptoms The researchers automatically assignthose with lesser symptoms to standard care and after a periodof treatment reassign standard care recipients with continuedlsquolsquosubstantial painrsquorsquo either to intervention or control
Three studies use nonrandomized statistical allocationmethods to create matched control groupsmdashcomposed oftwo33 or more8399 similar concurrent controls per inter-vention patientmdashfrom electronic medical records (EMRs)Two of these studies also apply propensity score meth-ods3383 in one case in a particularly innovative manner114
and in the other alongside additional statistical methods(including marginal structural models) to further adjust forconfounding83
Pragmatic allocation Of the 10 controlled trials that use(primarily) nonstatistical allocation strategies four698990 areprospective patient preference trials in which similar-sizedgroups of intervention and control patients concurrently selecttheir favored treatments (Fig 4) Two such studies use EMRsin one case to recruit intervention and control arm patients89
and in the other for a comparator group alone90 In the othertwo studies6970 both led by Hamre et al69 patients self-select to begin condition-specific care in anthroposophic andconventional care clinics respectively
In Hullender Rubin et alrsquos retrospective preference-basedstudy patients in three arms receive in vitro fertilization(IVF) alone IVF plus same-day acupuncture or IVF pluswhole systems Chinese medicine respectively81 Study an-alysts quasi-experimentally lsquolsquoadjust for covariates [via]
FIG 5 Controlledcomparative whole systems researchdesigns
Stand-alone qualitative publication linked to aPerlman 2012101 dBradley 201293 f Elder 201833 gRitenbaugh 201286bQualitative methods as dominant research approachcQualitative methods and analysis embedded in quantitative clinical outcomes publicationeQualitative results dually presented in stand-alone qualitative and quantitative papers
FIG 6 Qualitative methods in whole systems research
S36 IJAZ ET AL
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
multivariable logistic regression analysisrsquorsquo to lsquolsquominimizepotential biasrsquorsquo related to baseline intergroup differencesBradley et alrsquos prospective trialmdashconversely marked byintervention patientsrsquo lack of experience with or preferencefor naturopathic medicine (n = 40)mdashuses EMRs to assem-ble a substantially-larger (n = 329) demographically-similar(quasi-matched) control group93 Finally Joshi et al con-trast intervention outcomes with those from a similarly-sized demographically-similar healthy control group fromthe lsquolsquogeneral populationrsquorsquo73
Positive controls All but one78 of the controlled studiesreviewed have active (positive) comparator groups in al-most all cases with a usual care arm (Fig 4) Several amongthese737476869499103 engage complex individualized time-attention controls For instance Kessler et alrsquos osteoarthritisusual care control mirrors the studyrsquos multimodal pri-mary Ayurvedic intervention with an equivalent numberof individualized physiotherapy sessions paired with homeexercises dietary counseling and medication75 One studyconversely has notably low time-attention matching (vali-dated educational booklet vs multimodal naturopathic inter-vention)96 Some usual care comparators are innovative (ega residential lsquolsquovacationrsquorsquo to control for a mindndashbody re-treat76) Others more simply represent real-world usual care(eg conventional obstetric care compared with a caseloadmidwifery intervention100) Several studiesrsquo primary inter-ventions reflecting the normative context of biomedical careare furthermore designed as adjunctive to control that is theyinclude the same usual care as received by the comparatorgroup8183848789ndash9193ndash95119 (eg complementaryintegrativecancer care that includes conventional treatment838991)
One usual care-controlled study engages a crossoverwaiting list controlled design84 Those with multiple inter-vention arms73778185879299 almost universally implementintra-paradigmatic factorial comparator group designs(Fig 3) to trial a subset of or variation upon the primaryintervention (eg herbal mixture vs herbal mixture plusacupuncture77) Finally the single actively controlled studywithout a usual care comparator includes two distinct(nonfactorial) intra-paradigmatic intervention arms plus anuntreated healthy control group73
Placebosham controls Just two of the reviewed cohort-based controlled studies apply placebo andor sham controls(Fig 4) Cooley engages a multimodal naturopathic medi-cine design in which a multivitamin placebo forms part of acomplex open-label active usual care comparator94 Brin-
khaus et al trial verum versus sham acupuncture and anactive versus nonspecific herbal mixture78
Blinding Almost all controlled cohort-based WSR studieshave open label designs in which both patients and inter-ventionists are alert to participantsrsquo treatment allocationsassessoranalyst blinding is however almost universally ap-plied in these same studies (Fig 4) Brinkhaus et alrsquos ran-domized placebo-controlled study is the only cohort-basedstudy that implements full patient blinding78
Uncontrolled studies Seven of the reviewed studiesapply prospective uncontrolled cohort designs Four suchstudies are relatively small with fewer than 20 partici-pants293672104 2 are notably large with well over 1000patients each7198 Aside from the absence of comparatorarms most of these studies do not differ substantially inintervention or outcome design from the comparativecon-trolled trials discussed above That said a few have distinctmethodological features Silberman et alrsquos study uniquelyadopts a time series design to evaluate outcomes at (andbetween) intervals98 Sutherland et alrsquos study uses qualita-tive interviews (rather than quantitative measures) as itsprimary data generation approach36 and Ben-Arye et alrsquosstudy derives most outcomes prospectively from patientcharts88 rather than using quantitative outcome measuresalone
n-of-1 series Of the three n-of-1 series trials reviewed allrepresent adaptations of the classical n-of-1 single patientcrossover design Huang et alrsquos three-phase (ABABAB)comparative effectiveness design evaluates individualized(A) versus standardized (B) Chinese herbal mixtures forbronchiectasis80 using randomization to determine the orderof treatment versus control in each phase between wash-outs Bell et alrsquos placebo-controlled dynamically allocatedtwo-phase (AB) design (A placebo B treatment) compar-atively trials two different homeopathic insomnia remedieswith intermittent washouts92 Both of these studies reportpatient blinding with clinician blinding additionally appliedby Huang Jackson et alrsquos acupuncturetinnitus trial bycontrast uses a quasi-experimental open label two-period(AB) design (A treatment B no treatment) reporting in-dividual and combined outcomes from two-week pre- andpostintervention measurement periods82
Case study and case series This review includes one casestudy and two case series each of which presents a detailednarrative account of the effects of a particular complex
FIG 7 Economic evaluations in wholesystems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S37
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
treatment approach on specific individuals Like Kessleret alrsquos single case study75 Bredesen et alrsquos case series97 isretrospective detailing exceptional clinical outcomes from aparticular whole systems intervention Kessler et alrsquos75
study provides considerable detail about the Ayurvedictreatment approach applied well beyond the level of detailgiven in cohort-based studies within the same paradigmFlower and Lewithrsquos uniquely prospective case series79
designed as a preliminary clinical outcomes trial trackscommon Chinese medicine diagnostic patterns and otherinformative participant data to inform future study designs
Ethnography One reviewed study applies ethnographicmethods (eg participant observation interview question-naire and so on) within an action research framework toequally give voice to the perspectives of patients cliniciansand staff while also reporting clinical outcomes for aSwedish massage therapy intervention102
Mixed methods designs Seventeen reviewed studies en-gage mixed methods research designs either incorporatingqualitative alongside quantitative methods (n = 13) eco-nomic evaluations alongside clinical outcomes (n = 5) or inone complex design33 both
Qualitative methods As shown in Figure 6 the 14studies incorporating qualitative methods use open-endedquestionnaire items focus groups andor participant inter-views to investigate qualitative questions relating to treat-ment outcomes298489105123 treatment choices115 patientexperiences86110122 and protocol compliance105 One studyalso engages participant observation to document out-comes104 Content analysis with multianalyst corroborationof thematic results represents the most common qualitativeanalytic approach at times with numeric frequency calcu-lations andor quantitative corroboration Most studiespresent lsquolsquothick descriptiversquorsquo results using narrative andortable-based formats and report their qualitative findingseither in stand-alone publications or alongside quantitativeresults in mixed-methods clinical outcome articles
In just four studies qualitative methods dominate Kessleret alrsquos case study74 and Bredesen et alrsquos case series97111
provide narrative accounts of specific patientsrsquo therapeutictrajectories secondarily referring to quantitative data Su-therland et al uses in-depth interviews to explore clinicaland methodological questions relating to a healing touchintervention36 Welch et alrsquos ethnographic study uses mul-tiple qualitative methods to study stakeholder perspectivesand outcomes in an integrative medicine setting102 Theremaining nine studies deploy qualitative methods second-arily two do not report their qualitative results8589
The subordination of qualitative to quantitative methodsacross most studies might initially appear to convey a pos-itivist or post-positivist orientation133 consistent with thegeneral ethos of biomedical clinical research That saidmany studies use inductive data analytic approaches withintheir qualitative subcomponents suggesting a pragmaticapproach to mixed methods analysis that accommodatesconstructivist perspectives133 In Ritenbaugh et alrsquos study86
for instance study participants were repeatedly interviewedin an ethnographic mode over a year-long period The re-searchersrsquo initial intention to lsquolsquorelatequalitative narratives
to quantitative data on outcomesrsquorsquo was ultimately abandonedin light of the lsquolsquocomplexity of participantsrsquo [narrativeswhich]precluded a simplistic comparison between thesetwo disparate types of datarsquorsquo126
Economic evaluations Of the five reviewed studies thatinclude economic evaluations (Fig 7) two report their eco-nomic results within quantitative clinical outcomes articlesand three in stand-alone publications All report on directinstitutional expenditures associated with the interventions(vs comparators) under study the specified institutions in-clude the public purse99106 a corporate employer127131 anda nonprofit health maintenance organization105 Thoseeconomic evaluations published as stand-alone publica-tions106127131 additionally address indirect health-relatedcosts (eg work absenteeism and health related QoL) andreport from multiple expenditure vantage points beyond theinstitutional (eg individual societaltotal)
Model validity and practical considerations in WSR designselection Overall the research designs selected by wholesystems researchers are similar to those used by biomedicalresearchers at times with minor adaptations to enhance theirmodel validity It is unclear whether this emphasis on con-ventional paradigm compatible (and to a lesser degreeparadigm consistent) designs reflects these researchersrsquopreferences or is perhaps conversely indicative of theavailable financial support Regardless novel (ie para-digm specific) designs aremdashon the wholemdashnot evidentamong the reviewed exemplars That said just one of thereviewed cohort-based studies follows the classical RCTmodel in its concurrent use of randomized allocation par-ticipant and clinical blinding and placebo controls Echoingearly WSR critiques of the classical RCTrsquos model validity inTCIM contexts Brinkhaus et al explicitly recognize thatneither of their studyrsquos two adopted placebos is lsquolsquoentirelyinactiversquorsquo78
All other controlled cohort-based studies elect to either(1) select among a set of established modified RCT or non-RCT research designs (that demonstrate greater paradigmcompatibility than the classical RCT) or (2) implementedadaptations of such conventional study designs (to renderthem more paradigm consistent) Open label designs appearpreferable although assessoranalyst blinding does not ap-pear to compromise paradigm compatibility On the wholecomparative effectiveness designs with active lsquolsquousual carersquorsquocomparators show strong paradigm compatibility in WSRcontexts Multiarm designs with intra-paradigmatic facto-rial comparators also appear useful for comparing wholecomplex versus singularisolated TCIM practices Rando-mization remains the most common allocation approachacross controlled WSR studies with some form of stratifi-cation applied in most cases to increase balance The allo-cation alternatives engaged in a few studies (eg matchedcontrols preference-based allocation and design adaptiveassignment) are neither novel nor uniquely designed to ad-dress paradigmatic considerations in the TCIM field how-ever they each appear to have distinct advantages (andpotential disadvantages) for the WSR researcher
Matched control designs used in three studies have thepotential to produce results with internal validity similarto randomized trials at a lower cost and as McCulloch
S38 IJAZ ET AL
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
et alrsquos studies83120 demonstrate may be fruitfully used inretrospective designs that rely on existing patient dataPreference-based designs explicitly recognize patientsrsquodifferential choices of TCIM versus biomedical treat-ments strengthening studiesrsquo external validity In onesuch study baseline demographic characteristics differedsignificantly between preference-allocated cohorts com-promising internal validity90 This was however not thecase in other preference-allocated WSR studies reviewedtwo6981 of which designed specific strategies to preventsuch confounding
As exemplified in Ritenbaugh et alrsquos trial design adaptiveallocation (such as minimization) may match or exceed ran-domizationrsquos rigor while permitting implementation of in-novative experimental frameworks (eg lsquolsquostepped carersquorsquo86)Design adaptive assignment is furthermore cost-effective andlsquolsquosocially responsiblersquorsquo using lsquolsquothe smallest possible numberof study participants to reach definitive conclusions abouttherapeutic benefits and harmsrsquorsquo21 However as Aickin noteslsquolsquothe cultural bias in favor of randomization will probablyoutlast the failure to defend it on rational groundsrsquorsquo21 Re-searchers may thus be challenged to access funding for suchdesigns which may moreover be excluded from lsquolsquometa-analyses and structured evidence reviewsrsquorsquo As such theremay remain lsquolsquoa good argumentfor employing design-adaptation with a lsquolsquorandomizationrsquorsquo featurersquorsquo in WSR stud-ies21 such as in two of the reviewed exemplars8485
Uncontrolled quasi-experimental prendashpost designs bothlarge and small do not differ significantly from the con-trolled trials aside from the absence of comparator armsSuch paradigm-compatible designs may be more cost-effective than controlled studies particularly when based inexisting clinical settings and generate pragmatic outcomeswhile exploring controlled trial feasibility Scaled versionsof such studies exemplified by Hamre et alrsquos anthro-posophic chronic disease trial71 may themselves generatevaluable effectiveness data Large retrospective comparativedesigns have similar evidentiary potential whether relianton concurrent active control groups81 or electronicallymatched cohorts83
Adaptations to increase conventional study designsrsquoparadigm consistency are evident in the three n-of-1 trialsreviewed Conventional n-of-1 designs study authors ob-serve8082 readily accommodate interventions geared torapidly palliating symptoms but they fail to account forprogressive onset and extended carryover of treatment ef-fects associated with TCIM whole systemsrsquo emphasis onroot causes TCIM researchers may thus prudently considern-of-1 design adaptations a point scant raised in previousrelated literature134135 Huang et alrsquos actively-controlled n-of-1 design furthermore addresses challenges in recruitingpatients to placebo-controlled trials in the Chinese nationalcontext80 ethno-culturally situated considerations such asthese warrant greater attention by WSR scholars givenTCIMrsquos globalized context
Case series and case studies remain important WSR de-signs in their more explicit detailing of paradigm-specifictreatment considerations than is generally evident in otherstudy types Like n-of-1 trials they may draw attention toTCIM therapiesrsquo potential when lsquolsquousual carersquorsquo falls short74
and to understudied interventions with significant out-comes97 As Flower and Lewithrsquos study furthermore sug-
gests prospective case series may serve as feasibilitymodels for larger trial designs79
As proposed by early WSR advocates mixed methodsdesigns significantly increase studiesrsquo paradigm consistencyQualitative methods across the reviewed exemplars amplifyparticipant and clinician perspectives and suggest parame-ters for better outcome assessment tools However in lightof many TCIM whole systemsrsquo qualitative underpinningsthe dominance of quantitative methods across most WSRstudies reinforces the biomedically-dominant contexts inwhich TCIM researchers seek model validity in their re-search designs
Although some early RCT critics (eg Heron7) hadproposed participatory ethnographic designs as optimalmodes of TCIM research the ethnographic research modesadopted in just one study102 (and suggested in two oth-ers29126) are indeed unusual in biomedical clinical researchcontexts These studies move boldly from paradigm con-sistency toward paradigm specificity and their meth-odological propositions warrant careful attention Howethnographically-informed hybrid designs may fruitfullyenrich established clinical research approaches remains tobe seen as whole systems researchers carefully balance thepursuit of model validity with funding limitations and theirown resistance of TCIMrsquos biomedical co-optation
Part II interventions
This review undertakes a granular approach to analyzingthe features of WSR interventions both in terms of theirgeneral traits and in terms of the diagnostic and individu-alization strategies engaged As summarized in Figure 8and detailed in Figure 9 interventions across the reviewedstudies are typically complex multimorbid or multitar-get in focus behaviorally-focused and in some casesmultidisciplinary About half of the reviewed studies im-plement dual (multiparadigmatic) diagnoses and mosttreatments involve some form of individualization re-presenting a range of approaches across the individuali-zation spectrum
Complex interventions All but one92 of the reviewedWSR studies implement complex (ie multimodal andormulticomponent) interventions (Fig 9) treatments deliveredwithin particular paradigms in some cases exhibit dis-tinct traits In all studies of anthroposophic Ayurvedic and
FIG 8 Primary features of whole systems research in-terventions
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S39
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
naturopathic medicine care and in almost all Chinesemedicine studies interventions reflect the full range ofmultimodal treatments that typify these paradigms (Tables 1and 2) All studies reporting on complementaryintegrativemedicine interventions include lsquolsquousualrsquorsquo biomedical care asan adjunct to treatment from at least one additional wholesystems paradigm Participants assigned to preventivere-storative biomedical study interventions all received com-bined instruction or counseling in nutrition exercise andstress-reduction practices
Three Chinese medicine studies are not clearly multi-modal in character but their treatments include multiplecomponents (eg acupuncture with moxibustion82 multi-herb mixtures7980) Multicomponent interventions are alsoevident in studies centralizing manual therapies (eg mul-tiple types of chiropractic adjustments33 or various mas-sage techniques41101) as well as movement-based therapies(eg yogic poses + breathwork + visualizations104 multi-movement trsquoai chi series that concurrently target lsquolsquophysicalfunction balance and muscle strengthrsquorsquo103) Midwifery care
FIG 9 Interventions inwhole systems research
S40 IJAZ ET AL
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
in Forster et alrsquos study includes pre- intra- and postpartumcare components100
Behavioral interventions Behavioral interventions(Fig 9)mdashdesigned to facilitate patient implementation ofsalutogenic or preventive activities in their own livesmdashfeature in a significant majority of all studies reviewedAbout a quarter of these studies centralize behavioralapproachesmdashsuch as diet exercise stress managementmindndashbody practices andor movement-based therapiesmdashasprimary intervention(s)296072769497ndash99103ndash105 at timesalongside a standardized nutritional supplement or herbalproduct94105 Such behavioral interventions are either de-livered in a group setting29769899103104121 one-on-onewith a clinician729497105 or both29 In another group ofstudies similar types of behavioral interventions are deliv-ered secondarily as part of an individualized whole sys-tems treatment package constituted within the paradigmsof anthroposophic69ndash71 Ayurvedic7475 Chinese8183ndash86
chiropractic33 complementaryintegrative8890 or naturo-pathic939596 medicine
Individualization The vast majority of interventions inthe studies reviewedmdashwhether prospective or retrospectivemdashinclude some form of individualized treatment (Fig 9)
Generally-standardized designs are evident in all sixreviewed studies involving group-based interven-tions769899103104121 regardless of paradigm as well as inseveral nongroup based studies7780879296105 Exemplarswhose interventions are distinguished by their uncon-strained individualization include all of the reviewedanthroposophic69ndash71 and complementaryintegrative87ndash91
medicine trials each of which also involves team care thesingle chiropractic33 and energy medicine36 studies one93
(of five) naturopathic one74 (of eight) Ayurvedic andfour79818284 (of ten) Chinese medicine trials analyzedManualizedtailored studies include three (of five) natu-ropathic8594ndash96 five (of ten) Chinese medicine7880838586
and five (of seven) Ayurvedic29727375105 trials as well asthe single massage therapy101 and midwifery100 studiesreviewed Some studies falling generally under onecategoryrsquos auspices concurrently include features of an-other29367779878892105 (ie tailoredstandardized sub-components) Flower and Lewithrsquos Chinese medicinestudy for instance delivers a standard herbal formulationfor participantsrsquo lsquolsquoacutersquorsquo urinary tract infection usagealongside individualized (patient-specific) lsquolsquopreventa-tiversquorsquo herbal formulations79 Rioux et alrsquos study29 simi-larly implements a standardized yoga therapy componentalongside manualizedtailored Ayurvedic diet and life-style counseling
Manualized protocol developmentmdashas exemplified in Aliet alrsquos stand-alone publication41mdashgenerally occurred acrossstudies through expert consensus informed by paradigm-specific and peer-reviewed literatures Various manualizationapproaches are moreover evident among the reviewed ex-emplars Ritenbaugh et alrsquos trial for instance defines acu-puncture point lists and lsquolsquobase herbal formulasrsquorsquo for each of12 Chinese medicine diagnostic categories furthermore ar-ticulating optional subsets of pattern-specific acupoints andherbal additions for tailoring86 Cooley et alrsquos naturopathic
study by contrast more simply elaborates a set of predefinedparameters for tailored diet and lifestyle counseling94
Dual diagnosis In 21 of the 41 reviewed studies pa-tients are diagnosed both from a biomedical perspective andfrom within another paradigm(s) (Fig 9) These includeeach of the homeopathic and energy medicine studies 5 of 7Ayurvedic all 3 anthroposophic 1 of 5 complementaryin-tegrative 1 of 4 naturopathic and 8 of 10 Chinese medicinestudies reviewed
In 7 of these 21 studies little detail is provided beyond ageneral indication that multiparadigmatic diagnostics havetaken place3669ndash71758184 For instance Hullender Rubinet al note that each lsquolsquopatient was assessed according toTCM [Traditional Chinese Medicine] theoryrsquorsquo providingthe basis for a lsquolsquodetailed WS [whole systems]-TCM treat-ment planrsquorsquo81 Similarly Hamre et al refer to a set ofanthroposophy-specific principles (lsquolsquoformative force sys-temsrsquorsquo) a paradigm-specific lsquolsquoconstitutionalrsquorsquo diagnosticprocess and a set of distinct anthroposophic lsquolsquomedicationsand nonmedication therapiesrsquorsquo but do not detail the specificanthroposophic diagnoses made for study patients
The remaining 14 of the 21 identified dual diagnosisstudies explicitly identify the primary paradigm-specificdiagnoses given to participants All patients in JacksonrsquosChinese medicine study are for example lsquolsquodiagnosed with amixture of two predominant syndromes Liver Qi Stagnationand Kidney Deficiencyrsquorsquo In each of these 14 studies patienttreatments are individualized on the basis of paradigm-specific diagnoses A few moreover detail (typically intable- or appendix format) specific treatment protocols re-lated to such diagnoses29727880838586 Brinkhaus et al forinstance delineate a core set of acupuncture points and baseherbal formulation for all study patients specifying addi-tional points and herbal additions for each of five specificChinese medicine diagnoses78 In four cases72737980 studyauthors furthermore provide a detailed breakdown of allpatientsrsquo paradigm-specific diagnoses Study inclusion cri-teria in another four29737792 studies rely on paradigm-specific diagnoses Participants in Rioux et alrsquos study forinstance all exemplify one of two (kapha-aggravated)Ayurvedic constitutionimbalance profiles persons withother Ayurvedic diagnostic profiles are designated lsquolsquoineli-giblersquorsquo as paradigm-specific etiology lsquolsquofor these individualswould entail a causally distinct trajectoryrsquorsquo29
In addition four studies explicitly address intra-trialconsistency in the subjective determination of paradigm-specific diagnoses Kessler et alrsquos study relies on a team offour Ayurvedic practitioners to reach consensus on diag-nostic and treatment parameters for lsquolsquothe first 30 patientsrsquorsquo75
Similarly two Chinese medicine physicians lsquolsquoindependentlyassessedrsquorsquo each patient in Huangrsquos 2018 trial calling on athird lsquolsquodistinguished veteran doctor of TCMrsquorsquo to resolve anycontroversy between them A secondary publication125 as-sociated with Ritenbaugh et alrsquos Chinese medicine study86
details usage of a standardized questionnaire accompaniedby a clinician training process to enhance inter-rater reli-ability Azizi et alrsquos study notes its reliance on a singlediagnostician lsquolsquoto ensure uniform diagnosisrsquorsquo77 other stud-ies297282 also have just one diagnostician but do not linkthis point to the issue of paradigm-specific diagnostic con-sistency
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S41
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Multitargetmultimorbid interventions All of the re-viewed studies have clinical foci outcome measures andorintervention designs that are clearly multimorbid multi-target or both (Fig 9)
Some studies explicitly address more than one biomedicaldiagnostic category (eg cardiovascular disease and de-pression98 multiple chronic illnesses71) or nonbiomedicaldiagnoses for complex comorbid pathologies (eg a Chi-nese medicine diagnosis of lsquolsquoDamp-Heat in the Bladderrsquorsquocompounded in some patients with lsquolsquoSpleen Qi deficiencyand Liver Qi stagnationrsquorsquo andor lsquolsquoKidney deficiencyrsquorsquo79)Other studies set aside a singular disease-based emphasisin favor of multitarget conceptions of wellness implied byconstituting (for example) lsquolsquomedically-unexplained symp-tomsrsquorsquo84 or health-related QoL8991 as their primary clini-cal foci
Moreover (as detailed further on and shown in Figs 10and 11) almost two-thirds of the reviewed studies usemodes of outcome assessment designed to evaluate QoLandor psychosocial wellness parameters Such toolsmdashwhich typically assess for such health concerns as lsquolsquopainfatigue nausea depression anxiety drowsiness shortness ofbreath appetite sleep and feeling of well-beingrsquorsquo as well aslsquolsquophysical role emotional cognitive and social function-ingrsquorsquo89mdashare clearly multitarget in their focus
Even among the small number of studies that focus on asingular biomedical diagnosis and use no QoL-related psy-chosocial or qualitative outcome measures6972101105121 theinterventions studied are not only multimodal but also be-havioral in design suggesting a broadly conceived (iemultitarget) salutogenic focus
Multidisciplinaryteam care Twelve reviewed studiesreport on team-based interventions in which practitionersfrom across more than one discipline deliver bilaterallycoordinated care to participants (Fig 9) Team care inter-ventions take place intraparadigmatically in threeanthroposophic69ndash71 two Ayurvedic2976 and three preven-tiverestorative biomedical studies9899121 In other wordsin these studies disciplinarily diverse providers from withina single paradigmatic system deliver different aspects ofcare (eg anthroposophic physician care with referralsto anthroposophic art movement andor massage thera-
pists) Conversely in four3589ndash91 (of five) complementaryintegrative medicine studies and the one study involvingconcurrent Ayurvedicyoga therapy care29 teams arecomposed of providers representing more than one healthcare paradigm
In three additional studies8187120 two of which are ret-rospective81120 nonbiomedical health care providers uni-laterally coordinate their interventions with biomedicaltreatment (eg Hullender Rubin et alrsquos study practitionerstime their Chinese medicine infertility treatments to coin-cide with IVF)81 Three other studies deliver un-coordinatedmultidisciplinary care in which Chinese medicine84 or na-turopathic9395 care act as independent adjuncts to lsquolsquousualrsquorsquobiomedical treatment
Model validity and practical considerations in designingWSR interventions Across exemplars the evaluated in-terventions are generally paradigm-specific representingcomplex real-world practice rather than isolated compo-nents thereof A group of intervention traits furthermoreemerges as paradigm-consistent in WSR contexts as shownin Figure 8 WSR interventions are almost universally mul-timorbidmultitarget complex and individualized often in-clude salutogenic behavioral therapies and multiparadigmaticdiagnoses and at times feature multidisciplinary care Ex-cepting dual diagnoses these individual characteristics arenot necessarily uncommon in complex clinical trial designsacross other health care disciplines It is that these traitsappear repeatedly together in a single study that distin-guishes WSR interventions from those in other fields
Through diverse approaches to therapeutic individuali-zation WSR studies furthermore implement paradigm-specific research interventions Some individualizationmodes appear specifically relevant to particular TCIM par-adigms producing tension between model validity and re-search rigor more broadly conceived Traditional (Chineseand Ayurvedic medicine) exemplars commonly engagemanualization with tailoring approaches to align patientcare with paradigm-specific diagnoses and associatedtreatment parameters However in the context of (for in-stance) naturopathic medicine manualizedtailored proto-cols limit cliniciansrsquo treatment decisions lsquolsquoto a greaterdegree than is typicalrsquorsquo in routine practice94 threatening
FIG 10 Outcome assessment trends inwhole systems research
S42 IJAZ ET AL
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
model validity Unconstrained individualization is arguablya more suitable approach here and is also repeatedly en-gaged in anthroposophic and complementaryintegrativemedicine exemplars While standardized and manualizeddesigns lend themselves readily to replicability and gener-alizability (key markers of external validity) this provesmore challenging when cliniciansrsquo treatments are uncon-strained
Regardless it should be emphasized that dual diagnosticsemerge as a unique design feature across a significant pro-
portion of WSR exemplars clearly distinguishing WSRfrom conventional biomedical research Studies that applymanualizedtailored protocols tend to more explicitly detailthe paradigm-specific diagnoses engaged Such detailingmay enhance external validity by facilitating study replica-tion Strategies to promote inter-rater reliability furthermoreemerge as significant vis-a-vis paradigm-specific diagnosesIn addition to the approaches used in a few reviewed ex-emplars whole systems researchers may refer to a growingmethodological literature in this area26125136
FIG 11 Outcome assessment in whole systems research
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S43
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Multidisciplinary care is evident in several WSR exemplarssome of which implement lsquolsquousual care plusrsquorsquo designs in whichTCIM care serves as a biomedical adjunct Such designs ac-curately represent the broader context of biomedical domi-nance and are typical features of real-world practice for manyTCIM clinicians therefore lsquolsquousual care plusrsquorsquo designs mayenhance some studiesrsquo external validity In terms of modelvalidity however team care interventions which study multi-disciplinary care from within a single7071 or two compatibleTCIM paradigms29 are significant in their lsquolsquoarticulationrsquorsquo66 ofTCIM whole systems as distinct autonomous disciplines
Part III outcome assessment
Across the WSR studies reviewed a range of quantitative(and to a lesser extent qualitative) measurement instru-ments were used to evaluate outcomes at various intervalsAs summarized in Figure 10 the majority of studies usedpre- and postmeasures of treatment impacts often alongsideintermittent and follow-up assessments Primary outcomemeasures were more frequently subjective than objectiveand adverse event reporting was common Figure 11 pro-vides a detailed graphical representation of primary andsecondary outcome measure type and usage discussed andcontextualized in what follows actual study results receiveno attention in this analysis
Reporting intervals Most of the reviewed prospectivestudies implement concurrent evaluations of several primaryand secondary outcomes with measurements taking placeboth before and after the intervention About two-thirdssecondarily report outcomes as measured at intermittent in-tervals during the intervention period two-thirds reportlsquolsquofollow-uprsquorsquo outcomes from posttreatment measurementsand just under one-third do both (Fig 11) The four reviewedretrospective studies report postoutcomes only81 althoughthe single case report75 and one case series111 furthermoreelaborate on treatment progress over the intervention period
Primary and secondary outcome measures Over 70of the prospective studies reviewed adopt subjectivemeasuresmdashand more specifically PROMsmdashto evaluatetheir primary outcomes (Fig 10) About one-third by contrastapply objective endpointsmdashsuch as blood-based biomarkersanthropometrics such as weight or health outcomes likesurvival or live birth ratesmdashas primary outcomes in somecases alongside PROMs (Fig 11) Of the range of PROMsused to evaluate primary outcomes condition-specificsymptom severity scales dominate across studies almost allof these are validated scales developed with reference tobiomedical healthdisease conceptualizations ValidatedPROMs measuring QoL and wellness-related scores alsoappear in most studies as secondary outcome measures andin three studies as a primary measure The aforementionedoutcome types also serve as secondary (or co-primary)measures in some studies as do the following
Patient-generated outcome measures in which partici-pating patients individually define the health- andwellness-related parameters being measured at timeswith clinician support
PROMs to measure treatment expectation and treat-ment satisfaction
Quasi-objective clinician-assessed tests of physicalfunction (eg walking or spinal flexion tests) or diseaseprogression (eg radiologic tests for rheumatoid ar-thritis progression)
Health andor economic outcomes including medica-tion usage health service utilization and work absen-teeism (Fig 11)
All studies that include a standardized behavioral inter-vention specifically track patient adherence
Notably two specific sets of validated QoL and wellnessmeasurement PROMs appear in multiple studies These are
(1) the Short-Form 36 (SF-36)70717596103 and an ab-breviated version thereof the SF-129091 genericpredetermined scales designed to gather QoL- andwellness-related data from patients137138 and
(2) the patient-generated quantitative outcome measuresknown as lsquolsquoMYMOPrsquorsquo139 (Measure Yourself MedicalOutcome Profile)7982889495 and lsquolsquoMYCaWrsquorsquo (Mea-sure Yourself Concerns and Wellbeing)89 the latterof which also gathers qualitative data from patients inthe form of an open-ended questionnaire item140
Finally the reviewed retrospective studies generally useobjective health events (eg live birth and deathsurvival)alongside other subjective and objective assessment ap-proaches to express their outcomes
Adverse event reporting Most reviewed studies includeadverse event reporting monitoring for which occurredthrough questionnairesurvey live during interventions bytelephone andor online (Fig 11) Several herbal medi-cine studies also sampled blood andor urine at baselineduring and after the intervention as a safety monitoringmechanism78ndash8086105118
Paradigm-specific outcome assessment Paradigm-specific instruments to measure study outcomes appear injust two of the reviewed studies Rioux et al uses custom-designed tools lsquolsquoto capture data in five lifestyle-related areasidentified by Ayurveda as potential contributors or impedi-ments to weight lossrsquorsquo29 Forster et alrsquos midwifery studysimilarly uses a custom-modified PROM that emphasizesdimensions of care uniquely central to the midwifery para-digm100 noteworthy given an elsewhere-identified absenceof such tools in that field141
That said four additional studies produce paradigm-specific727377109 outcomes using biomedically developedinstruments to evaluate symptom scores and other outcomesassociated with singular paradigm-specific diagnoses (eglsquolsquokidney and liver yin deficiency accompanied by liver yanghyperactivityrsquorsquo77) The ensuing results are uniquely relevantto those working within or evaluating the tenets of a studyrsquosdriving paradigm Bell et alrsquos 2012 use of nonlinear dy-namical analyses to reinterpret objective study outcomesalso produces results that uniquely refer to homeopathicmedicinersquos explanatory tenets108
Complex outcome evaluation models Aside from Bellet alrsquos108 use of complexity theory described above just afew studies employ distinct outcome analytic models thataddress the multidimensional data generated Consistent
S44 IJAZ ET AL
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
with biomedical research approaches six mixed methodsstudies actively triangulate qualitative with quantitativefindings (Fig 6) and a few studies with standardized be-havioral interventions2960121 correlate adherence withtreatment effectiveness measures Many studies concur-rently report on a variety of outcome measures in a singlepublication but do not directly draw connections betweenthem Some studiesmdashsuch as Forster et alrsquos midwiferyRCT100116mdashuse separate publications to report upon dif-ferent sets of measured outcomes (eg patient satisfactionvs cesarean section rates)
Three additional studies engage with ethnographically-informed modes of outcome assessment which deliberatelydraw attention to multiple clinical outcomes andor con-textualize participantsrsquo experiences over the course of(rather than at discrete endpoints in) a whole systems in-tervention
Aiming to evaluate relationships between separatelymeasured outcomes Rioux et alrsquos Ayurvedicyoga therapyweight loss study begins to model the mixed-methods con-cept of a lsquolsquotopographical data setrsquorsquo informed by the lsquolsquoan-thropological notion of thick descriptionrsquorsquo29 To this endRioux et al29 graphically plot an overview of 15 distinctclinical lsquolsquodata collection measuresrsquorsquo alongside each mea-surersquos specific lsquolsquotime points for collectionrsquorsquo The 2014 pub-lication referenced in this study reports on anthropometricand adherence outcomes as well as some qualitative resultswhile complete outcomes from the trial including paradigm-specific measures are published in this JACM whole-systemsspecial issue for the first time Welch et alrsquos study in turnexplicitly uses ethnographic methods to report on contextualfactors from the clinical environment reporting minimally ontreatment outcomes102 While lsquolsquothick descriptionrsquorsquo is simi-larly evident across most studies using qualitative methodsthe substudy associated with Ritenbaugh and colleaguestrial uniquely engages trial participants in a series of qual-itative interviews at intervals during the study ethno-graphically theorizing process-related findings regardingpatientsrsquo treatment lsquolsquoexpectations and hopesrsquorsquo126
Model validity and practical considerations in WSR out-come assessment Aligned with conventional biomedicalresearch norms most reviewed studies engage quantitativeoutcome measures to report their results and subjective ra-ther than objective measures dominate as primary assessmenttools Measuring outcomes of direct relevance to patients is ofcourse no longer atypical in pragmatic biomedical trialsoutside of the WSR world Further suggesting paradigmcompatibility most primary PROMs used in the reviewedstudies had been developed in biomedical contexts Howevera set of complex outcome measurement trends emerged incommon across multiple studies indicating a paradigmconsistent approach distinct from clinical research norms
Exemplars commonly use symptom severity PROMsalongside QoLpsychosocial measures with reference tomultiple endpoints (ie pre- post- intermittent and follow-up) Such an approachmdashcomplemented in a quarter of ex-emplars with treatment satisfaction measuresmdashclearly re-flects the patient-centered salutogenic underpinnings ofTCIM paradigms and an emphasis on progressive enduringtreatment impacts Repeated usage of some QoLwellness
PROMs (eg SF-12 SF-36 MYMOP and MYCaW) someof which have been developed by TCIM researchers sug-gests that these particular tools may be considered particu-larly paradigm consistent
Objective outcome measures are certainly not absentamong WSR exemplars but rarely appear to the exclusionof concurrent PROMs Moreover about half of all objectivestudy outcomes refer to considerations of direct significanceto patients (eg weight change live birth and survival)rather than being concerned primarily with biomedicallyconceptualized disease causation Only one reviewed ex-emplar uses objective primary outcomes with the explicitaim of establishing biomedical mechanisms of action73
Bell et alrsquos use of objective measures to assess primaryhomeopathic outcomes is noteworthy in the context of aresearch paradigm routinely dismissed in biomedical con-texts as physiologically implausible92 Other studies thatengage objective primary outcomes appear to do so torender their results comparable with conventional biomed-ical trials addressing the same chronic health conditions(Type II diabetes cardiovascular disease) a considerationreasonably geared toward external validity
In contrast to the widespread engagement of paradigm-specific interventions across the WSR studies reviewed rel-atively few reviewed studies engaged paradigm-specificoutcome measures Some scholars have advised that paradigm-specific outcome measures be avoided as primary variables inTCIM research as they may limit studiesrsquo external validitywithin biomedically dominant health systems142 Regardlessparadigm-specific outcome measures toolsmdashnot presentlyin widespread WSR usagemdashmay usefully differentiate theimpacts of TCIM interventions delivered on the basis ofparadigm-specific diagnoses as now discussed
Conventional PROMs are certainly useful in gatheringoutcomes from the patientrsquos perspective patient-generatedoutcome measures have further potential to capture ef-fects not preconceptualized by researchers Such toolshowever are not designed to evaluate changing pathologieswith reference to a particular TCIM systemrsquos indigenousconcepts
PROMs custom developed to align in paradigm-consistent and paradigm-specific ways with TCIM systemsrsquodistinct conceptions of health and disease may begin to fillthis gap143 Such toolsmdashwhich will ultimately require rig-orous validationmdashmay be based on qualitative researchoutcomes as proposed in Sutherland et alrsquos exemplar36
purpose innovated as in Rioux et alrsquos29 and Forster et alrsquos100
studies andor formulated from the rich bodies of paradigm-specific literature that inform TCIM care143144 The Self-Assessment of Change tool145 a validated paradigm-consistent patient-centered outcome measure developed by agroup of whole systems researchers in 2011146147 was notused in any of the reviewed exemplars Aligned with previousresearch on lsquolsquowhole person healingrsquorsquo148 and informed by thelived experiences of TCIM patients this PROM aims toevaluate the lsquolsquoemergentrsquorsquo effects of therapeutic interventions
lsquolsquobeyond those [effects] associated withspecific treat-ment goals including unanticipated outcomes and multi-dimensional shifts in overall well-being energy clarity ofthought emotional and social functioning lifestyle patternsinner life and spiritualityrsquorsquo146
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S45
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Elsewhere applied149150 use of this tool may notablyimprove WSR studiesrsquo reporting of lsquolsquowhole personrsquorsquo patientoutcomes146147 moving forward Clinician-reportedparadigm-specific outcome measures144mdashcombined withinter-rater reliability strategiesmdashwill also likely prove im-portant Inspiration to renew a centralized open repository ofvalidated paradigm-compatible and paradigm-specific out-come measures for WSR informed by previous work byCanadarsquos INCAM Research Network151 might be furtherdrawn from the biomedical PROMIS3 project
Furthermore the application of complex evaluationmodels will prove critical in bringing the WSR imperativeto fruition in line with pioneersrsquo vision of holistically con-textualized outcomes Although applications of programtheory have begun to be explored in TCIM clinical researchcontexts152 uptake of complex system science in WSR hasbeen not as readily undertaken as anticipated despite pub-lication of multiple theoretical works on the subject Se-curing funding for such complex designs remains aconsiderable challenge in this regard Designs that empha-size the study of lsquolsquoprocessrsquorsquo rather than lsquolsquooutcomesrsquorsquo remainto be fully implemented1653 although the relationshipsbetween the two may fruitfully be studied through Riouxet alrsquos lsquolsquotopographicalrsquorsquo dataset proposition29 Methods thatfurther interrogate lsquolsquoindividual differences rather than groupaveragesrsquorsquo53 will also likely prove important as wholesystems researchers seek to integrate the multiple syner-gistic aspects of holistic clinical interventions
Discussion and Conclusions
This scoping review of WSR methods represents a firstsynthetic consolidation of over 15 years of advances in adistinctive field of scientific inquiry At first glance WSRhas much in common with conventional clinical research Itsrange of study designsmdashwhether controlled or uncontrolledmdashgenerally represent adaptations upon (rather than reinventionsof) established research methods and its predominantlyquantitative outcome measurements echo those applied inbiomedical research
On the whole WSR designs align with established normssurrounding the evaluation of complex clinical interven-tions2 Related features include the application of lsquolsquoappro-priate methodological choicesrsquorsquo the use of relevantrandomization alternatives identification of a lsquolsquocoherenttheoretical basisrsquorsquo for intervention design the engagement ofmultiple rather than singular primary and secondary out-comes and at times the inclusion of economic evaluations2
Reviewed post-facto in light of the PRECIS-2 pragmaticexplanatory study continuumrsquos nine domains1 mostcomparativecontrolled WSR studies also exhibit considerablymore pragmatic design features geared toward evaluatingthe real-world effectiveness of particular therapeutic inter-ventions This is evident across studies in the enrolment ofpatients and clinicians in existing clinical settings broadinclusion of multimorbid participants high levels of inter-vention flexibility (ie individualization) and primary out-come measures directly relevant to patients (eg symptomseverity and QoL)
As this review equally demonstrates WSR is distin-guished by a set of unique features Studies centralize theepistemological and practical features of health care para-
digms distinct from conventional biomedicine Many WSRstudies rely on dual diagnoses supplementing reframing orreplacing biomedical concepts of health and disease withparadigm-specific diagnostic and etiologic concepts Com-plex salutogenic interventions are commonly tailored to thepatient on this basis using various individualization strate-gies Whole systems researchers as this work makes evi-dent have successfully innovated a range of strategies forachieving a paradigmatic-methodological fit that is lsquolsquomodelvalidityrsquorsquo
Such strategies variously include alignment with specificestablished research designs (lsquolsquoparadigm compatibilityrsquorsquo)modification of conventional methods (lsquolsquoparadigm consis-tencyrsquorsquo) andor innovation of novel research strategies(lsquolsquoparadigm specificityrsquorsquo) As summarized in Figure 12 andelaborated throughout this work model validityrsquos dimensionsappear differentially relevant to study design selection inter-ventions delivered and outcomes evaluated in WSR contexts
Although some of WSRrsquos key features are not themselvesunique taken together as a synergistic set of design featuresthey become notable for their holistic patient-centered ori-entation These features include recruitment of multimorbidparticipants delivery of multitarget therapies centralizationof subjective patient-reported outcomes diversified andmultiple measurements of treatment effects and concurrentengagement of mixed (quantitative and qualitative) meth-ods Reflecting on the vision articulated by WSR pioneersjust after the turn of the century it is clear that the field hassignificantly advanced and TCIM researchers now have abody of WSR exemplars from which to learn
Challenges of course remain At a 2010 roundtable dis-cussion WSR leaders debated how to contend with largebodies of quantitative and qualitative data implement de-signs addressed to complexity undertake trials of suffi-ciently powered size to reach meaningful conclusionsaccommodate interpractitioner differences in practice styleprovide training for new researchers locate publicationvenues for multidimensional studies and address scientificskepticism about the field53 Echoing some of these issuesthe current review additionally calls for greater emphasis onethnographically-informed designs inter-rater reliabilityand paradigm-specific outcomes
It is hoped that this review will serve as a primary re-source for researchers practitioners funders and policy-makers interested in the rigorous evaluation of TCIM aswidely practised The previous absence of a syntheticanalysis of the fieldrsquos advances has perhaps presented abarrier to WSRrsquos centralization in strategic plans at coreTCIM hubs such as the US National Center for Com-plementary and Integrative Health (NCCIH formerlyNCCAM) A principal element of the enabling statute fromthe US Congress to that agency was to examine the inte-gration of these lsquolsquosystems and disciplines with conventional
Paradigm Compatibility
Paradigm Consistency
Paradigm Specificity
STUDY DESIGN
INTERVENTIONS
OUTCOMES
FIG 12 Model validity in whole systems research
S46 IJAZ ET AL
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
medicine and as a complement to such medicine and into thehealth care delivery systemsrsquorsquo153
Regardless as recently as 2016 former NCCIH leader-ship resisted calls to support WSR on the premise that itwas not yet clear what types of methods might be appro-priate for this purpose lsquolsquoProtocols to domesticate thewildness of integrative personalizationrsquorsquo in the context ofcomplex TCIM care would be needed NCCIH leadershipargued at the time154 As this work clearly documents rig-orous WSR methods do indeed exist further they have beensuccessfully implemented
Securing funding to conduct innovative WSR studies iscertainly a prominent challenge that researchers in this fieldcontinue to face53 Researchers may elect to align with es-tablished methods such as lsquolsquopragmaticrsquorsquo lsquolsquocomplexrsquorsquolsquolsquocomparative effectivenessrsquorsquo and lsquolsquomixed methodsrsquorsquo to so-licit support for their work and are wise to adhere to es-tablished guidelines in these areas It is however importantto recall that WSR as a maturing scholarly discipline ex-tends beyond the aforementioned approaches The incon-sistent use of lsquolsquoWSRrsquorsquo and lsquolsquomodel validityrsquorsquo terminologyacross the reviewed studies suggests that the field as itstands could benefit from greater cohesion ISCMR an ac-tive global organization of TCIM scholars whose foundingmission was to advance WSR155 might advantageouslyrenew its role in this regard
Discussion of the WSR field in which individualized carecomprises a vital component would not be complete with-out reference to the emerging trend toward lsquolsquopersonalizedrsquorsquobiomedical treatment In contrast to TCIM providersrsquo ho-listic reliance on paradigm-specific diagnoses patient pref-erences and contextual factors to personalize care objectivegenomic testing is rapidly becoming the primary driver ofindividualization in biomedicine As Mazer a biomedicaldoctor astutely observes lsquolsquo[t]he rise of lsquopersonalizedmedicinersquo is ironically a continuation of [a] reductionistmodethat deconstructs an individual into her facelessgenetic componentsrsquorsquo156
WSR is ultimately a hybrid phenomenon that stretches theboundaries of biomedical research to better accommodatediverse holistic health care approaches At a historicalmoment when TCIM providers find their long-held valuesmdashpersonalized patient-centered care salutogenesis andprevention complex interventions and patient-reportedoutcomesmdashto have become buzzwords within biomedicinersquoshighest echelons the potential for co-optation is significantDespite evident challenges WSR advocates and leaders whoseek to advance the field must continue to insist that themultiple dimensions of health cannot be reduced to an ob-jective set of biomarkers and that the whole is far more so-phisticated than the sum of its most evidenced parts
Acknowledgments
The authors wish to acknowledge the following wholesystems research area experts who each contributed to thelist of clinical exemplars reviewed in this work HeatherBoon Scott Mist Barb Reece Cheryl Ritenbaugh andDugald Seely Many thanks also to Katya Korol-O-Dwyerfor her research assistance with the project The develop-ment of this article was supported in part through Grant4221 from the Lotte and John Hecht Memorial Foundation
Author Disclosure Statement
No competing financial interests exist
References
1 Loudon K Treweek S Sullivan F et al The PRECIS-2tool Designing trials that are fit for purpose BMJ 2015350h2147
2 Medical Research Council Developing and evaluatingcomplex interventions [homepage on the Internet] 2018Online document at httpsmrcukriorgdocumentspdfcomplex-interventions-guidance accessed October 21 2018
3 Northwestern University PROMIS [homepage on theInternet] 2018 Online document at httphealthmeasuresnetexplore-measurement-systemspromis accessed No-vember 13 2018
4 Rathert C Wyrwich M Boren S Patient-centered careand outcomes A systematic review of the literature MedCare Res Rev 201270351ndash379
5 Ritenbaugh C Verhoef M Fleishman S et al Wholesystems research A discipline for studying complemen-tary and alternative medicine Altern Ther Health Med200393186ndash189
6 Verhoef M Lewith G Ritenbaugh C et al Com-plementary and alternative medicine whole systems re-search Beyond identification of inadequacies of the RCTComp Ther Med 200513206ndash212
7 Heron J Critique of conventional research methodologyComp Med Res 1986112ndash22
8 Dossey L How should alternative therapies be evaluatedAn examination of fundamentals Altern Ther Health Med199516ndash10 79ndash85
9 Carter B Methodological issues and complementary ther-apies Researching intangibles Complement Ther NursMidwifery 20039133ndash139
10 Kiene H A critique of the double-blind clinical trial Part2 Altern Ther Health Med 1996259ndash64
11 Kiene H A critique of the double-blind clinical trial Part1 Altern Ther Health Med 1996274ndash80
12 Kienle G Albonico H-U Fischer L et al Complementarytherapy systems and their integrative evaluation Explore(NY) 20117175ndash187
13 Bothwell L Greene J Podolsky S Jones D Assesing thegold standardmdashlessons from the history of RCTs N EnglJ Med 201637422
14 Elder C Aickin M Bell I et al Methodological chal-lenges in whole systems research J Altern ComplementMed 200612843ndash850
15 Long A Outcome measurement in complementary andalternative medicine Unpicking the effects J AlternComplement Med 20028777ndash786
16 Paterson C Baarts C Launso L Verhoef M Evaluatingcomplex health interventions A critical analysis of thelsquooutcomesrsquo concept BMC Complement Altern Med 2009918
17 Fonnebo V Grimsgaard S Walach H et al Researchingcomplementary and alternative treatmentsmdashthe gatekeep-ers are not at home BMC Med Res Methodol 200777
18 MacPherson H Pragmatic clinical trials ComplementTher Med 200412136ndash140
19 Walach H Falkenberg T Fonnebo V et al Circular in-stead of hierarchical Methodological principles for theevaluation of complex interventions BMC Med ResMethodol 200661ndash9
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S47
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
20 Boon H MacPherson H Fleishman S et al Evaluatingcomplex healthcare systems A critique of four approachesEvid Based Complement Altern Med 20074279ndash285
21 Aickin M Beyond randomization J Altern ComplementMed 20028765ndash772
22 Verhoef M Casebeer A Hilsden R Assessing efficacy ofcomplementary medicine Adding qualitative researchmethods to the lsquolsquogold standardrsquorsquo J Altern ComplementMed 20028275ndash281
23 Bell I Koithan M Models for the study of whole systemsIntregr Cancer Ther 20065293ndash307
24 Lewith G The use and abuse of evidence-based medicineAn example from general practice Complement TherMed 19964144
25 Jonas W The evidence house How to build an inclusive basefor complementary medicine West J Med 200117579ndash80
26 Mathie R Roniger H van Wassenhoven M et al Methodfor appraising model validity of randomised controlledtrials of homeopathic treatment Multi-rater concordancestudy BMC Med Res Methodol 2012121ndash9
27 Mathie R van Wassenhoven M Jacobs J et al Modelvalidity and risk of bias in randomised placebo-controlledtrials of individualised homeopathic treatment Comple-ment Ther Med 201625120ndash125
28 Bornhoft G Maxion-Bergemann S Wolf U et alChecklist for the qualitative evaluation of clinical studieswith particular focus on external validity and model va-lidity BMC Med Res Methodol 200661ndash13
29 Rioux J Thomson C Howerter A A pilot feasibility studyof whole-systems Ayurvedic medicine and yoga therapyfor weight loss Glob Adv Health Med 2014328ndash35
30 Lad V Textbook of Ayurveda Volume I FundamentalPrinciples Albuquerque NM Ayurvedic Press 2002
31 Kienle G Albonico H-U Baars E et al Anthroposophicmedicine An integrative medical system originating inEurope Glob Adv Health Med 2013220ndash31
32 Maciocia G The Foundations of Chinese MedicineLondon United Kingdom Churchill Livingstone 1989
33 Elder C DeBar L Ritenbaugh C et al Comparative ef-fectiveness of usual care with or without chiropractic carein patients with recurrent musculoskeletal back and neckpain J Gen Intern Med 2018331469ndash1477
34 Souza TA Differential Diagnosis and Management for theChiropractor Protocols and Algorithms 5th ed Burling-ton MA Jones and Bartlett Learning 2016
35 Ben-Arye E Livne Aharonson M Schiff E Samuels NAlleviating gastro-intestinal symptoms and concerns byintegrating patient-tailored complementary medicine insupportive cancer care Clin Nutr 2015341215ndash1223
36 Sutherland E Ritenbaugh C Kiley L et al An HMO-based prospective pilot study of energy medicine forchronic headaches Whole- person outcomes point to theneed for new instrumentation J Altern Complement Med200915819ndash826
37 Hahneman S Organon of Medicine 6th ed (Translated byKunzli JNA Pendleton P) Boston MA JP TarcherInc 1982
38 Sankaran R The Substance of Homeopathy MumbaiIndia Homoeopathic Medical Publishers 1994
39 Marshall J Raynor M Myles Textbook for MidwivesLondon United Kingdom Churchill Livingstone 2014
40 Wardle J Sarris J Clinical Naturopathy An Evidence-Based Guide to Practice 2nd ed Sydney AustraliaChurchill Livingstone 2014
41 Ali A Kahn J Rosenberger L Perlman A Developmentof a manualized protocol of massage therapy for clinicaltrials in osteoarthritis Trials 2012131ndash6
42 Wayne P Kaptchuk T Challenges inherent to trsquoai chiresearch Trsquoai chi as a complex multicomponent inter-vention J Altern Complement Med 20081495ndash102
43 Rioux J Yoga therapy research A whole-systems per-spective on comparative effectiveness and patient-centredoutcomes Int J Yoga Therap 2015259ndash19
44 International Association of Yoga Therapists Educationalstandards for the training of yoga therapists [homepage on theInternet] 2017 Online document at httpscdnymawscomwwwiaytorgresourceresmgraccreditationmaterials2017_11_Updates-Ed_Stds2017_IAYT_Educational_Standapdf accessed November 16 2018
45 Miller M Crabtree B Duffy B et al Research guidelinesfor assessing the impact of healing relationships in clinicalmedicine Altern Ther Health Med 20039A80ndashA95
46 Aickin M Participant-centred analysis in complementaryand alternative medicine comparative trials J AlternComplement Med 20039949ndash957
47 Koithan M Bell I Niemeyer K Pincus D A complexsystems science perspective for whole systems of com-plementary and alternative medicine research ForschKomplmentarmed 2012197ndash14
48 Rioux J A complex nonlinear dynamic systems per-spective on Ayurveda and Ayurvedic research J AltComp Med 201218709ndash718
49 Howerter A Hollenstein T Boon H et al State-space gridanalysis Applications for clinical whole systems com-plementary and alternative medicine research ForschKomplmentarmed 201219(suppl 1)30ndash35
50 Andrews GJ Boon H CAM in Canada Places practicesresearch Complement Ther Clin Pract 20051121ndash27
51 Baars E Hamre H Whole medical systems versus thesystem of conventional biomedicine A critical narrativereview of similarities differences and factors that pro-mote the integration process Evid Based ComplementAlternat Med 201720174904930
52 Ostermann T Beer A Bankova V Michalsen A Whole-systems research in integrative inpatient treatment EvidBased Complement Alternat Med 20132013962729
53 Ritenbaugh C Aickin M Bradley F et al Whole systemsresearch becomes real New results and next steps J Al-tern Complement Med 201016131ndash137
54 Zick S Schwabl H Flower A et al Unique aspects ofherbal whole system research Explore (NY) 2011597ndash103
55 Hawk C Khorsan R Lisi A et al Chiropractic care fornonmusculoskeletal conditions A systematic review withimplications for whole systems research J Altern Com-plement Med 200713491ndash512
56 Mills P Patel S Barsotti T et al Advancing research ontraditional whole systems medicine approaches J EvidBased Complementary Altern Med 201722527ndash530
57 Daudt H van Mossel C Scott S Enhancing the scopingstudy methodology A large inter-professional teamrsquosexperience with Arksey and OrsquoMalleyrsquos framework BMCMed Res Methodol 2013131ndash9
58 Levac D Colquhoun H OrsquoBrien K Scoping studiesAdvancing the methodology Implement Sci 201051ndash9
59 Arksey H OrsquoMalley L Scoping studies Towards amethodological framework Int J Soc Res Methodol 2003819ndash32
S48 IJAZ ET AL
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
60 Ornish D Scherwitz L Billings J et al Intensive lifestylechanges for reversal of coronary heart disease JAMA19882802001ndash2007
61 Ornish D Scherwitz L Doody R et al Effects of stressmanagement training and dietary changes in treating is-chemic heart disease JAMA 198324954ndash59
62 Saunders B Sim J Kingstone T et al Saturation inqualitative research Exploring its conceptualization andoperationalization Qual Quant 2018521893ndash1907
63 Fran S The constant comparative analysis method outsideof grounded theory Qual Rep 2013181ndash25
64 Baer H Toward an Integrative Medicine Merging Al-ternative Therapies with Biomedicine Walnut Creek CAAltaMira Press 2004
65 Hollenberg D Muzzin L Epistemological challenges tointegrative medicine An anti-colonial perspective on thecombination of complementaryalternative medicine withbiomedicine Health Sociol Rev 20101934ndash56
66 Weeks J Colonialism health justice and (re)examinationof the value of traditional practitioners and licensed in-tegrative health professionals J Altern Complement Med201824301ndash303
67 Ijaz N Boon H Medical pluralism and the state Reg-ulatory language requirements for traditional acupunctur-ists in English-dominant diaspora jurisdictions SAGEOpen 2018AprilndashJune1ndash15
68 Scaturo D The evolution of psychotherapy and the con-cept of manualization An integrative perspective ProfPsychol 200132522ndash530
69 Hamre H Pham V Kern C et al A 4-year non-randomized comparative phase-IV study of early rheu-matoid arthritis Integrative anthroposophic medicine forpatients with preference against DMARDs versus con-ventional therapy including DMARDs for patients withoutpreference Patient Prefer Adherence 201812375ndash397
70 Hamre H Witt C Glockmann A et al Anthroposophic vsconventional therapy for chronic low back pain A prospec-tive comparative study Eur J Med Res 200712302ndash310
71 Hamre H Kiene H Glockmann A Ziegler R Kienle GLong-term outcomes of anthroposophic treatment forchronic disease A four-year follow-up analysis of 1510patients from a prospective observational study in routineoutpatient settings BMC Res Notes 20136269
72 DuBroff R Lad V Murray-Krezan C A prospective trialof Ayurveda for coronary heart disease A pilot studyAltern Ther Health Med 20152152ndash62
73 Joshi K Nesari T Dedge A et al Dosha phenotype spe-cific Ayurveda intervention ameliorates asthma symptomsthrough cytokine modulations Results of whole systemclinical trial J Ethnopharmacol 2017197110ndash117
74 Kessler C Stapelfeldt E Michalsen A et al The effect ofa complex multi-modality Ayurvedic treatment in a caseof unknown female infertility Forsch Komplmentarmed201522251ndash258
75 Kessler C Dhiman K Kumar A et al Effectiveness of anAyurveda treatment approach in knee osteoarthritis - arandomized controlled trial Osteoarthritis Cartilage 201826620ndash630
76 Mills P Wilson K Pung M et al The self-directed bio-logical transformation initiative and well-being J AlternComplement Med 201622627ndash634
77 Azizi H Liu YF Du L et al Menopause-related symp-toms Traditional Chinese medicine vs hormone therapyAltern Ther Health Med 20111748ndash53
78 Brinkhaus B Hummelsberger J Kohnen R et al Acu-pucture and Chinese herbal medicine in the treatment ofpatients with seasonal allergic rhinitis A randomized-controlled clinical trial Allergy 200459953ndash960
79 Flower A Lewith G A prospective case series exploringthe role of Chinese herbal medicine in the treatment ofrecurrent urinary tract infections Eur J Integr Med 20124e421ndashe428
80 Huang H Yang P Wang J et al Investigation into theindividualized treatment of traditional Chinese medicinethrough a series of n-of-1 trials Evid Based ComplementAlternat Med 201820185813767
81 Hullender Rubin L Opsahl M Wiemer K et al Impact ofwhole systems traditional Chinese medicine on in-vitrofertilization outcomes Reprod Biomed Online 201530602ndash612
82 Jackson A MacPherson H Hahn S Acupuncture fortinnitus A series of six n = 1 controlled trials Comple-ment Ther Med 20061439ndash46
83 McCulloch M Broffman M van der Laan M et alLung cancer survival with herbal medicine and vitamins ina whole-systems approach Ten-year follow-up data ana-lysed with marginal structural models and propensity scoremethods Intergr Cancer Ther 201110260ndash279
84 Paterson C Taylor R Griffiths P et al Acupuncture forlsquofrequent attendersrsquo with medically-unexplained symp-toms A randomised controlled trial Br J Gen Pract 201161e295ndashe305
85 Ritenbaugh C Hammerschlag R Calabrese C et al Apilot whole systems clinical trial of traditional Chinesemedicine and naturopathic medicine for the treatment oftemporomandibular disorders J Altern Complement Med200814475ndash487
86 Ritenbaugh C Hammerschlag R Dworkin S et alComparative effectiveness of traditional Chinese medi-cine and psychosocial care in the treatment of temporo-mandibular disordersmdashassociated chronic facial pain JPain 2012131075ndash1089
87 Attias S Keinan Boker L Arnon Z et al Effectiveness ofintegrating individualized and generic complementarymedicine treatments with standard care versus standardcare alone for reducing preoperative anxiety J Clin An-esth 20162954ndash64
88 Ben-Arye E Doweck I Schiff E Samuels N Exploring anintegrative patient-tailored complementary medicine ap-proach for chemotherapy-induced taste disorders Explore201814289ndash294
89 Shalom-Sharabi I Samuels N Lavie O et al Effect of apatient-tailored integrative medicine program on gastro-intestinal concerns and quality of life in patients withbreast and gynecologic cancer J Cancer Res Clin Oncol20171431243ndash1254
90 Wayne P Eisenberg DM Osypiuk K et al A multidis-ciplinary integrative medicine team in the treatment ofchronic low back pain An observational comparative ef-fectiveness study J Altern Complement Med 201824781ndash791
91 Witt C Ausserer O Baier S et al Effectiveness of anadditional individualized multi-component complemen-tary medicine treatment on health-related quality of life inbreast cancer patients A pragmatic randomized trialBreast Cancer Res Treat 2015149449ndash460
92 Bell I Howerter A Jackson N et al Effects of homeo-pathic medicines on polysomnographic sleep of young
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S49
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
adults with histories of coffee-related insomnia SleepMed 201112505ndash511
93 Bradley R Sherman K Catz S et al Adjunctive naturo-pathic care for type 2 diabetes Patient reported andclinical outcomes after one year BMC Complement Al-tern Med 20121244
94 Cooley K Szczurko O Perri D et al Naturopathic carefor anxiety A randomized controlled trial PLoS One20094e6628
95 Seely D Szczurko O Cooley K et al Naturopathicmedicine for the prevention of cardiovascular disease Arandomized clinical trial CMAJ 2013185E409ndashE416
96 Szczurko O Cooley K Busse J et al Naturopathic carefor chronic low back pain A randomized trial PLoS One20072e919
97 Bredesen D Amos E Canick J et al Reversal of cogni-tive decline in Alzheimerrsquos disease Aging (Albany NY)201681250ndash1258
98 Silberman A Banthia R Estay I et al The effectivenessand efficacy of an intensive cardiac rehabilitation programin 24 sites Am J Health Promot 201024260ndash266
99 Zeng W Stason W Fournier S et al Benefits and costs ofintensive lifestyle modification programs for symtopmaticcoronary disease in Medicare beneficiaries Am Heart J2013165785ndash792
100 Forster D McLachlan H Davey M-A et al Continuity ofcare by a primary midwife (caseload midwifery) increaseswomenrsquos satisfaction with antenatal intrapartum andpostpartum care Results from the COSMOS randomisedcontrolled trial BMC Pregnancy Childbirth 20161628
101 Perlman A Gould-Fogerite S Keever T et al Exploringmassage benefits for arthritis of the knee (EMBARK)Osteoarthritis Cartilage 201624S534
102 Welch P Thomas C Bingley A Working at the coalfaceUsing action research to study lsquointegrative medicinersquo inthe NHS Eur J Integr Med 2013575ndash82
103 Wang C Schmid C Iverson M et al Comparative ef-fectiveness of tai chi versus physical therapy for kneeosteoarthritis Ann Intern Med 201616577ndash86
104 Litchke L Liu T Castro S Effects of multimodal mandalayoga on social and emotional skills for youth with autismspectrum disorder An exploratory study Int J Yoga 20181159ndash65
105 Elder C Aickin M Bauer V et al Randomized trial of awhole-systems ayurvedic protocol for type 2 diabetesAltern Ther Health Med 20061224ndash29
106 Hamre H Witt C Glockmann A et al Health costs inanthroposophic therapy users A two-year prospectivecohort study BMC Health Serv Res 200661ndash8
107 Hamre H Kiene H Ziegler R et al Overview of thepublications from the anthroposophic medicine outcomesstudy (AMOS) A whole system evaluation study GlobAdv Health Med 2014354ndash70
108 Bell I Howerter A Jackson N et al Nonlinear dynamicalsystems effects of homeopathic remedies on multiscaleentropy and correlation dimension of slow wave sleepEEG in young adults with histories of coffee-induced in-somnia Homeopathy 2012101182ndash192
109 Brooks A Bell I Howerter A et al Effects of homeopathicmedicines on mood of adults with histories of coffee-relatedinsomnia Forsch Komplmentarmed 201017250ndash257
110 Oberg E Bradley R Hsu C et al Patient-reported expe-riences with first-time naturopathic care for type 2 dia-betes PLoS One 20127e48459
111 Bredesen D Reversal of cognitive decline A noveltherapeutic program Aging 20146707ndash717
112 DeBar L Elder C Ritenbaugh C et al Acupuncture andchiropractic care for chronic pain in an integrated healthplan A mixed methods study BMC Complement AlternMed 201111118
113 Elder C DeBar L Ritenbaugh C et al Acupunctureand chiropractic care Utilization and electronic med-ical record capture Am J Manag Care 201521e414ndashe421
114 Johnson E Dickerson J Vollmer W et al The feasibilityof matching on a propensity score for acupuncture in aprospective cohort study of patients with chronic painBMC Med Res Methodol 2017171ndash11
115 Penney L Ritenbaugh C DeBar L et al Provider andpatient perspectives on opioids and alternative treatmentsfor managing chronic pain A qualitative study BMC FamPract 2016171ndash15
116 McLachlan H Forster D Davey M-A et al Effects ofcontinuity of care by a primary midwife (caseload mid-wifery) on caesarean section rates in women of low ob-stetric risk The COSMOS randomised controlled trialBJOG 20121191483ndash1492
117 Hamre H Witt C Kienle G et al Long-term outcomes ofanthroposophic therapy for chronic low back pain A two-year follow-up analysis J Pain Res 2009275ndash85
118 Huang H Yang P Xue J et al Evaluating the individu-alized treatment of traditional Chinese medicine A pilotstudy of n-of-1 trials Evid Based Complement AlternatMed 20142014148730
119 Witt C Michalsen A Roll S et al Comparative effec-tiveness of a complex Ayurvedic treatment and conven-tional standard care in osteoarthritis of the kneemdashstudyprotocol for a randomized controlled trial Trials 201314149
120 McCulloch M Broffman M van der Laan M et al Coloncancer survival with herbal medicine and vitamins com-bined with standard therapy in a whole-systems approachTen-year follow-up data analysed with marginal structuralmodels and propensity score models Integr Cancer Ther201110240ndash259
121 Ornish D Brown S Scherwitz L et al Can lifestylechanges reverse coronary heart disease Lancet 1990336129ndash133
122 Rugg S Paterson C Britten N et al Traditional acu-puncture for people with medically unexplained symp-toms A longitudinal qualitative study of patientsrsquoexperiences Br J Gen Pract 201161e306ndashe315
123 Ali A Rosenberger L Weiss T et al Massage therapyand quality of life in osteoarthritis of the knee A quali-tative study Pain Med 2017181168ndash1175
124 Perlman A Ali A Njike V et al Massage therapy forosteoarthritis of the knee A randomized dose-finding trialPLoS One 20127e30248
125 Mist S Ritenbaugh C Aickin M Effects of questionnaire-based diagnosis and training on inter-rater reliabilityamong practitioners of traditional Chinese medicine JAltern Complement Med 2009157097
126 Eaves E Ritenbaugh C Nichter M et al Modes of hop-ing Understanding hope and expectation in the context ofa clinical trial of complementary and alternative medicinefor chronic pain Explore 201410225ndash232
127 Herman P Szczurko O Cooley K Seely D A naturopathicapproach to the prevention of cardiovascular disease
S50 IJAZ ET AL
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
144 College StreetToronto ON M5S 3M2
Canada
E-mail nadineijazgmailcom
WHOLE SYSTEMS RESEARCH SCOPING REVIEW S51
Cost-effectiveness analysis of a pragmatic multi-worksiterandomized clinical trial J Occup Environ Med 201456171ndash176
128 Daubenmier J Weidner G Sumner M et al The contri-bution of changes in diet exercise and stress managementto changes in coronary risk in women and men in themultisite cardiac lifestyle intervention program Ann Be-hav Med 20073357ndash68
129 Govil S Weidner G Merritt-Worden T Ornish D So-cioeconomic status and improvements in lifestyle coro-nary risk factors and quality of life The Multisite CardiacLifestyle Intervention Program Am J Public Health 2009991263ndash1270
130 Frattaroli J Weidener G Merritt-Worden T et al Anginapectoris and atherosclerotic risk factors in the multisitecardiac lifestyle intervention program Am J Cardiol 2008101911ndash918
131 Herman P Szczurko O Cooley K Mills E Cost-effectiveness of naturopathic care for chronic low backpain Altern Ther Health Med 20081432ndash39
132 Schultz A Chao S McGinnis H Integrative Medicine andthe Health of the Public A Summary of the February 2009Summit Washington DC National Academy of Sci-ences 2009
133 Onwuegbuzie A Johnson R Collins K Call for mixedanalysis A philosophical framework for combiningqualitative and quantitative approaches Int J Mult ResApproach 20093114ndash139
134 Johnston B Mills E n-of-1 randomized controlled trials Anopportunity for complementary and alternative medicineevaluation J Altern Complement Med 200410979ndash984
135 Ulbrich-Zurni S Teut M Roll S Mathie R The n-of-1clnical trial A timely research opportunity in homeopa-thy Homeopathy 201810710ndash18
136 Weinschenk S Gollner R Hollman M et al Inter-raterreliability of neck reflex points in women with chronicneck pain Forsch Komplmentarmed 201623223ndash229
137 Rand Corporation 36-Item short form survey instrument[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos36-item-short-formsurvey-instrumenthtml accessed September 252018
138 Rand Corporation 12-Item short form survey (SF-12)[homepage on the Internet] 2018 Online document athttpsrandorghealthsurveys_toolsmos12-item-short-formhtml accessed September 25 2018
139 University of Bristol Questionnaires [homepage on theInternet] 2018 Online document at httpbrisacukprimaryhealthcareresourcesmymopquestionnaires ac-cessed September 25 2018
140 University of Bristol Measure yourself concerns andwellbeing [homepage on the Internet] 2018 Online doc-ument at httpbrisacukprimaryhealthcareresourcesmymopsisters accessed September 25 2018
141 Smith V Daly D Lundgren I Eri T Benstoem C DevaneD Salutogenically focused outcomes in systematic re-views of intrapartum interventions A systematic reviewof systematic reviews Midwifery 201430e151ndashe156
142 Levin J Glass T Kushi L et al Quantitative methods inresearch on complementary and alternative medicine Amethodological manifesto Med Care 1997351079ndash1094
143 Chen R Wong C Lam T Construction of a traditionalChinese medicine syndrome-specific outcome measure
The kidney deficiency syndrome questionnaire (KDSQ)BMC Complement Altern Med 20121273
144 Zhang Y Liu B Zhang R et al Clinician-reported out-comes in traditional Chinese medicine A critical reviewOA Alt Med 2013115
145 Ritenbaugh C Welcome to the self-assessment of changewebsite [homepage on the Internet] 2018 Online docu-ment at httpselfassessmentofchangeorg accessed De-cember 18 2018
146 Thompson J Kelly K Ritenbaugh C et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies II Refining content validitythrough cognitive interviews BMC Complement AlternMed 201111136
147 Ritenbaugh C Nichter M Nichter A et al Developing apatient-centered outcome measure for complementary andalternative medicine therapies I Defining content andformat BMC Complement Altern Med 201111135
148 Koithan M Verhoef M Bell I White M Mulkins ARitenbaugh C The process of whole person healinglsquolsquounstucknessrsquorsquo and beyond J Altern Complement Med200713659ndash668
149 Lillehei A Halcon L Gross C Savik K Reis R Well-being and self-assessment of change Secondary analysisof an RCT that demonstrated benefit of inhaled lavenderand sleep hygiene in college students with sleep problemsExplore 201612427ndash435
150 Johnson M Bertrand S Fermon B Coleman J Pathwaysto healing Patient-centered responses to complementaryservices Glob Adv Health Med 201438ndash16
151 Kania A Verhoef M Dryden T Ware M IN-CAM out-comes database Its relevance and application in researchand practice Int J Ther Massage Bodywork 200928ndash16
152 Skovgaard L Bjerre L Haahr N et al An investigation ofmultidisciplinary complex health care interventionsmdashsteps towards an integrative treatment model in therehabilitation of people with multiple sclerosis BMCComplement Altern Med 20121250
153 Weeks J How NCCAMrsquos lsquolsquoreal worldrsquorsquo congressionalmandate is optimal for NCCAMrsquos 2010ndash2015 strategicplan [homepage on the Internet] 2010 Online documentat httptheintegratorblogcomindexphpoption=com_contentamptask=viewampid=606ampItemid=189 accessed No-vember 20 2018
154 Weeks J Into the light Reflections on whole-systemsresearch after a case series finds reversal of Alzheimerrsquos JAltern Complement Med 201622585ndash587
155 ISCMR About ISCMR [homepage on the Internet] 2016Online document at httpsiscmrorgcontentabout-iscmraccessed November 16 2018
156 Mazer B Do physicians dream of electric sheep Ex-amining the conflict between compassion and technologyYale J Biol Med 201891343ndash344
Address correspondence toNadine Ijaz PhD
Leslie Dan Faculty of PharmacyUniversity of Toronto
