The Meta-Narrative ReviewSystematic Reviewing Across DifferentParadigms
Henry W. W. Potts
Centre for Health Informatics & MultiprofessionalEducation (CHIME),
Institute of Epidemiology & Health Care, UCL
With thanks to Trish Greenhalgh, Geoff Wong & others
From Taylor & Potts (2008),Eur J Cancer 44(6):798-807cancer detection rate
• Systematic reviewing has evolved over time
• Meta-analysis for quantitative outcomes
• Some degree of methodological heterogeneitycan be handled with sub-group analyses
• Various ‘mixed methods’ approaches developedto combine qualitative and quantitative studies
Problems of heterogeneity multiply with more complex questions, with multipleoutcomes, varying systems and different methodologies – different paradigms
Various approaches developed to review broad methods…
Moran-Ellis et al. (Qual Res 2006;6(1):45-59):
“Researchers who advocate the use of multiple methods often write interchangeablyabout ‘integrating’, ‘combining’ and ‘mixing’ methods […] [This] obscures the differencebetween (a) the processes by which methods (or data) are brought into relationship witheach other (combined, integrated, mixed) and (b) the claims made for the epistemologicalstatus of the resulting knowledge.”
Yardley & Bishop (In The SAGE Handbook of Qualitative Research in Psychology,2007: pp. 352-67):
‘Composite analysis’: retain integrity of each method – integrate findings rather than‘mixing methods’
Noblit & Hare (Meta-ethnography: Synthesising Qualitative Studies, 1988):
Distinction between integrative and interpretive reviews
Lewis & Grimes (Acad Manage Rev 1999;24:672-90):
Meta-triangulation: building theory from multiple paradigms
Meta-narrative review – key citations
1st: Greenhalgh, Robert, Macfarlane et al., MilbankQ 2004;82:581-629 / expanded as Diffusion of Innovations in
Health Service Organisations: A Systematic Literature Review,Blackwell BMJ Books
Methods paper: Greenhalgh, Robert, Macfarlane etal., Soc Sci Med 2005;61:417-30
2nd(ish): Greenhalgh, Potts, Wong et al., Milbank Q2009;87:729-88.
Publication standards: Wong, Greenhalgh,Westhorp et al., BMC Med 2013;11:20
Meta-narrative review – key principles
Use a historical and philosophical perspective as a pragmaticway of making sense of a diverse literature
• Pragmatism
• Pluralism
• Historicity
• Contestation
• Peer review
Key questions (from Kuhn, “The structureof scientific revolutions”)
• What research teams have researched this area?
• How did they CONCEPTUALISE the problem?
• What THEORIES did they use to link problem withpotential causes and impacts
• What METHODS did they define as ‘rigorous’ and‘valid’?
Application more post-Kuhnian than Kuhnian
Explore the literature
Open-ended question
Meta-narrative review (how to get started)
Research tradition C
Evaluate, summarise
Qualitycriteria
Theoreticalbasis
Research tradition B
Evaluate, summarise
Qualitycriteria
Theoreticalbasis
Research tradition A
Evaluate, summarise
Qualitycriteria
Theoreticalbasis
Meta-narrative map of underpinning traditions
Researchtradition
Disciplinaryroots
Definition &scope
General formatof researchquestion
EPRconceptualisedas...
EPR userconceptualisedas...
Contextconceptualisedas...
Healthinformationsystems
(Evidence-based)medicine,computerscience
Study ofstorage,computation& transmissionof clinical data.Focus often onbenefits ofEPRs andhow to achievethem
What is impactof technology X(EPR, DSS,etc.) onprocess Y (e.g.clinicianperformance)and outcomeZ?
Container forinformationabout patient;tool foraggregatingclinical datafor secondaryuses
Rationaldecision-makerwhosecognitive abilitysets limits towhat can beachievedwithoutcomputers
Potentialconfounderwhich can be‘controlledfor’ if right studydesign used
Changemanage-ment(withinhealthservicesresearch)
(Evidence-based)medicine,socialpsychology,management
Study ofachievingorganisation-level change inHealthcare
How can weimprovedelivery ofhealthcare andsustainimprovement?
Innovation that,if implementedwidely andconsistently,will improveprocess andoutcome ofcare
‘Resistant’agent whomust be trainedandincentivised toadopt newtechnologiesand ways ofworking
External milieuof interactingvariables thatserve as barriersor facilitators tochange efforts
Informationsystems(positivist)
Businessstudies,psychology,computerscience
Study of howorganisationsdo or do noradopt &assimilateinformationsystems
What factors(independentvariables)account forsuccess orfailure(dependentvariable) ofinformationsystem X inorganisation Y?
Unwelcomechange,likely to beresisted, andwhich mayfit poorly withorganisationalstructures &systems
Potentialadopter whomay engagewith or resistchange;member ofgroup whosepower basemay beenhanced orthreatened
External milieuof interactingvariables thatmediate ormoderate therelationshipbetween inputand outputvariables
Researchtradition
Disciplinaryroots
Definition &scope
General formatof researchquestion
EPRconceptualisedas...
EPR userconceptualisedas...
Contextconceptualisedas...
Informationsystems(interpret-ivist)
Management,sociology,socialpsychology,anthropology
Study of howorganisationalmembers makesense ofinformationsystems &therebyassimilate them
What meaningsdoesinformationsystem X holdfor members oforganization Y?How to achieveaccommoda-tion betweendifferent views?
Socio-technicalchange thatholds differentmeanings fordifferentindividuals andgroups
Stakeholderwhose ‘framing’of the EPR iscrucial to itsassimilation.Agent whosecreativity canbe drawn uponin this effort
Scene & settingfor an unfoldingstory; webs ofmeaning inwhichorganisationalactors aresuspended
Informationsystems(technology-in-practice)
Organizationalsociology,socialpsychology,philosophy
Study of howsocialstructuresrecursivelyshape & areshaped byhuman agency,& role oftechnology inthis
What is therelationshipbetweenorganisationalactors,technology X,and theorganisation –and how doesthis changeover time?
Itinerary andorganiserwhose physical& technicalpropertiesstructure &supportcollaborativeclinicalwork
Knowledgeablecreative agentfor whom socialstructures bothcreatepossibilities &limit thepossible
Generated ®eneratedthrough interplayof action &structure. Doesnot study‘technologies’ &‘contexts’separately buttechnologies-in-use
Computersupportedcooperativework
Computerscience,softwareengineering,psychology,sociology
Study of howgroups ofpeople workcollaboratively,supported byinformationtechnology
How cantechnologiessupport thework of multipleinteractingpeople?
Contextualizedartefact
Agent whoworks to localgoals incollaborationwith others &creativelyovercomeslimitations offormal tools
External milieuor emergentproperty ofaction(constituted by& inextricablefrom an activityinvolving people& technologies)
Researchtradition
Disciplinaryroots
Definition &scope
General formatof researchquestion
EPRconceptualisedas...
EPR userconceptualisedas...
Contextconceptualisedas...
Criticalsociology
Sociology,philosophy
Study ofrelationshipbetweenpeople & socialorder, & role oftechnologies inthis
What socialstructures& powerimbalances areembedded intechnology X, &what impactdoes this haveon social roles/relationships?
Implicated inmicro & macropowerdynamics(because of linkbetweenknowledge& power)
Constrained bydominant socialStructures,which may bebuilt intotechnologies bydesigners
Social & materialconditions intowhich theunequal socialorder isinscribed;more or lessstable structureof macro socialrelations
Empiricalphilosophy(actornetworkcasestudies)
Philosophy,sociology,linguistics
Study ofsociotechnicalnetworks:considers howrelationships &power shiftwithin network
How hasnetwork,with its variousrelationships,work practices& risks,changed as aresult oftechnology X?
Actor in anetwork
Actor in anetwork
EPR & itscontexttogether formthe network; theone cannot bestudied withoutthe other
Systemsapproaches
Systems &managementresearch,drawing oncognitivepsychology,CSCW & ANT
Systemsperspective
What role doesthe EPR playwithin acomplexhealthcaresystem?
Component ofcomplex socio-technicalsystem whosefeatures &properties maycome togetherinunpredictableways
Component ofcomplex socio-technicalsystem whosefeatures &properties maycome togetherinunpredictableways
Complex,changingenvironment
Synthesis phase
Highlight similarities and differences in the findingsfrom different traditions
Contestation between the disciplines is data (andleads to higher order constructs)
Offer conclusions of the general format “incircumstances such as X, don’t forget to thinkabout Y”
Summary• Techno-utopianism
– Promoting (health informatics) or challenging (technology-in-practice, CSCW) it
• Recursivity• Different affordances of paper and electronic
– Health informatics stresses advantages of electronic; HCI/CSCWand technology structuration stress paper has advantages too
• Records support work / nature of co-operative work– Different participants’ view of others’ work / hidden work (feminist
critiques of hidden work) and changed visibility– Different people do different things & EPRs help or hinder people
differently– Impacts on power relationships
• EPRs are not an agreed and agreeable commonaccount, but communicative, boundary objects
con
text
for
all
tech
no
logy
stu
die
sm
ate
ria
lity
na
ture
of
me
dic
alw
ork
na
ture
of
EH
Rs
Crisis
Thomas Kuhn“The Structure of Scientific Revolutions” (1962)
Para
dig
msh
ift
Normal science
time
Pre-science Normal science
Thomas Kuhn“The Structure of Scientific Revolutions” (1962)
A discipline sees a repeated cycle of ‘crises’,leading to ‘paradigm shifts’, out of which emerges‘normal science’.
Greenhalgh, Robert, Macfarlane et al.“Diffusion of Innovations in Service Organizations: SystematicReview and Recommendations” (2004)
Different disciplines separately develop aparadigm and conduct ‘normal science’.
Rise and fall of diffusion research in rural sociology
0
5
10
15
20
25
30
35
40
41 44 47 50 53 56 59 62 65 68 71 74 77 80
Nu
mb
ero
fp
ub
licat
ion
s
DEVELOPED NATIONS DEVELOPING NATIONS
Rise and fall of diffusion research in health related fields
0
20
40
60
80
100
120
140
160
1975-1976
1977-1978
1979-1980
1981-1982
1983-1984
1985-1986
1987-1988
1989-1990
1991-1992
1993-1994
1995-1996
1997-1998
1999-2000
2001-2002
Nu
mbe
ro
fpu
blic
atio
ns
NURSING MEDICAL EDUCATION
EBM OR GUIDELINES DELIVERY OF HEALTH CARE
Greenhalgh, Potts, Wong et al.“Tensions and Paradoxes in Electronic Patient Record Research: ASystematic Literature Review Using the Meta narrative Method” (2009)
Reflections
• The piles are subjective (but let’s not pretend‘traditional’ systematic reviewing isn’t)
• Synthesis difficult
• Very different picture to traditional Cochrane/EBMapproach
• Rich array of theories and methods
• Systematic, but interpretive
End of talk – turn offthe computer.
Thank you for yourattention.
Ask me questions.
Henry Potts,[email protected]
Cite as… Potts HWW (2013). “The Meta-narrative Review: Systematic ReviewingAcross Different Paradigms.” At Mixed Method Evidence Synthesis:How to Combine Quantitative and Qualitative Evidence in Systematic Reviewsworkshop, University of Manchester/NICE Evidence Synthesis Network,Manchester, 12 Mar 2013.
References:
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O (2004).Diffusion of innovations in service organisations: Systematic literaturereview and recommendations for future research. Milbank Quarterly, 82,581-629.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R(2005). Storylines of research in diffusion of innovation: A meta-narrativeapproach to systematic review. Social Science & Medicine, 61, 417-30.
Greenhalgh T, Potts HWW, Wong G, Bark P, Swinglehurst D (2009).Tensions and paradoxes in electronic patient record research: Asystematic literature review using the meta-narrative method. MilbankQuarterly, 87(4), 729-88.
Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R (2013).RAMESES publication standards: Meta-narrative reviews. BMC Medicine,11, 20.