Lithium 's place in treatment of mood disorder

Paul MontgomeryJoanne Yaffe David Albright

Completing a Campbell Systematic Review Society for Social Work and ResearchWashington, DCJanuary 15, 2012#Mission: The Campbell Collaboration (C2) helps people make well-informed decisions by preparing, maintaining and disseminating systematic reviews in education, crime and justice, and social welfare.Who are we?The Campbell Collaboration is an international research network that produces systematic reviews of the effects of social interventions. Campbell is based on voluntary cooperation among researchers of a variety of backgrounds.

What is a systematic review?

The purpose of a systematic review is to sum up the best available research on a specific question. This is done by synthesizing the results of several studies.

A systematic review uses transparent procedures to find, evaluate and synthesize the results of relevant research. Procedures are explicitly defined in advance, in order to ensure that the exercise is transparent and can be replicated. This practice is also designed to minimize bias.

Studies included in a review are screened for quality, so that the findings of a large number of studies can be combined. Peer review is a key part of the process; qualified independent researchers control the author's methods and results. Why systematic reviews?Evidence-based practice demands high quality evidence on which to base practice and policy decisions.

It is impossible for most social work practitioners to read all relevant research on a given topic.

Even if they could, many of the available studies are of low quality.

Studies accessible in social work journals are biased in favor of positive findings.

Through critical exploration, evaluation, and synthesis the systematic review separates the insignificant, unsound or redundant deadwood from the salient and critical studies that are worthy of reflection.

#Additional Rationale for Systematic Reviews:Decision Making

Practitioners need SRs to keep up with the literature in their field

Policy-makers need them to make sensible decisions

Researchers need them to identify gaps and establish protocols for future research.

Additional Rationale for Systematic Reviews:Efficiency

A review is usually less costly and time-consuming than embarking on a new study (although an SR may conclude that new studies are called for).

Reviews can also suggest or even establish that further research in a given area is not called for or even unethical.

Continuously updated reviews can shorten the time between research discoveries and real world implementation of effective diagnostic or treatment strategies.Additional Rationale for Systematic Reviews:Generalizability

Diversity of multiple reviewed studies provides an interpretive context not available in any one study. studies addressing similar questions often use different eligibility criteria for participants, different definitions of the problem, different variations of an intervention, different study designs, etc.Additional Rationale for Systematic Reviews:Examining Consistency of Relationships

Are effects in the same direction, and of the same general magnitudes, given the variation in study protocols?

Does consistency exist between studies of the same intervention or different dosages/intensities/classes of the same intervention?Additional Rationale for Systematic Reviews:Examining Inconsistency of Relationships

If an intervention strategy is effective in one setting and not in another, or among some people and not among others, this is important!

Furthermore, whether findings from a single study stand alone for any reason (uniqueness of study population, study quality or outcome measure) should be exploredAdditional Rationale for Systematic Reviews:Power

Quantitative SRs have the advantage of increased power. The advantage of increasing power is particularly relevant to conditions of relatively low event rates or when small effects are being assessed.Additional Rationale for Systematic Reviews:Precision

A measure of the likelihood of random errors in the results of a study, meta-analysis or measurement.

Confidence intervals around the estimate of effect from each study are a measure of precision.

The weight given to the results of each study in a meta-analysis (typically the inverse of the variance of the estimate of effect) is a measure of precision (i.e. the degree to which a study influences the overall estimate of effect in a meta-analysis is determined by the precision of its estimate of effect).Additional Rationale for Systematic Reviews:Accuracy

or at least an improved reflection of reality. Traditional (non-systematic) reviews can be criticized as biased and haphazard subject to idiosyncrasies of the reviewer

SRs and meta-analyses using scientific principles which aim to reduce random and systematic errors are superior.A Campbell systematic review must have: Clear inclusion/ exclusion criteria

An explicit search strategy

Systematic coding and analysis of included studies

Meta-analysis (where possible) How do Campbell systematic reviews differ from other systematic reviews?Campbell reviews must include a systematic search for unpublished reports (to avoid publication bias). Campbell reviews are usually international in scope. A protocol (project plan) for the review is developed in advance and undergoes peer review. Study inclusion and coding decisions are accomplished by at least two reviewers who work independently and compare results. Campbell reviews undergo peer review and editorial review.The Campbell Library of Systematic ReviewsCampbell reviews are published in the Campbell Library of Systematic Reviews, and are freely accessible online. Campbell systematic reviews can be updated as further relevant information emerges.

Completing a Campbell ReviewMajor Steps for a Campbell Systematic ReviewTitle Registration

Protocol Development

Protocol Publication

Review Development

Review Publication

Periodic UpdatesCampbell Systematic ReviewSelect a topic and formulate a focused question

Register the title

Prepare the protocol (Campbell Editorial Board & external peer review)

Electronic publication of protocol in The Campbell Library

Perform a comprehensive literature search

Prepare the Review (Editorial Board & external peer review)

Electronic publication of completed Review in the Campbell Library

Respond to comments/criticism

Keep the review up-to-date

Title RegistrationReserve the topicDo your homework!

ComponentsTitle: Intervention for Problem in PopulationE.g.: Does Eye Movement Desensitization and Reprocessing (EMDR) for Posttraumatic Stress Disorder (PTSD) in Combat VeteransBackground and objective of the reviewDefine the populationDefine the intervention(s)Outcome(s)MethodologyReview teamRoles and responsibilitiesPotential conflicts of interestSupportFundingPreliminary timeframe

ProtocolRoadmap for Review

ComponentsBackgroundObjective of the reviewMethodsAcknowledgementsReferencesAppendicesContribution of authorsDeclaration of InterestSources of SupportBackgroundDescription of the condition

Description of the intervention

How the intervention might work

Why it is important to do this reviewObjective of the ReviewExamples:The primary objective is to complete a SR of experimental and quasi-experimental studies of EMDR for combat veterans with PTSDThe secondary objective is to synthesize the results of these studies to asses the effect of EMDR on reducing PTSD in combat veteransMethodsCriteria for considering studies for the reviewTypes of studies (e.g., experimental, quasi-experimental)Types of participants (e.g., military combat veterans)Types of interventions (i.e., EMD/EMDR)Types of outcomes (e.g., level of PTSD symptoms via structured diagnostic interview or self-report questionnaire)

MethodsSearch methods for identification of studiesElectronic searches (e.g., EMBASE, MEDLINE, PILOTS)Search termsSearching other resources (e.g., correspondence, grey literature, hand-searching)

MethodsData Collection and analysisSelection of studies (stage 1 and 2)Data extraction and management (stage 3)Assessment of risk of bias within included studies (e.g., sequence generation, incomplete outcome data)Measures of treatment effect (how we will handle dichotomous and continuous data)Unit of analysis issuesDealing with missing data and incomplete dataAssessment of heterogeneity (e.g., Q-statistic, I statistic)Assessment of publication bias

MethodsData SynthesisSubgroup analysis, moderator analysis and investigation of heterogeneitySensitivity analysisAppendicesExamples:Screening: Stage 1Screening: Stage 2ExtractionDr Paul MontgomeryUniversity of Oxford

Dr Jane Dennis, School for Policy StudiesUniversity of Bristol

ValidityDid the review address a clearly focussed question?Were the right sort of studies selected?Was the search strategy explicit and comprehensive?Did the reviewers assess the quality of the identified studies?

1. Question formulation:Are non-drug CBT interventions effective in resolving the sleep problems of people aged over 60?ValidityDid the review address a clearly focussed question?Were the right sorts of studies selected?Was the search strategy explicit and comprehensive?Did the reviewers assess the quality of the identified studies?

2. Inclusion Criteria:Types of studies: Randomised controlled trialsTypes of participants Over 60s diagnosed with sleep problems via standardised measure screened to exclude dementia, depression and sleep apnoea or secondary insomnia (sleep disturbance caused by a psychiatric or medical disorder)

Sleep-CBT review Outcome measures: Sleep onset latency (SOL) Wake after sleep onset (WASO) Total wake time (TWT) Sleep duration (total) Early morning wakening Sleep efficiency Self-report of sleep satisfaction Scales related to sleep, eg the Pittsburgh Sleep Quality Index (PSQI); Stanford Sleepiness Scale Quality of life Outcomes were divided, where possible, into immediate post-treatment, medium term (3-12 months) and long-term (more than 12 months).ValidityDid the review address a clearly focussed question?Were the right sort of studies selected?Was the search strategy explicit and comprehensive?Did the reviewers assess the quality of the identified studies?3. Search strategy:Potential hazards:Publication BiasTower of Babel BiasUncritical use of electronic databasesPublication Biasthe tendency of investigators, reviewers and editors to differentially submit or accept manuscripts for publication on the direction or strength of the study findings.Cook DJ, Guyatt GH, Ryan G, Clifton J, Buckingham L, Willan A et al. Should unpublished data be included in meta-analyses? Current convictions and controversies. JAMA 1993; 269: 2749-2753Publication biasProspectiveRegistries of all trials in progressRetrospectiveTrial amnestyNegotiationFunnel plot

Unpublished dataControversialUnpublished data may not be a full or representative sample (Cook 1993)Publication is no guarantee of scientific quality (Oxman 1991)Cook DJ, Guyatt GH, Ryan G, Clifton J, Buckingham L, Willan A et al. Should unpublished data be included in meta-analyses? Current convictions and controversies. JAMA 1993; 269: 2749-2753Oxman AD, Guyatt GH, Singer J, Goldsmith CH, Hutchison BG, Milner RA et al. Agreement among reviewers of review articles. J.Clin.Epidemiol. 1991;44:91-98.Tower of Babel BiasStudies that find a treatment effect are more likely to be published in English-language journals.Opposing studies may be published in non-English-language journals.Gregoire G, Derderan F, Le Lorier J. Selecting the language of the publications included in a meta-analysis: is there a Tower of Babel Bias? J.Clin.Epidemiol. 1995;48:159-163Maintenance treatment review Search Strategy:Electronic databasesReference checkingHand-searchingPersonal communicationsSearch terms involvingsleep problemsthe older adultpsychological, physical, phototherapeutic, treatments

ValidityDid the review address a clearly focussed question?Were the right sort of studies selected?Was the search strategy explicit and comprehensive?Did the reviewers assess the quality of the identified studies?

4. Assessment of study quality:Should be routineDependent on sort of studies includedUse of rating scales with fixed cut-offs potentially misleading

Broad search results:2722 hits, of which 2486 were eliminated at once Of the 236 remaining, 79 were potential trials

Importance:Were the results similar from study to study?What is the overall result of the review?How precise are the results?Importance:Were the results similar from study to study?What is the overall result of the review?How precise are the results?1. HeterogeneityCombining apples and oranges?Sources of Heterogeneity:Differences in study participantsDifferent comparisonsDifferent duration of follow-upDifferent outcome measuresDifferences in methodological qualityTesting for heterogeneity:Look at plots of results

Formal tests of homogeneity. Importance:Were the results similar from study to study?What is the overall result of the review?How precise are the results?Results: CBT: Sleep onset latency(SOL) by diary

Results: CBT SOL (polysomnography)

Results: CBT SOL (at 12 months follow-up)

Results: CBT WASO (by diary)

Results: CBT WASO (by polysomnography) :

Results: CBT WASO (12 months follow-up)

Results: CBT Total wake time (by diary)

Results: CBT Total wake time (by polysom)

Results: CBT Total wake time (12 months followup)

Results: CBT Post-treat sleep duration (diary)

Results: CBT Post-treat sleep duration (polysom)

Results: CBT Post-treat sleep duration (12 months followup)

Results: CBT Early morning wakening (diary)

Results: CBT Early morning wakening (polysom)

Results: CBT Early morning wakening (12 months follow-up)

Results: CBT Sleep efficiency (by diary)

Results: CBT Sleep efficiency (by polysom.)

Results: CBT Sleep efficiency (at 12 months follow-up)

HeterogeneityEstimates from individual trials vary more than can be explained by the play of chance aloneN.B. Meta-analysis should NOT overlook important material differences in subgroup response

does it makes sense to average effects from the studies?

Heterogeneity approachesQualitative v. quantitativeQualitative reconsider poolingFixed v. random effects

Overallshort term improvements in sleep quality, sleep efficiency, and wake time and to long term improvement in self reported sleep duration Have these questions been asked before?YesWhat have other reviews in this area to say about non-drug treatments?Pallesen, Nordhus and Kvale (1998) Non-pharmacological interventions for insomnia in older adults.psychological interventions included in this meta-analysis produce significant and long-lasting changes in the sleep of older insomniacs.

However, they included non-randomised trials that included people without a diagnosis of a sleep problem and trials where more than 30% of participants dropped outMorin, Mimeault and Gagne (1999) Non-pharmacological treatment of late life insomnia Behavioural treatment methods are , for the most part, unknown and remain under-utilised

However this review includes non-randomised trials with participants as young as 17 as well as people who were not insomniacs

So in general, do non-drug treatments work for sleep problems in older adults?

Evidence currently is small, but the field is research activeConclusionsTitle Registration

Protocol Development

Protocol Publication

Review Development

Review Publication

Periodic UpdatesResourcesCochrane Handbook for Systematic Reviews of Interventions http://www.cochrane.org/resources/handbook/

Cochrane Open Learning Materials http://www.cochrane-net.org/openlearning/index.htm

EQUATOR Network (Enhancing the Quality and Transparency of Health Research) http://www.equator-network.org/index.aspx?o=1032

Centre for Reviews and Dissemination (CRD), University of York, UK http://www.york.ac.uk/inst/crd/methods.htm

David Wilson's page on "Meta-Analysis Stuff http://mason.gmu.edu/%7Edwilsonb/ma.html

All can be found via: http://www.campbellcollaboration.org/resources/research/Methods_Links.php Non-pharmacological treatments for sleep problems

in older adults_964001410.doc