RESEARCH ARTICLE Open Access Effectiveness of online and ... › content › pdf › 10.1186... ·...

RESEARCH ARTICLE Open Access

Effectiveness of online and mobiletelephone applications (‘apps’) for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysisKatrina Witt1*, Matthew J. Spittal2, Gregory Carter3, Jane Pirkis2, Sarah Hetrick4, Dianne Currier2, Jo Robinson4

and Allison Milner5

Abstract

Background: Online and mobile telephone applications (‘apps’) have the potential to improve the scalability ofeffective interventions for suicidal ideation and self-harm. The aim of this review was therefore to investigate theeffectiveness of digital interventions for the self-management of suicidal ideation or self-harm.

Methods: Seven databases (Applied Science & Technology; CENTRAL; CRESP; Embase; Global Health; PsycARTICLES;PsycINFO; Medline) were searched to 31 March, 2017. Studies that examined the effectiveness of digital interventionsfor suicidal ideation and/or self-harm, or which reported outcome data for suicidal ideation and/or self-harm, within arandomised controlled trial (RCT), pseudo-RCT, or observational pre-test/post-test design were included in the review.

Results: Fourteen non-overlapping studies were included, reporting data from a total of 3,356 participants. Overall,digital interventions were associated with reductions for suicidal ideation scores at post-intervention. There was noevidence of a treatment effect for self-harm or attempted suicide.

Conclusions: Most studies were biased in relation to at least one aspect of study design, and particularly the domainsof participant, clinical personnel, and outcome assessor blinding. Performance and detection bias therefore cannot beruled out. Digital interventions for suicidal ideation and self-harm may be more effective than waitlist control. It isunclear whether these reductions would be clinically meaningful at present. Further evidence, particularly with regardsto the potential mechanisms of action of these interventions, as well as safety, is required before these interventionscould recommended.

Keywords: Digital, Application, Mobile telephone, Suicide ideation, Self-harm, Suicide

BackgroundSelf-harm, which includes intentional self-injury orself-poisoning irrespective of type of motivation and/or degree of suicidal intent [1], and attempted sui-cide, which refers to any intentionally self-inflictedself-injurious and/or self-poisoning behaviour withclear suicidal intent [2, 3], are associated with

suicidal ideation [4], non-fatal repetition of self-harm,and completed suicide [5]. Although in the UnitedStates of America (USA) distinction is frequentlymade between non-suicidal self-injury and suicidalbehavior [6], outside of the USA these terms are yetto receive widespread acceptance [7]. Instead, this re-view includes all forms of self-inflected self-injury orself-poisoning, irrespective of type of motivation ordegree of suicidal intent, which we refer to collect-ively as ‘self-harm’ [8].

* Correspondence: [email protected] Health, Turning Point, Eastern Health Clinical School, MonashUniversity, 54-62 Gertrude Street, Fitzroy, Victoria 3065, AustraliaFull list of author information is available at the end of the article

© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Witt et al. BMC Psychiatry (2017) 17:297 DOI 10.1186/s12888-017-1458-0

http://crossmark.crossref.org/dialog/?doi=10.1186/s12888-017-1458-0&domain=pdf

mailto:[email protected]

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/publicdomain/zero/1.0/

Effective face-to-face treatments for self-harm and sui-cidal ideation are available [9–11]. Many effective psycho-therapeutic options for the treatment of suicidal ideationand self-harm are resource intensive and require specialistclinician training, however. Resource limitations in somelow-to-middle income countries therefore limits accessto these professionals. Negative associations have, forexample, been found between per capita availability ofmental health services and suicide rates in a number ofcountries [12–14].Even in countries where access to psychotherapy for

suicidal ideation and self-harm are available, less thanone-half of those who self-harm receive treatment [15].There are a number of barriers to treatment for thosewho experience self-harm or suicidal ideation, including:beliefs that treatment is not warranted and/or is likely tobe ineffective, stigma, shame, negative prior experienceswith mental health care providers, and financial difficulties[15]. Young people in particular also cite a preference forself-management as a major obstacle to help-seeking forself-harm from clinical services [16].Given that an estimated 85% of the global population

is covered by a commercially-available wireless signal, andfurther, over five billion persons have a mobile phone sub-scription [17], digital interventions, including both onlineprograms and mobile telephone applications (‘apps’) (col-lectively referred to here as ‘digital interventions’), havebeen proposed as one mechanism by which the scalabilityof effective treatments for self-harm and suicidal ideationmay be improved [18]. Such interventions may also helpto overcome some of the attitudinal and structural bar-riers which prevent those who engage in self-harm fromaccessing clinical services [19], and may therefore repre-sent a valuable addition to a stepped care treatment modelin which access is improved through the provision oflower intensity ‘self-help’ interventions in addition to highintensity psychotherapeutic treatments for those whosesymptoms do not resolve.To date, the effectiveness of these digital interventions

has not been routinely evaluated. This is problematicgiven that the widespread implementation of these inter-ventions without appropriate evaluation of their useabil-ity and effectiveness could lead to the development andpromulgation of ineffective, or even harmful, interven-tions [20]. We therefore present a systematic review andmeta-analysis of the characteristics and effectiveness ofdigital interventions, including both online resourcesand mobile telephone apps for suicidal ideation and self-harm.

MethodsThe reporting of this systematic review and meta-analysis conforms to the Preferred Reporting Items for

Systematic Reviews and Meta-Analyses (PRISMA)guidelines [21].

Search strategy and selection criteriaWe searched for literature indexed in seven electronicdatabases covering a wide range of disciplines, including:computing and information technology (Applied Science& Technology) clinical trials (Cochrane Central Registerof Controlled Trials [CENTRAL]), medicine (Embase;Medline), psychology (PsycARTICLES; PsycINFO), andpublic health (Global Health) as well as a database thatspecifically indexes literature on interventions for theprevention of suicide (Centre for Research Excellence inSuicide Prevention [CRESP]). Clinical trial registries werealso searched using these same keywords to identify on-going studies. All databases were searched from their re-spective start dates until 31 March, 2017.A two stage process was used to locate relevant studies.

At the first stage, keywords inclusive of digital interven-tions and platforms were combined. Next, using standardBoolean operators, these were combined with keywordsrelated to suicidal ideation and self-harm. There were norestrictions either on publication language or status. Forfurther information on this electronic search strategy,please see the Appendix.Ancestry searches were also conducted by manually

screening the reference lists of included studies and pre-vious reviews [22–26]. Where information on eitherstudy design, methodology, or results was either unclearor missing from the published study, we sought clarifica-tion from corresponding authors.

Selection criteriaStudies were eligible for inclusion if: (1) the effectivenessof a standalone digital intervention (i.e., any online ormobile telephone app) was evaluated; (2) the interventionwas designed for the self-management of suicidal ideationor self-harm; (3) data on the effectiveness of the interven-tion with respect to any suicidal outcome (i.e., suicidalideation, repetition of self-harm, attempted suicide, orcompleted suicide) were reported; and (4) either a ran-domized, pseudo-randomized, or observational pre-test/post-test design was used.Studies were excluded if: (1) the program was not a

standalone digital intervention. Thus multimodal inter-ventions, in which the digital intervention was intendedto serve either as a complement or adjunct to traditionalface-to-face psychosocial therapy or required significantinput or involvement from face-to-face treating clini-cians, were not eligible for inclusion in this review. Briefcontact-based programs delivered via text messaging ore-mail services were also excluded as no form of activepsychosocial therapy is typically provided in the contextof these programs. Studies were also excluded if: (2) the

Witt et al. BMC Psychiatry (2017) 17:297 Page 2 of 18

intervention targeted gatekeepers (i.e., carers, otherhealth care professionals, or bystanders who may comeinto contact with suicidal persons); or (3) no data on sui-cidal ideation, self-harm, attempted and/or completedsuicide were reported. Descriptions of programs withoutdata on effectiveness were also excluded.Two authors (KW and AM) independently screened

studies for inclusion. Firstly, titles of all retrieved studieswere screened. Next, only studies meeting inclusioncriteria following a full text screen were retained. Anydisagreements regarding study eligibility were resolvedfollowing consensus discussions with the broader group ofreview authors. Once again, corresponding authorswere contacted to request further information on programdesign, study design, data analysis, or methodology asrequired.

Data analysisMethodological details were extracted from included stud-ies using a standardized extraction form by two authors(KW and AM) working independently of one another.Disagreements were resolved through consensus discus-sions. Methodological details included: research design,treatment setting, and outcome ascertainment. We alsoassessed whether those evaluating the intervention wereindependent from those who developed the intervention.Data on the primary outcome, suicidal ideation, were

also extracted by KW and AM working independently ofone another. To make maximal use of the available data,information was extracted irrespective of whether theseoutcomes were measured continuously, for example asscores on a psychometrically validated measure of sui-cidal ideation or as numbers of repeat episodes of self-harm, or categorically, as the proportion of participantsreporting thoughts of suicide or number of self-harmevents. Care was taken, however, to ensure the itemsused to determine these outcomes were comparablebetween studies.Secondary outcomes included: episodes of self-harm,

attempted suicide, and completed suicide measured ac-cording to self-report and/or hospital or medical records.Where a study reported outcomes at multiple time points,for example at six and 12 months’ follow-up, only data forthe longest follow-up period were extracted, in line withrecommendations [27].

Statistical analysisData on the primary outcome (i.e., suicidal ideation)were synthesized using one of two approaches. Wherethese were reported categorically (e.g., as a difference inproportions reporting thoughts of suicide in the inter-vention group as compared to the control group), oddsratios (OR) and their accompanying 95% confidence in-tervals were calculated.

Where outcomes were reported as scores on a con-tinuous scale, such as total scores on the Beck Scale ofSuicidal Ideation (BSSI; [28]), the mean difference (MD)or the standard mean difference (SMD), with an accom-panying 95% confidence interval, was calculated as ap-propriate. Specifically, where the same scale was used toinvestigate the outcome of interest in all studies includedin a meta-analysis, the MD was used. The SMD, on theother hand, was used where outcomes were measuredusing a variety of different scales as recommended [27].Pooled ORs, MDs, or SMDs were calculated by using

the DerSimonian and Laird random effects model [29],as implemented by RevMan for Windows, version 3.5.The impact of between-study heterogeneity was quanti-fied by the I2 statistic [30]. We interpreted an I2 statisticof ≥75% as indicating substantial levels of between-studyheterogeneity. Where we found evidence of this, weundertook sensitivity analyses to explore potential causesof this between-study heterogeneity, as outlined below.

Sensitivity analysesGiven recent findings suggesting that psychological inter-ventions that directly address suicidal ideation and self-harm are more effective in reducing attempted andcompleted suicide than interventions which indirectlytarget the symptoms associated with suicidality (e.g., anx-iety, depression, hopelessness) [31], we conducted sensi-tivity analyses to investigate whether digital interventionsdeveloped specifically for the self-management of suicidalideation or self-harm would be more effective in pre-venting suicidal ideation and repetition of self-harmthan interventions developed for the self-managementof depression symptomatology more generally.

Sub-group analysesThe inclusion of RCTs and non-randomized observationalstudies within a single meta-analysis is becoming increas-ingly common as reliance on RCT evidence alone can leadto knowledge translation bias [32]. RCTs, for example,typically recruit highly selected patient populations withlower risk profiles as compared to “real world” popula-tions [33]. The inclusion of results from pre-test/post-testobservational studies together with those from RCTs,however, can also lead to over-estimation of the treatmenteffect size [34]. To balance these two concerns, all studieswere eligible for inclusion in our review, irrespective ofstudy design; however, we did not pool data from RCTstogether with data from observational studies. Instead,we calculated separate sub-group analyses by study de-sign to investigate what impact, if any, study design hadon the magnitude of the effect size observed for theseinterventions.


Risk of biasRisk of bias was assessed using the Cochrane Collabor-ation tool for randomised and pseudo-randomised con-trolled trials [35] and, for controlled before/after designedstudies, the Risk of Bias In Non-Randomized Studies ofInterventions (ROBINS–I; [36]). The Cochrane Collabor-ation tool assesses bias in seven domains including: ad-equacy of the random sequence generation, allocationconcealment, blinding of participants, clinical personnel,and outcome assessors, as well as incomplete data, select-ive outcome reporting, and other bias. The ROBINS–I as-sesses bias in eight domains including: confounding,participant selection, classification of the intervention,departures from the intended intervention, missing data,measurement of outcomes, selection of the reported re-sults, and overall bias.

Role of the funding sourceThe funder had no role in study design, data collection,data analysis, data interpretation, or writing of the re-port. KW and AM had full access to all the data in thisstudy, and all authors had final responsibility for the de-cision to submit for publication.

ResultsThe electronic search strategy outlined in the Appendixinitially identified a total of 9033 potentially relevant re-cords. Four additional records were retrieved following

ancestry searching. After excluding duplicates, this figurereduced to 6382 records. Of these, 6063 records werescreened out after a review of their titles, whilst 305 re-cords were omitted following full text screening. A totalof 14 independent, non-overlapping studies were there-fore included in the review, reporting data on a total of3356 participants (Fig. 1; Table 1).

Study characteristicsThe majority of these studies had been conducted eitherin Australia (four studies: [37–40]) or the USA (four stud-ies: [41–44]). Two studies were conducted in Germany[45, 46], one in the Netherlands [47], one in Sweden [48],and one in Switzerland [49]. One further study recruitedparticipants through online forums and therefore in-cluded participants from a number of different coun-tries, including countries in North America, Europe, andAustralasia [50].Most studies evaluated the effectiveness of online pro-

grams [37–39, 41–49]. Only two evaluated the effective-ness of mobile telephone apps [40, 50]. Most programswere developed for the self-management of depression[37, 38, 42, 43, 45, 46, 48, 49]. However, as these studiesassessed the effectiveness of these programs on to at leastone suicide-related primary or secondary outcome (i.e.,suicidal ideation, self-harm, attempted and/or completedsuicide) they were nonetheless eligible for inclusion in thisreview. Only five programs were developed specifically for

Fig. 1 PRISMA flow diagram of included and excluded studies


Table

1Stud

ycharacteristics,metho

dologicald

etails,and

riskof

bias

assessmen

tforthe14

stud

iesinclud

edin

thisreview

Stud

yand

Reference

Num

ber

Stud

yDesign

Cou

ntry

NParticipantSource

Interven

tionCon

ditio

nCon

trol

Con

ditio

nMeasures

Risk

ofBias

INV

CTL

Christensen

,2013

[37]

RCT

Australia

3835

Callersto

Lifeline,a

charitable24

hteleph

one

coun

selling

serviceproviding

crisissupp

ortandsuicide

preven

tion.Person

swho

werehigh

lydistressed

and/

orsuicidalat

intake

were

exclud

ed.

MoodG

YM:Internetself-guide

dpsycho

educationprog

ram

coup

ledwith

self-gu

ided

iCBT

consis-tingof

sixmod

ules

delivered

once

aweekover

asixweekpe

riod.

Participantswere

allocatedto

await-listcontrol

cond

ition

.

Suicidalideation:scores

onfour

suicidalideatio

nitemsfro

mthe28-item

Gen

eralHealth

Questionn

aire

(GHQ–28).

Neither

participantsno

rclinicalpe

rson

nelw

ere

blindto

treatm

ent

allocatio

n.Prop

ortio

ncompletingtreatm

ent

notstated

.

Franklin,

2016

[50]

Recruitedfro

mon

line

forumsforself-injury.

Participantshadto

self-

repo

rttw

oor

moreep

isod

esof

self-cuttingin

pastmon

thto

beeligiblefor

participation.

Therapeutic

Evaluative

Condition

ing:App

-based

gamified

cond

ition

ing

prog

ram

inwhich

self-harm

relatedstim

uliw

ere

sequ

entially

pairedwith

adversestim

uli.Participants

wereen

couraged

tointeract

with

theprog

ram

asoftenas

necessaryover

aon

emon

thpe

riod.

Participantswere

allocatedto

anattentionalcon

trol

cond

ition

.


ontheSelf-Injurio

usThou

ghtsand

Behaviou

rsInterview,

indicatin

gnu

mbe

rof

days

inwhich

the

participanthadhad

thou

ghtsof

suicide.

Self-ha

rm:scoreson

the

Self-Injurio

usThou

ghts

andBehaviou

rsInterview,

indicatin

gthenu

mbe

rof

days

inwhich

the

participanthaden

gage

din

self-cuttingor

NSSI.

Neither

participantsno

rclinicalpe

rson

nelw

ere

blindto

treatm

ent

allocatio

n.Stud

y1

RCT

International55

59

Stud

y2

RCT

International62

69

Stud

y3

RCT

International75

84

Guille,2015

[41]

RCT

USA

100

99Med

icalinternscommen

cing

theirreside

ncyyear

recruited

from

oneof

twoho

spitals.

MoodG

YM:Internetself-guide

dpsycho

educationprog

ram

coup

ledwith

self-gu

ided

iCBT

consis-tingof

four

mod

ules

delivered

once

aweekover

afour

weekpe

riod.

Participantswere

allocatedto

anattentionalcon

trol

cond

ition

.

Suicidalideation:

prop

ortio

nof

participants

self-repo

rtingany

thou

ghtsof

suicide

accordingto

scores

onitem

nine

ofthePatient

Health

Questionn

aire–9

(PHQ–9).

Neither

participantsno

rclinicalpe

rson

nelw

ere

blindto

treatm

ent

allocatio

n.One

-half

(49%

)did

notcomplete

treatm

ent.

Hed

man,

2014

[48]

Pre-

test/

Post-

test

Swed

en1203

Recruitedfro

maun

iversity

hospitalp

sychia-tric

clinic.

iCBT:con

sistingof

10mod

ules

ofbe

haviou

ralactivationand

cogn

itive

therapyde

livered

over

a12

weekpe

riod.

N/A.


onitem

9of

theMon

tgom

ery

ÅsbergDep

ressionRatin

gScale(M

ADRS).

App

ropriate

adjustmen

twas

madeforpo

tential

time-varyingconfou

nding.

Neither

participantsno

rclinicalpe

rson

nelw

ere

blindto

treatm

ent

allocatio

n.One

-quarter

(25%

)did

notcomplete

treatm

ent.


Table

1Stud


dologicald

etails,and

riskof

bias

assessmen

tforthe14

stud

iesinclud

edin

thisreview

(Con

tinued)

Hill,2016

[44]

RCT

USA

4040

Recruitedfollowingan

advertisingcampaign.All

participantshadto

self-

repo

rtsign

ificant

levelsof

perceivedbu

rden

somen

ess

(i.e.,score

of17

orgreater

ontheInter-pe

rson

alNeeds

Question-naire

Perceived

Burden

-som

enesssub-scale).

LEAP:Interne

tself-gu

ided

prog

ram

inform

edby

the

InterpersonalThe

oryof

Suicide(prin

-cipallype

rceived

burden

-som

eness)consistin

gof

twomod

ules

delivered

over

atw

oweekpe

riod.

Participantswere

allocatedto

psycho

education.


ontheBeck

ScaleforSuicidal

Ideatio

n.

Participantswereno

tblindto

treatm

ent

allocatio

n.Con

trol

cond

ition

didno

taccoun

tforinterven

tion

timeor

engage

men

t(i.e.,

didno

tsatisfyattentional

controlreq

uiremen

ts).

Overon

e-third

(39.0%

)didno

tcompletetreatm

ent.

Kordy,2016

[46]

RCT

Germany

7780

Recruitedfro

mon

eof

six

psychiatric

departmen

ts.A

llpatientshadto

meet

diagno

sticcriteria

formajor

depression

accord-in

gto

the

Structured

Clinical

Interven

tionforDSM

-IVand

toself-repo

rtahistoryof

atleastthreepriorep

isod

esof

depression

tobe

eligiblefor

participation.

SUMMIT:Interne

tself-gu

ided

emotioncoping

prog

ram.The

numbe

rof

treatm

entmod

ules

was

notstated

,but

participants

wereen

couraged

toaccess

the

interven

tionas

oftenas

necessary

over

a12

mon

thpe

riod.

Participantswere

allocatedto

usual

psychiatric

care,

includ

ingph

armaco-

therapyand/or

psycho

therapyas

necessary.

Self-ha

rman

d/or

attempted

suicide:re-hospitalisation

followingan

episod

eof

self-injury

and/or

asuicide

attempt.

Neither

participantsno

rclinicalpe

rson

nelw

ere

blindto

treatm

entallocatio

n.Prop

ortio

ncompleting

treatm

entno

tstated

.

Mew

ton,

2015

[38]

Pre-

test/

Post-

test

Australia

484

Patientsdiagno

sedwith

depression

who

were

prescribed

acourse

ofiCBT

bytheirprim

arycare

provider.

iCBT:C

onsistingof

sixmod

ules

delivered

over

sixweekpe

riod.

N/A.


onitem

9of

thePatient

Health

Questionn

aire–9

(PHQ–9).

App

ropriate

adjustmen

twas

madeforpresen

ceof

missing

data

andfor

potentialtim

e-varying

confou

nding.

Neither

participantsno

rclinical

person

nelw

ereblindto

treatm

entallocatio

n.Alm

oston

e-half(43.2%

)didno

tcompletetreatm

ent.

Mortiz,2012

[45]

Pseudo

-RC

TGermany

105

105

Recruitedfro

mweb

sites

providingsupp

ortto

person

swith

depression

.

Deprexis:self-gu

ided

iCBT

prog

ram

consistin

gof

10mod

ules

delivered

over

aneigh

tweekpe

riod.

Participantswere

allocatedto

await-listcontrol

cond

ition

.


ontheGerman

lang

uage

versionof

theSuicide

BehaviorsQuestionn

aire-

Revised(SBQ

–R).

Neither

participantsno

rclinicalpe

rson

nelw

ereblind

totreatm

entallocatio

n.Participantswerealternately

allocatedto

theinterven

tion

andcontrolg

roup

s.One

-fifth

(21.9%

)didno

tcomplete

treatm

ent.

Robinson

,2016

[39]

Pre-

test/

Post-

test

Australia

21Second

aryscho

olstud

ents

presen

tingto

scho

ol-based

coun

sellorswith

suicidal

ideatio

nin

thepastmon

th.

Refra

me-IT:Interne

tself-gu

ided

iCBT

prog

ram

consistin

gof

eigh

tmod

ules

delivered

over

aneigh

tweekpe

riod.

N/A.


ontheSuicidalIdeatio

nQuestionn

aire-Jun

ior

(SIQ-JR)

ortheAdu

ltSuicidalIdeatio

nQuestionn

aire

(ASIQ)

(asapprop

riate

tothe

participant’s

age).

Analysesbasedon

treatm

ent

completerson

ly(N

=21).

Neither

participantsno

rclinicalpe

rson

nelw

ereblind

totreatm

entallocatio

n.


Table

1Stud


dologicald

etails,and

riskof

bias

assessmen

tforthe14

stud

iesinclud

edin

thisreview

(Con

tinued)

Tigh

e,2017

[40]

RCT

Australia

3130

Recruitedthou

gha

commun

ity-based

suicide

preven

tionorganisatio

n.All

participantshadto

iden

tify

asAbo

riginaland/orTorres

StraitIsland

er.A

llparticipants

also

hadto

repo

rtclinically

sig-nificantlevelsof

depres-

sion

(i.e.,scoresof

10or

greateron

thePatient

Health

Questionn

aire-9),psycho

-logicald

istress(i.e.,scoresof

25or

greateron

theKessler

Psycho

logicalD

istressScale),

andhadto

self-repo

rtsuicidalthou

ghtsin

thepast

twoweeks.

iBobbly:App

-based

self-gu

ided

CBT

prog

ram

consistin

gof

three

mod

ules

completed

over

asix

weekpe

riod.

Partici-p

antswere

also

encouraged

tocomplete

self-assessmen

tson

functio

ning

,suicidalthinking

,mind-fulness

exercises,self-soothing

activities,

andcultu

raleng

agem

ent

activities.

Participantswere

allocatedto

await-listcontrol

cond

ition

.


ontheDep

ressive

Symptom

sInventory-

Suicidality

Subscale

(DSI-SS).

Due

tochange

sin

inclusion

criteria

afterthe

commen

cemen

tof

thetrial,

one-qu

arter(26.2%

)of

the

includ

edparticipantsdid

notmeetthecriterio

nfor

frequ

ency

ofsuicidal

thou

ghts.Participants,

clinicalpe

rson

nel,and

outcom

eassessorswere

notblindto

treatm

ent

allocatio

n.Dataon

usage

was

availablefor65.6%

oftheinclud

edparticipants.

Ofthese,15.0%

didno

tcompletetreatm

ent.

VanSpijker,

2014

[47]

RCT

Nethe

rland

s116

120

Recruitedfollowingan

advertisingcampaign.All

participantshadto

repo

rtmild

tomod

eratesuicidal

ideatio

nto

beeligiblefor

participation.Participants

self-repo

rtingsevere

depression

wereexclud

ed.

iCBT:C

onsistingof

sixmod

ules

delivered

over

asixweekpe

riod

with

adde

dactivities

from

DBT,

PST,andmindfulne

ss-based

cogn

itive

therapy.

Participantswere

allocatedto

anattentionaland

onlinebibliotherapy.


ontheBeck

Suicide

Ideatio

nScale.

Neither

participantsno

rclinicalpe

rson

nelw

ereblind

totreatm

entallocatio

n.Aroun

don

e-half(44.0%

)did

notcom-plete

atleastthree

treatm

entmod

ules.

Van

Voorhe

es,

2009

[42]

Pre-

test/

Post-

testa

USA

83Ado

lescen

tsreceiving

treatm

entforde

pression

atprim

arycare

facilities

registered

with

oneof

five

major

Health

Care

Organisations

(HMOs).

CATCH-IT:Internetself-gu

ided

prog

ram

consistin

gof

14mod

-ules

ofCBT,IPT,and

comm-unity

resiliencyactivities

delivered

over

a12

weekpe

riod.

N/A.


onitem

9of

thePatient

Health

Questionn

aire–9

(PHQ–9).

Alm

ostall(96.4%)

participantscompleted

treatm

ent.Asdata

were

analysed

aspre-test/post-test

compare-son

sin

thisreview

,no

approp

riate

adjustmen

tcouldbe

madeforpo

tential

time-varyingconfou

nding.

Neither

participantsno

rclin-

icalpe

rson

nelw

ereblindto

treatm

entallocatio

n.

Wagne

r,2014

[49]

RCT

Switzerland

3230

Recruitedfro

mun

iversity

web

sites,ne

wspapers,po

ster

adverts,andfro

mreferralsto

ade

pression

self-he

lpgrou

p.Participantshadto

scoreat

least12

ontheBeck

Dep

res-

sion

Inventory-IIat

the

screen

inginterview.

iCBT:C

onsistingof

seven

mod

ules

delivered

over

aneigh

tweekpe

riod.

Participantswere

allocatedto

eigh

tweeks

ofmanualised

face-to-face

CBT

delivered

bya

registered

psycho

logist.


ontheBeck

Suicide

Ideatio

nScale.

Neither

participantsno

rclinicalpe

rson

nelw

ereblind

totreatm

entallocatio

n.Despite

rand

-omisation,a

sign

ificantlyhigh

ernu

mbe

rof

females

wereallocatedto

theinterven

tion.One

-fifth

(22.0%

)didno

tcomplete

treatm

ent.


Table

1Stud


dologicald

etails,and

riskof

bias

assessmen

tforthe14

stud

iesinclud

edin

thisreview

(Con

tinued)

Whiteside

,2014

[43]

Pre-

test/

Post-

test

USA

39Prim

arycare

patients

receivingtreatm

entfora

new

episod

eof

depression

andwho

wereno

talready

receivingeither

pharm-

acothe

rapy

orpsycho

-therapy.Thosewho

had

received

treatm

entfora

depressive

episod

ein

the

lastsixmon

thswereno

teligibleto

participate.

Thrive:iCBT

prog

ram

consistin

gof

threemod

ules

delivered

over

athreeweekpe

riod.

Partici-p

ants

werealso

encouraged

toaccess

asecure

messaging

platform

atleaston

ceaweekforeigh

tweeks

toreceiveamoti-vational

interven

tionfro

maqu

alified

iCBT

coach.

N/A.


onitem

13of

the

Symptom

Che

cklist-20.

Noapprop

riate

adjustmen

twas

madeforpo

tential

time-varyingconfou

nding.

Neither

participantsno

rclinicalpe

rson

nelw

ere

blindto

treatm

ent

allocatio

n.One

-fifth

(22.0%

)didno

tcomplete

treatm

ent.

Abb

reviations:C

TLcontrol,CB

Tcogn

itive

beha

viou

ralthe

rapy

,iCB

Tinternet-based

cogn

itive

beha

viou

ralthe

rapy

,DBT

dialectical

beha

viou

rtherap

y,DSM

-IVDiagn

ostic

andStatistical

Man

ualo

fMen

talD

isorde

rs,fou

rth

revision

,INVinterven

tion,

IPTinterpersona

lthe

rapy

,NSSIn

on-suicida

lself-injury,P

STprob

lem-solving

therap

y,RC

Trand

omised

controlledtrial,TA

Utreatm

entas

usua

l,UKUnitedKing

dom,U

SAUnitedStates

ofAmerica

a Alth

ough

participan

tswererand

omised

totheinterven

tionan

dcontrolcon

ditio

ns,allpa

rticipan

tsin

thisstud

yreceived

access

tothedigitalintervention.

Forthepu

rposes

ofthisreview

,the

refore,d

atafrom

thisRC

Twas

treatedas

pre-test/post-test

compa

rison

sinstead


the self-management of suicidal ideation [39–41, 44, 47],and only one was developed for the self-management ofself-harm [50].In terms of study design, eight were randomised con-

trolled trials [37, 40, 41, 44, 46, 47, 49, 50], four were ob-servational pre-test/post-test studies [38, 39, 43, 48], andone was a pseudo-randomised controlled trial in whichsequential participants were alternately allocated to theintervention and control conditions [45]. For one RCT,although participants were randomised to the interven-tion and control conditions, all participants received ac-cess to the digital intervention [42]. For the purposes ofthis review, data from this RCT was therefore treated aspre-test/post-test comparisons instead. For two add-itional RCTs, outcomes at pre-test/post-test were alsoreported for the intervention group, enabling their inclu-sion in pre-test/post-test comparisons as well as in RCTcomparisons [45, 49]. However, to ensure results fromthese studies were not double-counted, analyses werepooled separately by study design for all outcomes re-ported in this review. In one further RCT, participantsassigned to the wait-list control condition crossed overto receive the intervention after six weeks [40]. To avoidcontamination from any ‘carry-over’ effects, we only ex-tracted data for the first six week period prior to cross-over as recommended [51].

Types of digital interventionsMost programs were developed by clinical psychologistsand/or psychiatrists with experience treating suicidalideation and/or self-harm [37, 39–45, 47–50], and wereevaluated by those who developed the intervention [37–40, 42, 44, 46–50]. In terms of therapeutic approach,most programs were based on the principles of cognitivebehavioural therapy (CBT). Some also included elementsof ‘third wave’ CBT, such as mindfulness [45], dialecticalbehaviour therapy [47], or mentalisation-based cognitivetherapy [47]; all of which hold the rationale that challen-ging thoughts, a principle feature of CBT, is less import-ant than understanding and accepting thoughts in anon-judgemental manner. Other programs included avariety of treatment approaches, including: acceptance-based therapy [40], problem-solving therapy [47], inter-personal therapy [42], mood monitoring [46], and crisisplanning [46]. Only one program utilised gamification inwhich participants were presented with a series of visualstimuli pairs designed to condition aversive reactions toself-harming thoughts or behaviours [50].

Types of control conditionsFor the eight RCTs and one pseudo-RCT designed trials,the interventions were compared against a number oftypes of control conditions, including: wait-list control[37, 40, 45], attentional control [41, 47, 50],

psychoeducation [44], treatment as usual [46], or face-to-face psychotherapy [49].

Ongoing studiesAn additional 10 ongoing studies were identified at thetime of the systematic search [52–59]. Further details ofthese ongoing trials are reported in Table 2.

Suicidal ideationFour studies reported data on the number of participantsself-reporting suicidal ideation. At post-intervention,there was some suggestion of a reduction in the propor-tion of participants self-reporting suicidal ideation in threeobservational pre-test/post-test studies (OR 0.36, 95% CI0.27 to 0.49, 3 studies, I2 = 0%, p < 0.0001; Fig. 2). How-ever, by the final follow-up assessment, data from oneRCT suggested no evidence of a treatment effect for theseinterventions (Fig. 2). There was no evidence of a signifi-cant difference in the magnitude of the treatment effect bystudy design for this outcome (χ2 = 0.61, df = 1, p = 0.44).As all four studies included in this analysis investigatedthe effectiveness of digital interventions specifically devel-oped for the self-management of depression symptoms(rather than suicidal ideation or self-harm specifically),sensitivity analyses could not be undertaken.One study covering three related RCTs reported infor-

mation on the frequency of self-reported episodes of sui-cidal ideation at post-intervention (for all three studies)and at the conclusion of the final follow-up assessment(for one of these three studies) [50]. However, no evi-dence of a treatment effect was found for these interven-tions at either time point (Fig. 3). As only one RCT wasincluded in this analysis, neither tests for subgroup dif-ferences nor sensitivity analyses could be undertaken.Eight studies reported data on suicidal ideation scores

using a number of different psychometric instruments,including: the BSSI [44, 47, 49], the Depressive SymptomInventory–Suicidality Subscale (DSI-SS; [60]) [40], theSuicide Behaviors Questionnaire–Revised (SBQ–R; [61])[45], the Suicidal Ideation Questionnaire–Junior (SIQ–J;[62]) [39], the four item suicidal ideation sub-scale ofthe General Health Questionnaire (GHQ–28; [63]) [37],and the suicidal ideation item of the Hopkins SymptomChecklist (HSCL–20; [64]) [43]. As raw means andstandard deviations were not reported for suicidal idea-tion scores in one RCT, data were instead estimatedfrom graphics in the original report for this study [37].At post-intervention, there was evidence these inter-

ventions were associated with a significant reduction insuicidal ideation scores in five RCTs (SMD -0.26, 95% CI-0.44 to −0.08, 5 studies, I2 = 0.0%, p = 0.005; Fig. 4).There was no evidence of a significant treatment effectfor these interventions in either one pseudo-randomisedcontrolled trial or four observational studies, however


Table

2Stud


dologicalinformation,andtrialreg

istrationinform

ationforthe10

ongo

ingstud

iesiden

tifiedby

thisreview

Stud

yand

Reference

Num

ber

Stud

yDesign

Cou

ntry

Target

NParticipantSource

Interven

tionCon

ditio

nCon

trol

Con

ditio

nMeasures

TrialR

egistration

Num

ber

INV

CTL

Boele,2014

[52]

RCT

Nethe

rland

s63

63Adu

ltswith

gradeII,III,orIV

glioma

(assessedaccordingto

WHOcriteria)

scoring12

orgreateron

theCen

ter

forEpidem

iologicalStudies

De-

pression

Scale.

Internet

self-gu

ided

prob

lem

solving

therapyconsistin

gof

fivemod

ules

delivered

once

aweekover

afive

weekpe

riod.

Waitlist.


ontheBeck

ScaleforSuicidalIdeatio

n(BSSI).

Nethe

rland

sTrial

Register:N

TR3223.

Eylem,2015

[53]

RCT

Nethe

rland

sandUK

100

100

Com

mun

ity-dwellingadultseither

born

inTurkey,orwith

oneor

both

parentsbo

rnin

Turkey

with

ascore

ofon

eor

greateron

theBeck

SuicidalIdeatio

nScale.

iCBT:C

onsistingof

betw

eensixand

eigh

tmod

ules

delivered

once

aweekover

asixweekpe

riod.

Waitlist.


ontheBeck


n(BSSI)and

theSuicidalIdeatio

nAttrib

uteScale

(SIAS).

Not

stated

.

Gum

pert,

2016

[Not

Publishe

d]

Pre-

test/

Post-

test

Swed

en43

Ado

lescen

tsdiagno

sedwith

NSSI,

defined

accordingto

sectionthree

oftheDSM

-5,w

ithat

leaston

eep

i-sode

ofNSSIo

ccurrin

gwith

inthe

pastmon

th.

Internet

emotionregu

latio

ntraining

.Noinform

ationon

thenu

mbe

rof

mod

ules

orthedu

ratio

nof

therapy

wereprovided

.

N/A.

Self-Harm:self-rep

ortedself-harm

ac-

cordingto

theDeliberateSelf-Harm

Inventory(DSH

I).Suicidalideation:scores

onthe

SuicidalIdeatio

nQuestionn

aire-

Junior

(SIQ-JR).

ClinicalTrialsRegister:

NCT02697019.

Kaslow

,2016

[Not

Publishe

d]

Pre-

test/

Post-

test

USA

25Referralsforbe

haviou

ralh

ealth

treatm

entthroug

han

outpatient

men

talh

ealth

servicefollowinga

suicideattempt

orclinically

sign

ificant

suicidalideatio

n.

RefliefLink:A

pp-based

moo

dtracking

andstress

managem

entinterven

tion

consistin

gof

mod

ules

ofrelaxatio

nexercises,gu

ided

med

itatio

n,pro-

gressive

re-laxatio

n,andmindfulne

ssskillstraining

.Noinform

ationon

the

numbe

rof

mod

ules

orthedu

ratio

nof

therapywereprovided

.

N/A.

Suicidalideationan

dbeha

viours:

measuredaccordingto

scores

ontheColum

biaSuicideSeverity

Ratin

gScaleScreen

er(C-SSRS)

aswellasthenu

mbe

rof

participants

self-repo

rtingep

isod

esof

self-injury.


NCT02691221.

Müh

lmann,

2017

[54]

RCT

Den

mark

175

175

Com

mun

ity-dwellingadultsscoring

threeor

greateron

theBeck

Scale

forSuicidalIdeatio

n.

iCBT:C

onsistingof

sixmod

ules

delivered

once

aweekover

asix

weekpe

riod.

Waitlist.


ontheBeck


n(BSSI)and

theSuicidalIdeatio

nAttrib

uteScale

(SIAS).

Self-Harm:n

umbe

rof

participants

with

aho

spitaland

/ormed

ical

record

forself-harm

.Suicide:nu

mbe

rof

suicidede

aths

accordingto

anatio

nalm

ortality

register.


NCT02872610.

Perry,2015

[55]

cRCT

Australia

1600

(30

scho

ols)

Com

mun

ity-dwellingadoles-cen

tsen

rolledin

theirfinalyear

ofhigh

scho

ol.

SPAR

X-R:Internet

gamified

iCBT

prog

ram

consistin

gof

seven

mod

ules

delivered

once

aweekover

sevenweeks.

Atten

tion

placeb

o.Suicidalideationan

d/or

attempted

suicide:scores

onthesuicidal

thou

ghts,p

lans,and

attempted

suicideitemsof

theYo

uthRisk

Behaviou

rSurvey

(YRBS).

AustralianandNew

ZealandClinicalTrials

Register:

ACTRN12614000316606.

Robinson

,2014

[56]

cRCT

Australia

170

(28

scho

ols)

Com

mun


tsen

rolledin

high

scho

olself-

repo

rtinganysuicidalideatio

nover

thepastmon

th.

Refra

me-IT:iCBT

prog

ram

consistin

gof

eigh

tmod

ules

delivered

over

a10

to12

weekpe

riod.

TAU.


onthe

SuicidalIdeatio

nQuestionn

aire

(SIQ).

Self-Harm

andAttempted

Suicide:

scores

onaspecifically-develop

edinstrumen

t.

AustralianandNew


Register:

ACTRN12613000864729.

USA

>2500

TAU.


Table

2Stud


dologicalinformation,andtrialreg

istrationinform

ationforthe10

ongo

ingstud

iesiden

tifiedby

thisreview

(Con

tinued)

Simon

,2016

[57]

Zelen

RCT

Referralsto

anou

tpatient

men

tal

health

and/or

gene

ralm

edical

centre

who

repo

rtdaily

oralmost

daily

suicidalideatio

naccordingto

scores

onitem

9of

thePatient

Health

Questionn

aire-9.

Internet

DBT

with

aspecificfocuson

thede

velopm

entof

mindfulne

ssskills,no

n-judg

emen

talaccep

tance

ofem

otions,opp

osite

actio

n,and

pacedbreathing.

Thisprog

ram

will

consistof

four

self-pacedmod

ules.


ona

specifically

develope

dsix-item

ver-

sion

oftheColum

biaSuicideSever-

ityRatin

gScaleScreen

er(C-SSRS).


NCT02326883.

Stallard,

2016

[58]

Pre-

test/

Post-

test

UK

50Com

mun


tswith

ahistoryof

repe

ated

self-harm

who

arecurren

tlyreceivingon

-go

ingcare

inou

t-patient

child

and

adolescent

men

talh

ealth

services.

BlueIce:App

-based

toolbo

x,de

vel-

oped

inconsultatio

nwith

youn

gpe

oplewho

self-harm

,including

amoo

ddiary,moo

d-liftin

gexercises

draw

nfro

mCBT

andDBT,p

hysical

activities,and

mindfulne

ssexercises.

Part-icipantsarealso

encouraged

toup

load

andshareinspiratio

nalp

ho-

tosandmusicfiles.Participantswill

useBlueIcedaily

for10

weeks.

N/A.

Self-Harm:m

easuredaccordingto

self-repo

rt,and

GPrepo

rt,and

/or

med

icalrecords.

Not

stated

.

VanSpijker,

2015

[59]

RCT

Australia

570

Com

mun

ity-dwellingadults

repo

rtingcurren

tsuicidalideatio

naccordingto

item

fiveof

the

Colum

bia-SuicideSeverityRatin

gScale,bu

twho

dono

tself-repo

rtattemptingsuicidewith

inthepast

mon

th.

iCBT:C

onsistingof

sixmod

ules

delivered

once

aweekover

asix

weekpe

riod.

Atten

tion

placeb

o.Suicidalideation:scores

onthe

Intensity

ofSuicidalIdeatio

nsub-

scaleof

theColum

biaSuicideSever-

ityRatin

gScaleScreen

er(C-SSRS)

andtheSuicidalIdeatio

nAttrib

ute

Scale(SIAS).

AustralianandNew


Register:

ACTRN12613000410752.

Abb

reviations:C

TLcontrol,CB

Tcogn

itive

beha

viou

ralthe

rapy

,iCB

Tinternet-based

cogn

itive

beha

viou

ralthe

rapy

,cRC

Tclusterrand

omised

controlledtrial,DBT

dialectical

beha

viou

rtherap

y,DSM

-IVDiagn

ostic

andStatis-

tical

Man

ualo

fMen

talD

isorde

rs,fou

rthrevision

,DSM

-5Diagn

ostic

andStatistical

Man

ualo

fMen

talD

isorde

rs,fifthrevision

,GPge

neralp

ractition

er,INVinterven

tion,

IPTinterpersona

lthe

rapy

,NSSIn

on-suicida

lself-

injury,P

STprob

lem-solving

therap

y,RC

Trand

omised

controlledtrial,TA

Utreatm

entas

usua

l,UKUnitedKing

dom,U

SAUnitedStates

ofAmerica


(Fig. 4). The test for subgroup differences was non-significant suggesting there was no difference in magni-tude of the effect size by study design for this outcome(χ2 = 0.25, df = 2, p = 0.88). Sensitivity analyses includ-ing only those studies in which the intervention was spe-cifically developed for the self-management of suicidalideation or self-harm also did not materially affect theseresults (results not shown).Two RCTs reported data on suicidal ideation at the

final follow-up assessment [37, 44]. For one of these tri-als, however, data on suicidal ideation scores at finalfollow-up had to be estimated by review authors fromgraphics presented in the original trial report [37]. Datafrom these trials suggested these digital interventionswere not associated with a significant treatment effect

for suicidal ideation by this time point (SMD -0.34, 95%CI –0.70 to 0.01, 2 studies, I2 = 0.0%, p = 0.06; Fig. 4).Given that both studies included in this analysis wereRCTs, neither tests for subgroup differences nor sen-sitivity analyses could be undertaken. One furtherRCT presented data on outcomes at final follow-upfor the intervention group only [65]. Analyzing thesedata as pre-test/post-test comparisons suggested asignificant reduction in suicidal ideation scores atthe final follow-up assessment in this trial (MD-4.90, 95% CI -7.07 to −2.73, 1 study, I2 = not ap-plicable, p < 0.001). As only one RCT reported dataat the final follow-up assessment, neither tests forsubgroup differences nor sensitivity analyses couldbe undertaken.

Fig. 2 Random effects odds ratio (OR) and accompanying 95% confidence interval (CI) for digital interventions on the proportion of participantsreaching defined clinical thresholds for suicidal ideation

Fig. 3 Random effects mean difference (MD) and accompanying 95% confidence interval (CI) for digital interventions on frequency of self-reported suicidal ideation


Fig. 4 Random effects standard mean difference (SMD) and accompanying 95% confidence interval (CI) for digital interventions of suicidalideation scores

Fig. 5 Random effects mean difference (MD) and accompanying 95% confidence interval (CI) for digital interventions on frequency of self-reportedself-cutting and non-suicidal self-injury (NSSI)


Self-harmOne study, covering three related RCTs, evaluated theeffectiveness of digital interventions on frequency ofself-harm episodes [50]. At post-intervention, there wasno indication of a treatment effect for these interventionson either self-reported frequency of self-cutting or non-suicidal self-injury in these three RCTs (Fig. 5). There wasalso no indication of a treatment effect for this interven-tion on frequency of self-reported self-cutting or non-suicidal self-injury at the final follow-up assessment (atone month) in one of these RCTs (Fig. 5). Sensitivity ana-lyses could not be undertaken for this outcome as onlyone study investigated outcomes relating to repetition ofself-harm. As all three studies were RCTs, subgroup ana-lyses to investigate the impact of study design on the mag-nitude of the effect size could also not be undertaken.

Combined self-harm and attempted suicideOne RCT evaluated the effectiveness of a digital inter-vention on attempted suicide and self-harm [46]. As re-sults for self-harm could not be disaggregated from thatfor attempted suicide, they are instead analysed here as acombined outcome. There was no evidence of a reduc-tion in the proportion of participants who attemptedsuicide and/or engaged in self-harm over a 24 monthfollow-up period in this study (OR 2.11, 95% CI 0.19 to23.81, 1 study, I2 = not applicable, p = 0.55). Once again,as only one study investigated outcomes relating to com-bined attempted suicide and/or self-harm, sub-groupand sensitivity analyses could not be undertaken.

Attempted suicideOne RCT evaluated the effectiveness of a digital interven-tion on attempted suicide [47]; however, no evidence of areduction in the proportion of participants self-reporting asuicide attempt was noted by the post-intervention assess-ment in this study (OR 0.58, 95% CI 0.16 to 2.02, 1 study,I2 = not applicable, p = 0.39). As only one study investi-gated outcomes relating to combined attempted sui-cide, sub-group and sensitivity analyses could not beundertaken.

DiscussionThis systematic review and meta-analysis evaluated theeffectiveness of digital interventions, including both onlineand mobile telephone applications (‘apps’) for the self-management and/or treatment of suicidal ideation or be-haviours. A total of 14 studies were included in thepresent review, reporting data for a total of 3356participants.Overall, this review found some evidence that digital

interventions may be associated with reductions in sui-cidal ideation, particularly at the post-intervention as-sessment. It is notable, however, that where these

interventions were associated with significant treatmentbenefits for suicidal ideation, these effects tended to bestronger in observational pre-test/post-test-designed stud-ies as compared with RCTs. Most of the included studiesconducted to date, however, have utilised a pre-test/post-test observational design. There was no evidence to sug-gest these interventions are associated with reductions inself-harm or attempted suicide, although only three stud-ies investigated these outcomes [46, 47, 50]. Few of thesestudies would have been adequately powered to evaluaterare outcomes, including repetition of self-harm and sui-cide reattempts.Adherence was poor in majority of these studies; al-

though this was not clearly related to the number oftreatment modules. Of those studies that reported infor-mation on adherence [38–50], for example, up to one-half of participants allocated to the intervention groupdid not complete all treatment modules. Adherence dur-ing the long-term follow-up period, which was reportedin one RCT [50], was also poor; over one-half (64%) ofparticipants allocated to the intervention group in thistrial did not access the intervention at all during the onemonth follow-up period in this RCT [50]. This suggeststhat the use of digital innovations, including gamifica-tion, may be insufficient to keep these interventions en-gaging over the longer term.

Limitations of the included studiesMost studies were of low to moderate quality, as assessedusing the Cochrane Collaboration’s tool for randomisedand pseudorandomised controlled trials [35], or theROBINS–I tool for controlled pre-test/post-test studies[36], with biases most apparent for the domains of partici-pant, clinical personnel, and outcome assessor blinding.Performance and detection bias therefore cannot be ruledout.Of those studies utilising an RCT design, most com-

pared the intervention to either waitlist control [37, 45]or attentional control [41, 47, 50] conditions. Variabilityin the control condition has been found to be associatedwith the magnitude of the treatment effect in office-based psychosocial interventions for depression and anx-iety [66, 67]. Yet, despite this, variability in the controlcondition is a rarely investigated source of heterogeneityin meta-analyses of digital interventions.The included studies report data on a variety of differ-

ent outcomes. Although all studies included at least onesuicidal ideation and/or behaviour relevant outcome,these were measured in a variety of different ways, in-cluding from psychometric scales, self-report, or accord-ing to hospital or medical records. Given that theseoutcomes were measured using the same methodologywithin individual studies, the effect of outcome measuredefinition on case identification at the post-intervention


and follow-up assessments could be expected to affectthe intervention and control groups equally. Betweenstudies, however, outcome measure definition may haveaffected our results in light of research findings suggest-ing that self-reported self-harm underestimates hospital-treated self-harm [68]. We were unable to assesswhether variability in the magnitude of the treatment ef-fect size was related to the way in which the outcomemeasure was assessed in this review, however, owing tothe small number of studies included in any one meta-analysis. Further work will be necessary to pick apart theinfluence of outcome definition on the apparent effect-iveness of interventions, including digital interventions,for the prevention of self-harm.Most studies (71.4%) also reported information on de-

pression [38–40, 42–45, 47–49]. Fewer studies reporteddata on other clinically relevant outcomes, such as hope-lessness [40], and none reported information onproblem-solving following the completion of the inter-vention. The National Institute for Health and ClinicalExcellence (NICE) guidelines recommend that all inter-ventions for self-harm should, at a minimum, investigatethe effect of psychological interventions for self-harm onpotential mechanisms of action, including depression,hopelessness, and problem-solving [69], echoing morerecent calls to this effect in the international scholarship[70, 71]. Online resources may normalize self-harm andmay provide vulnerable individuals with access to self-harm content and imagery, including information onmethods of self-harm [72]. Additionally, digital interven-tions that provide some form of active psychosocial ther-apy, and particularly those that include a focus on moodmonitoring, may lead to increased negative affectivityand rumination [73]. Outcomes relating to negativeaffect and rumination should therefore also be reportedfor all evaluations of digital interventions for this patientgroup in future.Most programs (57.1%) were developed for the self-

management of depression [37, 38, 42, 43, 45, 46, 48,49]. Recent findings for psychosocial interventions, how-ever, suggest that those developed specifically for themanagement of suicidal thinking are associated withgreater impacts for the reduction of attempted and com-pleted suicide as compared to those targeting indirectsymptoms associated with suicidal behaviour, such asanxiety, depression, or hopelessness [31]. Future studiesin this area should evaluate the degree to which thesedigital interventions lead to meaningful change in theproposed mechanism(s) of action in order to identify thetreatment module(s) associated with the greatest impactsin reducing suicidal ideation and self-harm in thispopulation.The included studies also examined interventions of

varying intensity. Whilst it could be expected that

treatments of greater intensity will have greater impactson reducing suicidal ideation and self-harm, a recentmeta-regression review found no evidence to suggestthat treatment intensity, measured as the total number ofavailable treatment sessions, was associated with greatereffectiveness for office-based psychosocial therapy for self-harm repetition [74].Finally, it is also likely that these populations will have

a very low risk for suicidal behaviour as all studies re-cruited participants from the community. Despite this, al-most all (78.6%) studies included indicated samples,such as callers to telephone or online counselling ser-vices [37, 45, 50], or those already in contact with pri-mary care [38, 42, 43], counselling [39, 40, 49] orpsychiatric services [46, 48].

Strengths and limitations of the present reviewThe majority of these interventions were based on theprinciples of standard cognitive behavioural therapy(CBT), which has been found to have efficacy in redu-cing repetition of self-harm in clinical populations in arecent systematic review of psychosocial interventionsfor the treatment of self-harm [10]. The findings of thepresent review significantly extend this by suggestingthat digital interventions which incorporate the princi-ples of standard CBT may have promising effects in re-ducing suicidal ideation in non-clinical populations, atleast in the short-term.Whilst we utilized a comprehensive search to locate all

relevant trials of digital interventions for the self-management of self-harm, we identified only two mobiletelephone apps [40, 50]. Given that a recent Australianstudy identified a total of 24 apps for the prevention ofsuicidal behaviour are currently available for downloadfrom the Australian Google Play and iOS store [75], thiswould suggest that a large number of these apps have noevidence to support their effectiveness. It is likely that asimilar proportion of online interventions would havelittle evidence to support their effectiveness.

ConclusionsAlthough a growing number of both online and mobiletelephone applications (‘apps’) for the self-managementand treatment of suicidal thinking and behaviours arenow available, few of these have been evaluated for theireffectiveness in reducing these outcomes. We identifiedjust 14 in this review. Overall, while there is some prom-ise of these interventions to reduce suicidal ideation,how this translates into reductions in self-harm and/orattempted suicide is unclear at present. Given the preva-lence of suicidal ideation in clinical populations, add-itionally, it is unclear whether these reductions would beclinically meaningful at present.


AppendixElectronic Search Strategy for the Medline and theCochrane LibraryProvides keywords used and electronic search strategyfor the Medline and the Cochrane Library. Current to 31March, 2017.Electronic search strategy

AbbreviationsApp: Application; BDI–II: Beck Depression Inventory, version two; BSSI: Beck Scaleof Suicidal Ideation; CBT: Cognitive behavioural therapy; CDRS–R: Children’sDepression Rating Scale–Revised; CENTRAL: Cochrane Central Register ofControlled Trials; CES–D: Center for Epidemiologic Studies Depression scale;CI: Confidence Interval; CRCT: Cluster randomised controlled trial; CRESP: Centrefor Research Excellence in Suicide Prevention; DBT: Dialectical behaviour therapy;DSI–SS: Depressive Symptom Inventory-Suicidality Subscale; DSM-5: Diagnosticand Statistical Manual of Mental Disorders, fifth revision; GHQ–28: General HealthQuestionnaire, 28 item; HSCL–20: Hopkins Symptom Checklist, 20 item;MADRS: Montgomery-Åsberg Depression Rating Scale; ME: Mean difference;OR: Odds ratio; PHQ–9: Patient Health Questionnaire, 9 item; PRISMA: PreferredReporting Items for Systematic Reviews and Meta-Analyses; RADS–2: ReynoldsAdolescent Depression Scale-Version 2; RCT: Randomised controlled trial; ROBINS–I: Risk of Bias in Non-Randomized Studies of Interventions; SBQ–R: SuicideBehaviors Questionnaire-Revised; SIQ–J: Suicidal Ideation Questionnaire-Junior;SMD: Standard mean difference; TAU: Treatment as usual; UK: United Kingdom;USA: United States of America; WHO: World Health Organization

AcknowledgementsThe authors wish to thank Constance Guille, Joe Franklin, Bregje van Spijker, andBen van Voorhees for providing raw data and/or methodological clarificationrelating to their studies.

Table 3 Cochrane Library

Keyword Number of ‘hits’

#1 ‘mhealth’ 4

#2 ‘ehealth’ 2

#3 web 790

#4 online 178

#5 internet 126

#6 ‘mobile device*’ 10

#7 ‘mobile phone’ 23

#8 ‘cell* phone’ 18

#9 smartphone 4

#10 ‘phone app*’ 39

#11 ‘mobile app*’ 36

#12 ‘cell* app*’ 1134

#13 ‘tablet computer*’ 6

#14 ipad* 0

#15 iphone* 0

#16 samsung 0

#17 android 0

#18 ‘windows phone*’ 0

#19 #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7OR #8 OR #9 OR #10 OR #11 OR #12 OR #13OR #14 OR #15 OR #16 OR #17 OR #18

2007

#20 ‘auto$mutilat*’ 0

#21 ‘self$harm’ 19

#22 ‘self$injur*’ 11

#23 ‘self$mutilat*’ 1

#24 ‘self$cutt*’ 0

#25 ‘self$poison*’ 5

#26 ‘drug overdos*’ 16

#27 overdos* 22

#28 ‘self$destructive behavio*’ 0

#29 ‘suicid* attempt*’ 14

#30 suicid* 69

#31 #20 OR #21 OR #22 OR #23 OR #24 OR #25OR #26 OR #27 OR #28 OR #29 OR #30

102

#32 #19 AND #31 32

Dates searched: Start Date to 30.06.2016

Table 4 MEDLINE (OVID interface)

Keyword Number of ‘hits’

#1 ‘mhealth’.mp 741

#2 ‘ehealth’.mp 1199

#3 web.mp 61,883

#4 online.mp 56,082

#5 internet.mp 76,766

#6 ‘mobile device*’.mp 1159

#7 ‘mobile phone’.mp 2999

#8 ‘cell* phone’.mp 1540

#9 smartphone.mp 2325

#10 ‘phone app*’.mp 229

#11 ‘mobile app*’.mp 2291

#12 ‘cell* app*’.mp 19,603

#13 ‘tablet computer*’.mp 340

#14 ipad*.mp 696

#15 iphone*.mp 400

#16 samsung,mp 527

#17 android.mp 1057

#18 ‘windows phone*’.mp 11

#19 #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7OR #8 OR #9 OR #10 OR #11 OR #12 OR #13OR #14 OR #15 OR #16 OR #17 OR #18

186,347

#20 ‘auto?mutilat*’.mp 123

#21 suicide, attempted/ or self-injurious behavior/or‘self?harm’.mp.

22,723

#22 suicidal ideation/ or suicide, attempted/ orsuicide/or suicid*.mp.

75,063

#23 overdos*’.mp 16,949

#24 #20 OR #21 OR #22 OR #23 92,822

#25 #19 AND #24 1265

Dates searched: Start Date to 31.03.2017


FundingThis work was supported by grants awarded to AM by beyondblue and theMovember Foundation. No other specific funding was received for theremaining authors. Funders had no role in either the design, interpretationof the findings, or writing of this manuscript.

Availability of data and materialsThis review involves analysis of previously published data available in thestudies included in this review. The final dataset is available from thecorresponding author upon reasonable request.

Authors’ contributionsKW had the idea for the paper, conducted the systematic review, extractedand analysed the data. AM and MS assisted with data extraction and analysis.All authors contributed to the writing of the manuscript and approved thefinal version of the manuscript for publication.

Ethics approval and consent to participateEthical approval and participant consent were not required for this review,since the study involved review and analysis of previously published data.

Consent for publicationNot applicable.

Competing interestsTwo of the authors of this review (SH, JR) were authors of one of theincluded studies. The remaining authors have no other competing interestto declare.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.

Author details1Population Health, Turning Point, Eastern Health Clinical School, MonashUniversity, 54-62 Gertrude Street, Fitzroy, Victoria 3065, Australia. 2MelbourneSchool of Population and Global Health, University of Melbourne, Melbourne,Victoria, Australia. 3Centre for Translational Neuroscience and Mental Health,Faculty of Health and Medicine, University of Newcastle, Callaghan, NewSouth Wales, Australia. 4Orygen, the National Centre of Excellence in YouthMental Health and the Centre for Youth Mental Health, University ofMelbourne, Melbourne, Victoria, Australia. 5Centre for Health Equity,Melbourne School of Population and Global Health, University of Melbourne,Melbourne, Victoria, Australia.

Received: 15 May 2017 Accepted: 8 August 2017

References1. Hawton K, Zhal D, Weatherall R. Suicide following deliberate self-harm: long

term follow-up of patients who presented to a general hospital. Br J Psychiatry.2003;182:537–42.

2. Logan C. Managing Clinical Risk: A Guide to Effective Practice. In: Logan C,Johnstone L, editors. Suicide and self-harm: Clinical risk assessment andmanagement using a structured professional judgement approach.Abingdon: United Kingdom: Routledge; 2013. p. 115–25.

3. McLean J, Maxwell M, Platt S, Harris F, Jepson R. Risk and protective factorsfor suicide and suicidal behaviour: a literature review. Edinbrugh, UnitedKingdom: Scottish Government; 2008.

4. Paul E, Tsypes A, Eidlitz L, Ernhout C, Whitlock J. Frequency and functions ofnon-suicidal self-injury: associations with suicidal thoughts and behaviors.Psychiatry Res. 2015;255:276–82.

5. Carroll R, Metcalfe C, Gunnell D. Hospital presenting self-harm and risk offatal and non-fatal repetition: systematic review and meta-analysis. PLoSOne. 2014;9(2):e89944.

6. Franklin J, Nock M. The Oxford Handbook of Behavioral Emergencies andCrises. In: Keleespies P, editor. Nonsuicidal self-injury and its relation tosuicidal behavior. New York, NY: Oxford University Press; 2017.

7. Kapur N, Cooper J, O'Connor R, Hawton K. Non-suicidal self-injury v.Attempted suicide: new diagnosis or false dichotomy? Br J Psychiatry. 2013;202(5):326–8.

8. Hawton K, Harriss L, Hall S, Simkin S, Bale E, Bond A. Deliberate self-harm inOxford, 1990-2000: A time of change in patient characteristics. Psychol Med.2003;33:987–95.

9. Hawton K, Witt K, Taylor Sailsbury T, Arensman E, Gunnell D, Townsend E,van Heeringen K, Hazell P. Interventions for self-harm in children andadolescents. Cochrane Database Syst Rev. 2015;9:CD012013.

10. Hawton K, Witt K, Taylor Salisbury T, Arensman E, Gunnell D, Hazell P, vanHeeringen K. Psychosocial interventions for self-harm in adults. CochraneDatabase Sys Rev. 2016;14:CD012189.

11. Hawton K, Witt KG, Salisbury TLT, Arensman E, Gunnell D, Hazell P,Townsend E, van Heeringen K. Psychosocial interventions following self-harm in adults: a systematic review and meta-analysis. Lancet Psychiatry.2016;3(8):740–50.

12. Tondo L, Albert M, Baldessarini R. Suicide in relation to health care access inthe United States: an ecological study. J Clin Psychiatry. 2006;67(4):517–23.

13. Kapusta N, Poschm N, Niederkrotenthaler T, Fischer-Kern M, Etzersdorfer M,Sonneck G. Availability of mental health service providers and suicide ratesin Austria: a nationwide survey. Psychiar Serv. 2010;61(12):1198–203.

14. Kawaguchi H, Koike S. Association between the density of physicians andsuicide rates in Japan: Nationwide ecological study using a spatial Bayesianmodel. PLoS One. 2016;11(2):e0148288.

15. Bruffaerts R, Demytenaere K, Hwang I, Chiu W, Sampson N, Kessler R, Alonso J,Borges G, de Girolamo G, de Graaf R, et al. Treatment of suicidal peoplearound the world. Br J Psychiatry. 2011;199(1):64–70.

16. Czyz E, Horwitz A, Eisenberg D, Kramer A, King C. Self-reported barriers toprofessional help seeking among college students at elevated risk forsuicide. J Am College Health. 2013;61(7):398–406.

17. Aggarwal N. Applying mobile technologies to mental health service deliveryin South Asia. Asian J Psychiatr. 2012;5(3):225–30.

18. de Beurs D, Kirtley O, Kerkhof A, Portzky G, O'Connor R. The role of mobilephone technology in understanding and preventing suicidal behavior.Crisis. 2015;36(2):79–82.

19. Kerkhof A, van Spijker B, Mokkenstorm J. Suicide Prevention and NewTechnologies: Evidence-Based Practice. In: Mishara B, Kerkhof A, editors.Reducing the burden of suicidal thoughts through online cognitivebehavioural therapy self-help. New York, NY: Palgrave Macmillian; 2013.

20. Hayes J, Maughan D, Grant-Peterkin H. Interconnected or disconnected?Promotion of mental health and prevention of mental disorder in thedigital age. Br J Psychiatry. 2016;208(3):205–7.

21. Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items forsystematic reviews and meta-analyses: the PRISMA statement. BMJ.2009;339(7):b2535.

22. Christensen H, Batterham P, O'Dea B. E-health interventions for suicideprevention. Int J Environ Res Public Health. 2014;11(8):8193–212.

23. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H.Smartphones for smarter delivery of mental health programs: a systematicreview. J Med Internet Res. 2013;15(11):e247.

24. Lai M, Maniam T, Chan L, Ravindran A. Caught in the web: a review ofweb-based suicide prevention. J Med Internet Res. 2014;16(1):e30.

25. Jacob N, Scourfield J, Evans R. Suicide prevention via the internet: adescriptive review. Crisis. 2014;35(4):261–7.

26. Perry Y, Werner-Seidler A, Calear A, Christensen H. Web-based and mobilesuicide prevention interventions for young people: a systematic review. JCan Acad Child Adolesc Psychiatry. 2016;25(2):73–9.

27. Deeks J, Higgins J, Altman D. Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S, (editors). Cochrane Handbook forSystematic Reviews of Interventions Version 5.1.0. The CochraneCollaboration; 2011. Available from http://handbook.cochrane.org.

28. Beck A, Kovacs M, Weissman A. Assessment of suicidal intention: the scalefor suicidal ideation. J Consult Clin Psychol. 1979;47(2):343–52.

29. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials.1986;7(3):177–88.

30. Higgins J, Thompson S, Deeks J, Altman D. Measuring inconsistency inmeta-analysis. BMJ. 2003;327(7414):557–60.

31. Meerwijk E, Parekh A, Oquendo M, Allen I, Franck L, Lee K. Direct versusindirect psychosocial and behavioural interventions to prevent suicide andsuicide attempts: a systematic review and meta-analysis. Lancet Psychiatry.2016;3:544–54.

32. Faber T, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Meta-analysesincluding non-randomized studies of therapeutic interventions: amethodological review. BMC Med Res Methol. 2016;16:35.


http://handbook.cochrane.org

33. Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literaturereview on the representativeness of randomized controlled trial samplesand implications for the external validity of trial results. Trials. 2015;16:495.

34. Parker L, Saez N, Porta M, Hernández-Aguado I, Lumbreras B. The impact ofincluding different study designs in meta-analyses of diagnostic accuracystudies. Eur J Epidemiol. 2013;28:713–20.

35. Higgins J, Altman D, Sterne J. Assessing risk of bias in included studies. In:Higgins JPT, Green S, (editors). Cochrane Handbook for Systematic Reviewsof Interventions Version 5.1.0. The Cochrane Collaboration; 2011. Availablefrom http://handbook.cochrane.org.

36. A tool for assessing Risk Of Bias In Non-randomized Studies of Interventions(ROBINS-I), version 7.0. http://www.riskofbias.info. Accessed 7 Mar 2016.

37. Christensen H, Farrer L, Batterham P, Mackinnon A, Griffiths K, Donker T. Theeffect of a web-based depression intervention on suicidal ideation:secondary outcome from a randomised controlled trial in a helpline. BMJOpen. 2013;3(6):e002886.

38. Mewton L, Andrews G. Cognitive behaviour therapy via the internet fordepression: a useful strategy to reduce suicidal ideation. J Affect Disord.2015;170(1):78–84.

39. Robinson J, Hetrick S, Cox G, Bendall S, Yuen H, Yung A, Pirkis J. Can aninternet-based intervention reduce suicidal ideation, depression andhopelessness among secondary school students: results from a pilot study.Early Interv Psychiatry. 2016;10(1):28–35.

40. Tighe J, Shand F, Ridani R, Mackinnon A, de la Mata N, Christensen H. iBobblymobile health intervention for suicide prevention in Australian indigenousyouth: a pilot randomised controlled trial. BMJ Open. 2017;7:e013518.

41. Guille C, Zhao Z, Krystal J, Nichols B, Brady K, Sen S. Web-based cognitivebehavioral therapy intervention for the prevention of suicidal behaviour inmedical interns: a randomized clinical trial. JAMA Psychiatry. 2015;72(12):1192–8.

42. van Voorhees B, Fogel J, Reinecke M, Gladstone T, Stuart S, Gollan J,Bradford N, Domanico R, Fagan B, Ross R, et al. Randomized clinical trial ofan internet-based depression prevention program for adolescents (projectCATCH-IT) in primary care: 12-week outcomes. J Dev Behav Pediatr.2009;30(1):23–37.

43. Whiteside U, Richards J, Steinfeld B, Simon G, Caka S, Tachibana C, StuckeyS, Ludman E. Online cognitive behavioral therapy for depressed primarycare patients: a pilot feasibility project. Perm J. 2014;18(2):21–7.

44. Hill RM, Pettit JW. Pilot Randomized Controlled Trial of LEAP: A SelectivePreventive Intervention to Reduce Adolescents’ Perceived Burdensomeness.J Clin Child Adolesc Psychol. 2016. ePub ahead of print. doi:10.1080/15374416.15372016.11188705.

45. Moritz S, Schilling L, Hauschildt M, Schröder J, Treszel A. A randomizedcontrolled trial of internet-based therapy in depression. Behav Res Ther.2012;50(7–8):513–21.

46. Kordy H, Wolf M, Aulich K, Bürgy M, Hegerl U, Hüsing J, Puschner B,Rummel-Kluge C, Vegger H, Backenstrass M. Internet-delivered diseasemanagement for recurrent depression: a multicentre randomized controlledtrial. Psychother Psychosom. 2016;85(2):91–8.

47. van Spijker B, van Straten A, Kerkhof A. Effectiveness of online self-help forsuicidal thoughts: results of a randomised controlled trial. PLoS One. 2014;9(2):e90118.

48. Hedman E, Ljótsson B, Kaldo V, Hesser H, Alaoui S, Kraepelien M, AnderssonE, Rück C, Svanborg C, Andersson G, et al. Effectiveness of internet-basedcognitive behaviour therapy for depression in routine psychiatric care. JAffective Disord. 2014;155(2):49–58.

49. Wagner B, Horn A, Maercker A. Internet-based versus face-to-face cognitivebehavioral intervention for depression: a randomized controlled non-inferoritytrial. J Affective Disord. 2014;152(7):113–21.

50. Franklin J, Fox K, Franklin C, Kleiman E, Ribeiro J, Jaroszewski A, Hooley J,Nock M. A brief mobile app reduces nonsuicidal and suicidal self-injury:evidence from three randomized controlled trials. J Consult Clin Psychol.2016;84(6):544–57.

51. Elbourne D, Altman D, Higgins J, Curtin F, Worthington H, Vail A. Meta-analyses involving cross-over trials: methodological issues. Int J Epidemiol.2002;31:140–9.

52. Boele F, Verdonck-de-Leeuw I, Cuijpers P, Reijneveld J, Heimans J, Klein M.Internet-based guided self-help for glioma patients with depressivesymptoms: design of a randomized controlled trial. BMC Neurol. 2014;14:81.

53. Eylem O, van Straten A, Bhui K, Kerkhof AJ. Reducing suicidal ideationamong Turkish migrants in the Netherlands and in the UK: effectiveness ofan online intervention. Int Rev Psychiatry. 2015;27(1):72–81.

54. Mühlmann C, Madsen T, Hjorthøj C, Kerkhof A, Nordentoft M, Erlangsen A.The self-help online against suicidal thoguhts (SOS) trial: study protocol fora randomized controlled trial. Trials. 2017;18:45.

55. Perry Y, Calear A, Mackinnon A, Batterham P, Licinio J, King C, Thomsen N,Scott J, Donker T, Merry S, et al. Trial for the prevention of depression(TriPoD) in final-year secondary students: study protocol for a clusterrandomised controlled trial. Trials. 2015;16(6):451.

56. Robinson J, Hetrick S, Cox G, Bendall S, Yung A, Yuen H, Templer K, Pirkis J.The development of a randomised controlled trial testing the effects of anonline intervention among school students at risk of suicide. BMC Psychiatry.2014;14(5):155.

57. Simon G, Beck A, Rossom R, Richards J, Kirlin B, King D, Shulman L, LudmanE, Penfold R, Shortreed S, et al. Population-based outreach versus care asusual to prevent suicide attempt: study protocol for a randomized controlledtrial. Trials. 2016;17:452.

58. Stallard P, Porter J, Grist R. Safety, acceptability, and use of a smartphoneapp, BlueIce, for young people who self-harm: protocol for an open phase Itrial. JMIR Res Protoc. 2016;5:e217.

59. van Spijker B, Calear A, Batterham P, Mackinnon A, Gosling J, Kerkhof A,Solomon D, Christensen H. Reducing suicidal thoughts in the Australiangeneral population through web-based self-help: study protocol for arandomized controlled trial. Trials. 2015;16(8):62.

60. Joiner T, Pfaff J, Acres J. A brief screening tool for suicidal symptoms inadolescents and young adults in general health settings: reliability andvalidity data from the Australian national general practice youth suicideprevention project. Behav Res Ther. 2002;40:471–81.

61. Osman A, Bagge C, Gutierrez P, Konick L, Kopper B, Barrios F. The suicidalbehaviors Questionniare-revised (SBQ-R): validation with clinical andnonclinical samples. Assessment. 2001;8(4):443–54.

62. Reynolds W. Suicidal Ideation Questionnaire Junior. Odessa, FL:Psychological Assessment Resources; 1987.

63. Goldberg D, Hillier V. A scaled version of the general health questionnaire.Psychol Med. 1979;9(1):139–45.

64. Derogatis L, Lipman R, Rickels K, Uhlenhuth E, Cori L. The Hopkins symptomchecklist (HSCL): a self-report symptom inventory. Behav Schi. 1974;19(1):1–15.

65. van Spijker B, van Straten A, Kerkhof A. Online self-help for suicidalthoughts: 3-month follow-up results and participant evaluation. InternetInterventions. 2015;2:283–8.

66. Watts S, Turnell A, Kladnitski N, Newby J, Andrews G. Treatment-as-usual(TAU) is anything but usual: a meta-analysis of CBT versus TAU for anxietyand depression. J Affect Disord. 2015;175:152–67.

67. Wampold B, Budge S, Laska K, Del Re A, Baardseth T, Flükiger C, Minami T,Kivlighanm D, Gunn W. Evidence-based treatments for depression andanxiety versus treatment-as-usual: a meta-analysis of direct comparisons.Clin Psychol Rev. 2011;31:1304–12.

68. Mars B, Cornish R, Heron J, Boyd A, Crane C, Hawton K, Lewis G, Tilling K,Macleod J, Gunnell D. Using data linkage to investigate inconsistentreporting of self-harm and questionnaire non-response. Arch Suicide Res.2016;20:113–41.

69. NICE. Self-harm: the short-term physical and psychological managementand secondary prevention of self-harm in primary and secondary care.Leicester, UK: British Psychological Society; 2004.

70. Witt K. The use of emergency department-based psychological interventionsto reduce repetition of self-harm behaviour. Lancet Psychiatry. 2017;4:428–9.

71. Milner A, Spittal M, Kapur N, Witt K, Pirkis J, Carter G. Mechanisms of briefcontact interventions in clinical populations: a systematic review. BMCPsychiatry. 2016;16:194.

72. Daine K, Hawton K, Singaravelu V, Stewart A, Simkin S, Montgomery P. Thepower of the web: a systematic review of studies of the influence of theinternet on self-harm and suicide in young people. PLoS One. 2013;8:e77555.

73. Swinkles A, Giuliano T. The measurement and conceptualization of moodawareness: attention directed towards one's mood states. Personal SocPsychol Bull. 1995;21:934–49.

74. Hetrick S, Robinson J, Spittal M, Carter G. Effective psychological andpsychosocial approaches to reduce repetition of self-harm: a systematicreview, meta-analysis, and meta-regression. BMJ Open. 2016;6:e011024.

75. Larsen M, Nicholas J, Christensen H. A systematic assessment of smartphonetools for suicide prevention. PLoS One. 2016;11:e0152285.


http://handbook.cochrane.org

http://www.riskofbias.info

http://dx.doi.org/10.1080/15374416.15372016.11188705

http://dx.doi.org/10.1080/15374416.15372016.11188705

Date post:	07-Jun-2020
Category:	Documents
Upload:	others
View:	11 times
Download:	0 times

RESEARCH ARTICLE Open Access Effectiveness of online and ... › content › pdf › 10.1186... ·...

Documents