Establishing Measurement-based Care in Integrated PrimaryCare: Monitoring Clinical Outcomes Over Time

Lisa K. Kearney1,2 • Laura O. Wray3,4 • Katherine M. Dollar5 • Paul R. King3

Published online: 8 December 2015! Springer Science+Business Media New York (outside the USA) 2015

Abstract Full implementation of the patient-centeredmedical home requires the identification and treatment of

patients with behavioral health concerns, leading to

improved patient outcomes and reduced health care costs.Measurement-based care (MBC) for mental health condi-

tions is an essential step in achieving these goals. Inte-

grated primary care (IPC) administrators and providers arekey leaders in MBC that spans initial screening for con-

ditions to monitoring clinical outcomes over time. The

purpose of this article is to assist IPC leaders, in partnershipwith their primary care team, in developing standard

operating procedures for screening and follow up evalua-

tions in order to lay a foundation for assessing program

outcomes and improving quality of care in their uniquesettings.

Keywords Screening ! Population health ! Patientcentered medical home ! Measurement based care !Outcomes measurement

The early identification and monitoring of patients withmental health conditions in primary care is an essential

component for supporting full implementation of the

Patient-Centered Medical Home (PCMH; Croghan &Brown, 2010). While primary care has been identified as

the de facto mental health system (Kessler & Stafford,

2008), the majority of mental health conditions are fre-quently undetected and/or under-treated within primary

care (Young, Klap, Sherbourne, & Wells, 2001). Even after

patients are identified as needing treatment, a significantproportion never returns for a second appointment or

connects with specialty mental health providers (Wray,

Syzmanski, Kearney, & McCarthy, 2012; Zanjani, Miller,Turiano, Ross, & Oslin, 2008). Further, primary care

patients with depression and anxiety have higher health

care costs and report more medical concerns (Katon, Lin,& Kroenke, 2007; Petterson et al., 2008). Likewise, indi-

viduals with comorbid chronic medical conditions and

mental health disorders demonstrate poorer outcomes (e.g.,Doering, Chen, McGuire, Bodan, & Irwin, 2014; Garfield

et al., 2014) related to unhealthy life styles and behavioral

choices which are leading factors in mortality/morbidity(Mokdad, Marks, Stroup, & Gerberding, 2004). Integrated

primary care (IPC) programs are designed to address

these gaps in primary care. Enhanced implementation ofPCMH can be accomplished when IPC providers use

& Lisa K. Kearneylisa.kearney3@va.gov

Laura O. Wraylaura.wray@va.gov

Katherine M. Dollarkatherine.dollar@va.gov

Paul R. Kingpaul.king2@va.gov

1 Department of Veterans Affairs, Veterans HealthAdministration, Office of Mental Health Operations,Washington, DC, USA

2 Department of Psychiatry, University of Texas HealthScience Center at San Antonio, San Antonio, TX, USA

3 Department of Veterans Affairs, VA Center for IntegratedHealthcare, Buffalo, NY, USA

4 Departments of Medicine and Psychology, University atBuffalo, Buffalo, NY, USA

5 Department of Veterans Affairs, VA Center for IntegratedHealthcare, West Haven, CT, USA

123

J Clin Psychol Med Settings (2015) 22:213–227

DOI 10.1007/s10880-015-9443-6

measurement-based care (MBC) to support early identifi-

cation of patients with behavioral health concerns andmonitoring of progress over time, thus laying a foundation

for improved patient outcomes and reduced care costs.

Measurement based care (MBC) can be defined as thepractice of basing clinical care on client data collected

throughout treatment (Scott & Lewis, 2015). Throughout

this paper, we will employ the term MBC to refer to thesystematic collection of data to monitor treatment progress,

assess outcomes, and guide treatment decisions, from ini-

tial screening to completion of care. The benefits of MBC,from initial screening to monitoring of outcomes longitu-

dinally, are widely noted. Screening alone assists with

improved outcomes by identifying mental health condi-tions for initiation of early interventions, promoting cost

savings through decreasing inappropriate referrals, and

identifying patients with mental health conditions who maybe over-utilizing services (e.g., Auxier, Farley, & Seifert,

2011; Derogatis & Lynn, 2000; Martin, Williams, Haskard,

& DiMatteo, 2005; Valenstein, Vijan, Zeber, Boehm, &Buttar, 2001). Likewise, monitoring clinical progress over

time has multiple beneficial effects, including the

improvement of patient level outcomes, increased patient-provider communication, enhanced patient involvement in

and understanding of care, improved treatment fidelity, and

facilitation of quality improvement efforts (e.g., Dowricket al., 2009; Eisen, Dickey, & Sederer, 2000; Lambert,

Harmon, Slade, Whipple, & Hawkins, 2005).This standardization and wide spread implementation of

MBC is a crucial next step for IPC. Providing a flexible,

evidence-based framework of care will allow IPC providerswithin the PCMH to monitor progress and outcomes,

general functioning, and quality of life for their patients,

while providing relevant information to multiple levels ofstakeholders, including the patient, other treating providers,

community partners, and facility leadership (Scott &

Lewis, 2015; Trivedi et al., 2007). IPC administrators areencouraged to create MBC systems that can provide

feedback: (1) at the patient-level to patients and providers

(Lambert et al., 2003; Trivedi et al., 2007); (2) at the panel-level to support coordinated, high quality care by inter-

professional teams (Chaney, Bonner, Vivell, Cohen,

Young, & Rubenstein, 2011; Liu et al., 2003); and (3) atthe population level to guide development of decision-

support tools, quality improvement efforts and for admin-

istrative review and decision-making (Greenhalgh, 2009).The purpose of this article is to outline the rationale and

practical considerations in developing MBC protocols for

screening and patient outcome monitoring to assist IPCadministrators and clinicians in assessing program out-

comes and improving quality of care. We begin with an

overview of screening approaches and review of appliedexamples of screening for common conditions (e.g.,

depression and anxiety). We then review the benefits of

routine outcome monitoring and discuss examples of suc-cessfully implemented outcome monitoring programs.

Finally, we offer practical step-by-step advice in develop-

ing and implementing screening and MBC processes.

Screening

Developing a program of MBC may begin with the

implementation of screening protocols for both chronicmedical and mental health conditions to assist with pre-

vention and early intervention in primary care (Linton,

2004). In order to promote early interventions for mentalhealth conditions, IPC providers must assist primary care

providers in identifying patients who would benefit most

from these interventions through the incorporation ofscreening processes for high base rate conditions (e.g.,

depression) and for low base rate but high-risk concerns

(e.g., suicidality). Both the United States preventativeservices task force (USPSTF) and Department of Veterans

Affairs and Department of Defense (VA/DoD) Clinical

Practice guidelines outline the importance of screening formany conditions within primary care, including depression,

alcohol misuse, and posttraumatic stress disorder (PTSD;

U.S. Preventive Services Task Force [USPSTF], 2015; VA/DoD Management of Major Depressive Disorder Working

Group, 2009; VA/DoD Management of Substance Use

Disorders Working Group, 2009; VA/DoD Management ofPost-Traumatic Stress Working Group, 2010). IPC provi-

ders need to work with all relevant stakeholders to assist in

identifying the conditions to be targeted for screening.

Selecting Conditions for Initial Screening

Screening measures are used in IPC settings to detectcommon, but often underdiagnosed, conditions (e.g.,

depression, substance misuse, anxiety). Numerous practical

and clinical considerations exist in determining whetherto screen for a given condition, as well as in identifying

the most efficient, effective, and meaningful screening

approach (Kessler, 2009). One essential consideration iswhether or not extant evidence supports screening for a

specific condition. Using the literature, it is also important

to determine whether a population-based or selectivescreening approach is warranted. Determinants of popula-

tion-based screening initiatives are rooted in the overall

prevalence of a condition or conditions in a target demo-graphic. In this context, screens are administered irre-

spective of identifiable symptoms. This includes the

administration of specific tests or questions to all membersof a population in order to identify risk for highly prevalent

214 J Clin Psychol Med Settings (2015) 22:213–227

123

conditions (e.g., depression, alcohol misuse) which can be

treated, or for which early prevention approaches areavailable (Kessler, 2009). In contrast, selective screening

seeks to identify conditions of concern within specific

high-risk subgroups (e.g., suicide risk in patients presentingwith depression; Arndt, Schultz, Turvey, & Petersen, 2002;

Burns, Gray, & Smith, 2010; Gerber, Iverson, Dichter,

Klap, & Latta, 2014; Kessler, 2009; Mattocks, 2015).Several additional questions must be considered in

evaluating whether and then how specific screening prac-tices should be implemented; these are outlined in Table 1

for consideration. It is the role of the IPC administrator to

facilitate discussion with PCMH team members in order todecide which screens will be implemented and to assist in

leading the development of standard operating procedures

for implementation, including appropriate recording andreporting of findings. After review of the literature, IPC

administrators frequently assume key leadership roles in

identifying the appropriate measurement instruments toutilize for their clinic’s specific patient population and in

teaching the staff to administer, score, and report screening

results in order to facilitate patient care. IPC administratorsmust also consider whether sufficient resources exist within

the clinic to support implementation of screening programs

and importantly, if adequate staffing is present to provideappropriate follow-up intervention (U.S. Preventive Ser-

vices Task Force [USPSTF], 2009c).

Population-Based Screening

Specific recommendations regarding whether routine popu-

lation-based screening (i.e., screening that occurs at standard

intervals or frequency, such as annually, and has been incor-porated into the standard process of care) should be conducted

vary based on the agency source, nature of the concern, and

special features of the patient population (see Table 2 for

sample conditions and recommendations made by theUSPSTF). For example, although strong evidence supports

routine screening for tobacco use among all adult primary care

patients (U.S. Preventive Services Task Force [USPSTF],2009a), routine screening for suicide risk has only been rec-

ommended for individuals with diagnosed mental health

disorders (LeFevre, on behalf of the USPSTF, 2014).Features of the clinic setting (e.g., geographic locale,

access to resources for further evaluation and treatment) andservice population (e.g., age, sex, health status or known

comorbidities) alter recommended methods for implemen-

tation of population-based screening as well. For example,although the USPSTF (2009c) recommends routine screen-

ing for depression in clinics with access to depression care

resources (that is, depression care managers or qualifiedmental health staff by referral to general mental health

clinics in the system), it advises against routine screening

among those without such resources. As clinics acrossDepartment of Veterans Affairs (VA; Kearney, Post,

Pomerantz, & Zeiss, 2014) and Department of Defense

(DoD; Hunter & Goodie, 2012) sites typically have access toembedded mental health providers, these settings are often

resourced to provide adequate primary care-based evalua-

tion, follow-up, and/or referral management in response topositive screens for depression (VA/DoD, The Management

of MDD Working Group, 2009), alcohol misuse (VA/DoD,

The Management of Substance Use Disorders WorkingGroup, 2009), and PTSD (VA/DoD, The Management of

Post-Traumatic Stress Working Group, 2010).

Population factors can help prioritize among variousoptions for screening. Unique risk factors in a specific

population may call for population based screening for

particular disorders, For example, because unique popula-tion risk factors have been documented in veterans, pop-

ulation-based screening initiatives for conditions such as

Table 1 Decisions on whether and how to screen for a particular condition

Whether to screen

What is the evidence for population-based screening or selective screening?

What is the feasibility of population-based or selective screening (e.g., staffing resources, clinic set up)?

What are the unique features of the screening context and target demographic? Do they support implementation of screening?

Are screening results actionable? Are treatment interventions available to offer individuals who screen positive?

How to screen

What method will be used for screening (verbal, paper and pencil, computer assisted)?

Who will administer and score screens?

What is the best screening measure to use based on its sensitivity and specificity for the identified population (i.e., ability to detect orcorrectly rule out the condition)?

How long does the instrument take to administer?

What is the cost and copyright of the selected instruments?

How frequently will the screening occur (e.g., once, annually)?

How are results recorded and communicated to the care team for further intervention?

J Clin Psychol Med Settings (2015) 22:213–227 215

123

military sexual trauma (Kimerling, Gima, Smith, Street, &

Frayne, 2007) and traumatic brain injury (VA, 2010) havebeen implemented in VA. Likewise, patient population

factors may also influence decisions regarding how popu-

lation screening is implemented. For example, low com-puter literacy in a population may call for paper/pencil

administration over kiosk or computer-based administra-

tion of screens (e.g., Bradford & Rickwood, 2015). EachIPC administrator will need to consider the population’s

unique demographics and risk factors and the clinic

resources when deciding which screens to implement in theclinic.

Table 2 Selected U.S. preventive services task force recommendations for mental and behavioral health screening in primary care

Condition Age range(years)

Qualifiers Routinescreeningrecommended?

Grade ofevidence

Source

Abuse,violence,neglect

14–46 years Recommendation pertains to screening for intimatepartner violence among women of reproductive agewho are asymptomatic

Yes B Moyer, on behalf ofUSPSTF (2013d)

C18 years Recommendation pertains to vulnerable and older adultswho are asymptomatic

No I Moyer, on behalf ofUSPSTF (2013d)

0–18 years Recommendation statement pertains to intervention forchildren who are asymptomatic, not presenting withsigns of abuse/neglect

No I Moyer, on behalf ofUSPSTF (2013a)

Alcoholmisuse

C18 years Recommendations pertain only to individuals who are notseeking alcohol-specific evaluation/treatment

Yes B Moyer, on behalf ofUSPSTF (2013c)12–17 years No I

Cognitiveimpairment

C65 years Recommendation pertains only to adults with no signs/symptoms of cognitive impairment

No I Moyer, on behalf ofUSPSTF (2014)

Depression C18 years Recommendation pertains to settings with adequatedepression care supports in place (e.g., resources foradditional evaluation, treatment, capable of follow-up)

Yes B USPSTF (2009c)

C18 years Recommendation pertains to settings without adequatedepression care supports in place

No C USPSTF (2009c)

12–18 years Recommendation pertains to settings with adequatedepression care supports in place

Yes B U.S. PreventiveServices TaskForce (USPSTF,2009b)

7–11 years N/A No I USPSTF (2009b)

Illicit druguse

All ages Recommendation pertains only to individuals (children,adolescents, adults, pregnant women) not previouslydiagnosed with a substance use disorder

No I USPSTF (2014)

Obesity C18 years N/A Yes B Moyer, on behalf ofUSPSTF (2012)

6–18 years N/A Yes B U.S. PreventiveServices TaskForce (USPSTF,2010)

Tobacco use C18 years Recommendation pertains to all adults Yes A USPSTF (2009a)

Pregnantwomen

Recommendation pertains to all pregnant womenregardless of age

Yes A USPSTF (2009a)

School-agedchildren andadolescents

Recommendation statement pertains to intervention forchildren and adolescents of school age, not screening

No B Moyer, on behalf ofUSPSTF (2013b)

Suicide risk Adolescents,adults, olderadults

Recommendation pertains only to individuals without adiagnosed psychiatric disorder

No I LeFevre, on behalf ofUSPSTF (2014)

Per U.S. Preventive Services Task Force (USPSTF) definitions: Grade A evidence implies that benefits of routine screening are substantial;routine screening is recommended; Grade B = evidence implies that benefits of routine screening are moderate; routine screening is recom-mended; Grade C evidence implies that benefits of routine screening are small; individual screening is recommended; Grade D evidence impliesno benefit from routine screening, or that harms of screening outweigh benefits; screening is discouraged; Grade I evidence implies insufficientevidence to support or refute routine screening. For readers interested in a more detailed listing of number of items, psychometric properties, andadministration time for each instrument, please reference Kearney, Wray, Dollar, King, & Vair, (2014)

216 J Clin Psychol Med Settings (2015) 22:213–227

123

Selective Screening

In implementation of MBC, it is critical to create a steppedcare approach to the assessment process. While all indi-

viduals in a clinic will be involved in population-based

screening, a smaller number will be selectively screened toidentify potential conditions based on the presence of

observable signs, symptoms, or other known risk factors

relative to a single patient. Thus, the purpose of selectivescreening is to further evaluate patients who are at high risk

for other conditions. Such warning signs may be self-re-

ported by patients, identified by caregivers, or noticed bymedical or mental health providers during routine care.

Selective screening may also be indicated after specific

events, as in screening for PTSD after an assault or motorvehicle accident (Beck & Coffey, 2007), or screening for

depression after childbirth (Gjerdingen & Yawn, 2007) or

other medical events such as stroke (Kneebone, Neffgen, &Pettyfer, 2012). Though population-based or selective

screening in primary care is not warranted if a given con-

dition has already been identified, or if the patient isalready involved in active treatment for that condition,

screening for known co-morbidities (e.g., alcohol or sub-stance misuse among patients with PTSD) is appropriate

for additional selective screenings (Kessler, 2009; Weaver,

Gilbert, El-Bassel, Resnick, & Noursi, 2015). For example,a patient already being treated for depression may still

warrant additional selective screening for other disorders

related to depression if reporting additional symptoms,such as anxiety or alcohol misuse.

As with population-based screening, decisions about

selective screening procedures must consider clinic setting,available resources, and patient demographics. Though

selective screening procedures place less demand on clinical

resources than population-based screening procedures, indi-vidual cliniciansmust still assess the extent towhich theyhave

access to viable screening instruments or procedures, are

capable of interpreting results and providing meaningfulclinical feedback, and can provide adequate follow-up eval-

uation or intervention (including referrals) in response to

positive screens. Adequate intervention may include referralsto specialty mental health services in cases warranting more

intensive intervention. Table 3 highlights a number of con-

ditions that may be selected for individual screening in pri-mary care settings, as well as several standard screening tools.

Stepped Care Screening and Assessment

Successful implementation of MBC often uses a steppedapproach to screening, wherein positive screens are fol-

lowed by more pointed evaluations of specific symptoms

and general functioning. Initial screening instruments

should be extremely brief (often B5 items), psychometri-cally sound, and repeatable to allow for tracking of indi-

vidual and population outcomes. Of note, positive initial

screening results do not indicate that a diagnosis is war-ranted, but rather that additional assessment is needed.

Whereas initial population-based screenings are fre-

quently administered by nursing staff prior to the patientbeing seen by the primary care provider (PCP), selective

screening is often conducted by the PCP or IPC providerafter positive screen results or high risk factors are iden-

tified. Positive results from population-based or selective

screening initiatives should typically be followed by a brieffunctional assessment (described below) and symptom

measurement by the PCP or IPC provider (consistent with

clinic standard operating procedures). Results may in turnlead to referral for a more comprehensive evaluation (See

Table 4) or active treatment as is clinically indicated.

Follow-up by PCPs may involve a warm hand-off to IPCproviders, which involves the PCPs bringing patients

directly to the IPC providers for introduction and a dis-

cussion of needs. As part of this warm hand-off, PCPs oftenprovide IPC providers with a brief overview of screening

results and other relevant clinical findings that emerged

during their assessment.Following handoff, IPC providers integrate positive

screening results to guide an initial brief, focused clinical

interview and functional assessment that emphasizes cur-rent symptoms and functioning in key domains. This

assessment is designed to focus on current concerns, and

intentionally avoids gathering extraneous information(Hunter, Goodie, Oordt, & Dobmeyer, 2009). Additional

selective screens or brief symptom measures may be added

based on the symptoms reported. Systematic use of brief,repeatable measures focused on presenting problems pro-

vides the opportunity to monitor individual patient change,

and to better inform clinical decision-making (e.g., Broten,Naugle, Kalata, & Graynor, 2011; Donohue & Draper

2011), facilitate, referrals (e.g., Greenhalgh, 2009), and

ultimately establish a baseline to track effectiveness ofinterventions (Eisen et al., 2000; Dowrick et al., 2009).

Stepped Care Assessment: Applied DepressionExample

In order to demonstrate how stepped care assessments can

be applied within an integrated care setting, we have chosen

to provide an example of this process applied to depression,one of the most common mental health concerns within the

primary care setting. Highlighting the importance of initial

screening in primary care, 13–20 % of primary care patientsscreen positive for depressive symptoms (Foster et al., 1999)

J Clin Psychol Med Settings (2015) 22:213–227 217

123

Table 3 Selected conditions and screening/measurement-based care tools

Condition Potential indications for individual screening Potential screening tools

Abuse/neglect Behavioral warning signs or physical/emotional symptoms of abuse orneglect by caregiver; presence of multiple known parent or child riskfactors (e.g., family substance abuse, family violence); red flaginjuries (e.g., repeated or unexplained bruises or broken bones,suspicious trauma)

Safe Environment for Every Kid (SEEK) ParentQuestionnaire (Dubowitz, Feigelman, Lane, &Kim, 2009)

Alcoholmisuse

Reports or evidence of drinking above recommended daily/weeklylimits; binge-drinking, even if infrequent or sporadic; alcohol-relatedfunctional impairments, such as missing work or school; alcohol-related risky behaviors, such as impaired driving, or risky sexualbehaviors; evidence or recent treatment for intoxication or withdrawal

Alcohol Use Disorder Identification Test (AUDIT;Saunders et al., 1993)

Alcohol Use Disorder Identification Test-Consumption questions (AUDIT-C; Bradley et al.,2007)

Cut-down, Annoyed, Guilty, Eye-opener (CAGE;Ewing, 1984)

Cognitiveimpairment

Suspected neurocognitive disorder: Reliable informant reports ofimpairment or decline; combination of multiple risk factors, such as:functional decline, difficulties with activities of daily living orinstrumental activities of daily living, memory complaints, executivedifficulties or other neuropsychiatric symptoms, age C65 years;recent neurological history (e.g., stroke, head injury); family history(i.e., first-degree relatives)

Blessed Orientation Memory Concentration test(BOMC; Katzman et al., 1983)

General Practitioner assessment of Cognition

(GPCOG; Brodaty et al., 2002)

Mini-Cog (Borson et al., 2000)

Montreal Cognitive Assessment (MoCA; Nasreddineet al., 2005)

Short Test of Mental Status (STMS; Kokmen et al.,1987)

St. Louis University Mental Status (SLUMS; Banks& Morley, 2003)

Suspected Attention-Deficit Hyperactivity Disorder (ADHD): Persistentdifficulty with concentration and focus; impulsivity; extremerestlessness or excessive movement; history of academic difficulty;childhood onset of symptoms; reliable informant report

Adult ADHD Self-Report Scale, 6-item (ASRS-6;Kessler et al., 2005)

Adult ADHD Self-Report Scale, 18-item (ASRS-18;Kessler et al., 2005)

Wender Utah Rating Scale (WURS; Ward, Wender,& Reimherr, 1993)

Depression Positive depression screen; current depressive symptoms; post-partumcare; recent significant loss

Patient Health Questionnaire- 9 Item (PHQ-9;Kroenke, Spitzer, & Williams, 2001)

Intimatepartnerviolence

Self-report of victimization; unexplained recent or remote injury(including bruises, broken bones, bleeding); veteran status; priorhistory of victimization

HARK (Humiliation-Afraid-Rape-Kick) scale Sohal,Eldridge, and Feder, (2007)

HITS (Hurt-Insult-Threaten-Scream) scale (Sherin,Sinacore, Li, Zitter, & Shakil, 1998)

Partner Violence Screen (PVS; Feldhaus et al., 1997)

Pain Recent or remote injury (including traumatic injuries, falls, motorvehicle accidents) or diagnosis (e.g., cancer, fibromyalgia) associatedwith acute or chronic pain; observable symptoms such as limping,wincing, or favoring an appendage; patient self-report or reliableinformant report

Brief Pain Inventory (BPI; Cleeland, 2009)

Short Form McGill Pain Questionnaire (SF-MPQ;Melzak, 1987)

Pain Outcome Questionnaire-Short Form (Clark,Gironda, & Young, 2003)

PTSD Recent or remote trauma exposure and/or significant occupationalhazard; veteran status; self-reported symptoms such as: intrusivememories, nightmares, insomnia, isolation, alcohol or substance use,depression, irritability

Primary Care-PTSD Screen (PC-PTSD; Prins et al.,2004)

PTSD Checklist for DSM5a (PCL5; Weathers et al.,2013)

Sleep Repeated difficulty initiating or maintaining sleep; early waking; non-restorative sleep; excessive snoring; daytime fatigue; new onset orworsening of stressors related to work, child-rearing, etc.;hypersomnolence; reliable sleep partner reports of disrupted sleep orexcessive snoring

Epworth Sleepiness Scale (ESS; Johns, 1991)

Insomnia Severity Index (ISI; Morin, 1993)

Pittsburgh Sleep Quality Index (PSQI; Buysse,Reynolds, Monk, Berman, & Kupfer, 1989)

Substance usedisorders

Reports or evidences use of illicit or street drugs (e.g., marijuana,cocaine, inhalants); misuse of prescription drugs (e.g.,benzodiazepines, opioids); drug-related functional impairments orlegal consequences stemming from drug use; drug-related risky

Drug Abuse Screening Tool (DAST-10; Skinner,1982)

Mini-International Neuropsychiatric Interview(MINI; Sheehan et al., 1998)

218 J Clin Psychol Med Settings (2015) 22:213–227

123

and systematic screening has been shown to improve iden-

tification of depression by 10 % to 47 % (Pignone et al.,

2002). As can be seen in Table 3, there are several stan-dardized, brief, and repeatable depression screening instru-

ments that are appropriate for the PC setting.

The information obtained from any initial screeningmeasures should guide the subsequent functional assess-

ment and administration of additional brief symptomsmeasurements. Key functional domains to consider when

further assessing depression, include but are not limited to,

suicidality, mania, anxiety, and interference with dailyactivities (e.g., missed work days, unfulfilled obligations).

Most standardized depression measures include at least one

item addressing suicidality (e.g., PHQ-9 item # 9, BDI-IIitem # 9). An independent screen, such as the 5-item

Paykel questionnaire, can also be used to probe for suicidal

ideation and attempts over the past year (Paykel, Myers,

Lindenthal, & Tanner, 1974). If these items are positive,

the clinician must focus attention to further evaluatingthese areas (Bryan, Corso, Neal-Walden, & Rudd, 2009;

LeFevre, 2014). For example, if the patient reports feeling

hopeless, the next step of assessment might include itemsspecifically addressing this domain (e.g., Beck Hopeless-

ness Scale). Based on the specific functional concernsidentified by a depressed patient, the clinician might also

consider additional brief symptom assessment of manic

symptoms associated with potential bipolar disorder, andscreening for indicators of anxiety symptoms as a potential

area of concern (see Table 4). Brief symptom assessment

instruments completed at this initial assessment can berepeated throughout the course of treatment to track pro-

gress over time.

Table 3 continued

Condition Potential indications for individual screening Potential screening tools

Suicidalideation

Diagnosable mental health or substance use disorder; trauma history;veteran status; recent inpatient admission; new or worsening medicaldiagnosis; recent substantial grief/loss

Columbia-Suicide Severity Rating Scale (C-SSRS;Posner et al., 2011)

Paykel questionnaire (Paykel et al., 1974)

Tobacco use Reports or evidences use of smoking cigarettes, pipes, cigars, orhookah; smokeless tobacco (e.g., ‘‘dip’’, ‘‘chew’’, or ‘‘snuff’’);e-cigarettes or ‘‘vaping’’; presence of medical conditions exacerbatedby smoking or tobacco use (e.g., lung cancer, COPD)

Fagerstrom Test for Nicotine Dependence (FTND;Heatherton, Kozlowski, Frecker, & Fagerstrom,1991)

Hooked on Nicotine Checklist (HONC; DiFranzaet al., 2002)

Weightmanagement

Population-based screening for excess weight by body mass index(BMI) is indicated for all adults and children, though BMI mayadditionally be useful in terms of identifying underweight status

BMI (Moyer, on behalf of USPSTF; USPSTF, 2010)

Conditions, clinical indications, and screening tools are samples, and are not to be considered exhaustivea There are no current publications of repeated use of the PCL5. However, the PCL has been validated in several prior studies. As updatedinformation becomes available on validation of repeated use of the PCL5, it will be provided on this website: http://www.ptsd.va.gov/professional/pages/assessments/assessment-pdf/PCL-handout.pdf

Table 4 Stepped care assessment related to depression after a positive PHQ-2

Follow-up brief symptom assessment for depressionmeasures

Symptoms of concern after functionalassessment

Further stepped brief symptom measures

9 item Patient Health Questionnaire (PHQ-9) Suicidality Affective States Questionnaire

Center for Epidemiological Studies-Depression scale(CES-D)

Beck Hopelessness Scale

Geriatric Depression Scale Beck Scale for SI

Beck Depression Inventory 2 Modified Scale for SI

Scale for Suicidal Ideation

Bipolar symptoms Mood Disorder Questionnaire

Mini-International NeuropsychiatricInterview

Anxiety Generalized Anxiety Disorder-7

Beck Anxiety Inventory

Beck Anxiety Inventory—Primary Care

Penn State Worry Questionnaire

J Clin Psychol Med Settings (2015) 22:213–227 219

123

Evidence in Support of Measurement-Based Care:Monitoring Outcomes Over Time

Systematic collection of population-based, selective

screening and individual assessment data improves the

recognition of mental health conditions in the PCMH andthe ability to make initial treatment plans. Developing a

system for measuring outcomes over time is an important

factor in improving the quality of mental health caredelivered in practice. It is critical to explore progress over

time at the patient level, panel level, and at the population

level for IPC administrators to assess program success.

Patient-Level Benefits of MBC

Both patients and providers benefit from MBC feedback.

Patients tend to receive structured instruments favorablyand seeing progress on measures over time can be an

affirming acknowledgement of their improving mental

health (Eisen et al., 2000; Dowrick et al., 2009). Con-versely, reviewing unimproved or worsening scores pro-

vides a natural transition into a discussion of therapy or

medication adherence, alternative treatments options, andsupports a truly patient-centered approach to treatment

decision-making (Greenhalgh, 2009). Patients who reflect

on their clinical outcomes on these measures with theirtherapists have actually been shown to improve at greater

rates than those whose therapist alone reviews the MBCoutcomes (Lambert et al., 2005).

Four large-scale studies by Lambert and colleagues that

were conducted in a specialty mental health clinic settingprovide important lessons for therapists wishing to imple-

ment MBC in primary care (Hawkins et al., 2004; Lambert

et al., 2001, 2002; Whipple et al., 2003). In these studies,the Outcome Questionnaire-45 (OQ-45; Lambert et al.,

1996) was used to assess four domains of psychological

functioning including symptoms (predominantly depres-sion and anxiety), interpersonal problems, social func-

tioning, and quality of life. The OQ-45 was repeated at

each weekly visit, and the therapists were provided with aweekly change score and its interpretation in terms of

prognosis for treatment outcomes based on the patient’s

rate of change. In a meta-analysis of patient outcomesacross studies including 2811 patients and 133 therapists,

therapy outcomes were better for patients whose therapists

received weekly feedback on the patient’s progress. Fur-ther, the best therapy outcomes were attained when both

the therapists and the patients received this feedback. The

results were convincing enough that the authors recom-mended that routine systems for therapy progress be

implemented in standard practice (Lambert et al., 2005).

The sequenced treatment alternatives to relieve depres-

sion (STAR-D) project demonstrated a large-scale use ofMBC to monitor care. The MBC system developed for the

study was employed so that the investigators could track

the effectiveness of antidepressant medications in realworld conditions across many providers and clinics. In this

pharmacotherapy trial, care coordinators provided primary

care prescribers with feedback on patient response andadjustment to medication dosing, ensuring that adequate

doses were provided. Decision rules, based on prescribingguidelines were developed in advance of the trial and care

coordinators used these rules to provide feedback to pro-

viders in both primary care and specialty mental healthsettings. In this way, investigators were able to ensure

guideline concordant care was provided and a single

antidepressant medication was trialed for a sufficientamount of time and at a sufficient dose before the medi-

cation was switched or an additional medication was added

(Trivedi et al., 2007).

Panel-Level Benefits of MBC

Collaborative care trials have repeatedly demonstrated

improved quality of primary care mental health treatmentusing measurement-based monitoring by a care manager

who provides information back to the primary care provider

and the psychiatric consultant (Butler et al., 2008). MBC isat the core of these interventions and patient data provides

the common language allowing teammembers from primary

care medicine, mental health, and nursing to communicateeffectively and work collaboratively. In these models of

collaborative care, mental health expertise of nurse care

managers and psychiatric supervision of the care managerswere factors associated with improved patient outcomes

(Bower, Gilbody, Richards, Fletcher, & Sutton, 2006). This

supervision is generally accomplished by regular meetingsin which the mental health teammembers review their panel

of patients. This discussion is facilitated by review of a

database where patients whose outcomes are not improvingare flagged. Flagged patients are discussed and treatment

plan adjustments are made and then discussed with the pri-

mary care provider. Tracking and team review of patientdata for improvement, or the lack there of, and side effects of

medications when they are used, allows the team to work

together to adjust treatments effectively.Panel-level data lends itself to team quality improve-

ment efforts using Plan-Do-Study-Act (PDSA) cycles. In

the VA, the Translating Initiatives in Depression intoEffective Solutions (TIDES) program is an excellent

example of the use of patient outcomes measures to both

directly assess patient progress and to improve quality

220 J Clin Psychol Med Settings (2015) 22:213–227

123

across the panels of patients (Chaney et al., 2011; Liu et al.,

2003). In the TIDES program, care managers were trainedto collect data and maintain registries of patients in their

care. Care managers entered data on processes and out-

comes (including PHQ-9 scores) of care for each patientinto a Microsoft Excel-based registry. As part of this evi-

dence-based quality improvement (EBQI) process, suc-

cessive PDSA cycles were used to support implementationof TIDES across sites.

Once the routine of MBC is in place, technology can beemployed to organize these data so that questions about

efficacy and quality of PCMH programs can be answered

more efficiently. Widely used in the VA, the BehavioralHealth Laboratory (BHL) model’s software is one such

example (Oslin et al., 2006). The software, which is

available outside the VA (VISN 4 MIRECC Resources andTools, n.d.), provides support to guide health technicians in

the administration of structured interview and it provides

decision-support by recommending specific referrals basedon the outcomes of the baseline screening assessment. It

supports patient tracking to make it easier for a clinician to

identify which patients are due for follow-up. In regards topanel review, the software is designed to easily output

clinical reports for individuals and panels of patients.

Population-Level Benefits of MBC

At the population level, patient reported outcomes can be used

to develop decision-support tools, to monitor population

status and assess program outcomes. Aggregation of dataallows administrators or teams of providers to monitor the

success of their approach to treatment within a population.

While some care is directed by nationally developed clinicalguidelines, many day-to-day clinical decisions are made

without such guidance. For example, while it may be clear

that a patient with newly diagnosed major depression shouldreceive treatment, the decision to send a patient to specialty

mental health treatment versus to provide him or her with

treatment in primary care is often not as clear. Aggregateprocess and outcomes data from a site can be reviewed to

determine whether there is differential effectiveness of an

intervention based on the severity of the initial assessmentscores. Let us suppose that we have implemented a 4-session

treatment of depression that is provided by IPC staff

throughout the primary care clinics in a system. We would beable to evaluate the treatment based on a large number of

patients who had been enrolled. If we were to determine that

patients with a PHQ-9 score of less than 15 on entry into careresponded fairly well, but patients with higher scores fre-

quently showed insufficient improvement, we could then

change our decision-support rule to require a discussion of thebenefits of referral to any patient whose PHQ-9 score was 15

and above while patients with a score between 10 and 15

would be routinely offered a choice of either type of treat-ment. Thus, the specific decision-rule would have been

developed based on the data from within our own system. In

addition to supporting referral to the best treatment option fora patient, a decision-support tool can speed the process of

getting the patient to the most effective care.

Creating a Foundation for Screeningand Measurement Based Care

Below we offer practical step-by-step guidance for devel-oping and implementing screening and MBC processes.

Step 1: Identify Target Conditions with StakeholderInput

In order for administrators of IPC programs to identify asuccessful structure for screening and MBC, they must begin

first with identifying the precise concerns and outcomes,

which are important to the multiple stakeholders in the clinic(Robinson & Reiter, 2007). Integrated care administrators

would benefit from considering the input from all relevant

stakeholders in this process, including but not limited to allproviders within the clinic (e.g., physicians, physician assis-

tants, nurse practitioners, RNs, LPNs, clinical pharmacists,

social workers, dietitians, etc.), administrative support staff,primary care leadership, mental health leadership, patients,

specialty and general mental health providers, and commu-

nity partners. Creation of an initial needs assessment mayassist administrators in identifying the specific conditions to

be targeted. This may be a simple survey of disorders desired

to be targeted by integrated care providers and specific out-comes to be addressed. Inmany clinical settings, theremay be

particular performance measures or monitors already identi-

fied by upper management for particular conditions (e.g.,Type II Diabetes and HgA1C\ 9, Hypertension with Blood

Pressure\140/90). Beginning with a needs assessment of

each of these stakeholders will be a critical piece for thedevelopment of buy-in across all levels of the system. By

building a program of outcomes measurement focused on the

stakeholders’ areas of concern, a strong foundation to estab-lish partnerships with each group will be laid during the

implementation phase (Fortney et al., 2009).

Step 2: Identify the Best Measures Validatedfor Your Setting

After deciding about the disorders forwhich the clinicwishes

to screen and the disorders to be targeted in treatment by the

Integrated Primary Care team, managers should review theliterature to identify the specific instruments validated for

J Clin Psychol Med Settings (2015) 22:213–227 221

123

screening and outcomes measurement in their setting.

Table 3 provides a selection of validated instruments thatcan be used for screening andMBC, which may be of benefit

for administrators to review. There are many additional

options for repeated measures instruments to be utilizedwithin integrated primary care settings (e.g., Hunter et al.,

2009; Kearney et al., 2014).

Step 3: Create Methods to Improve Easeof Administration of Instruments and DataExtraction

The first step to ease the administration burden for individualsinvolved is to provide training in proper administration and

use of the instruments (Robinson & Reiter, 2007). Fidelity to

administration guidelines is critical for accurate data to beobtained. Modeling of proper administration techniques and

having individuals practice the administration techniques

with observational feedback will also increase fidelity toadministration guidelines. Some clinics have transitioned

from verbal or paper administration of instruments to elec-

tronic methods of administration (e.g., tablet administration,kiosk administration, computer-based administration), which

has decreased the time burden on providers. When imple-

menting technology for measurement administration, it willbe critical for staff members to be fully trained in its use so

that they can answer patients’ questions.

WhenMBC is implemented in a systemwith an electronicmedical record (EMR), it will be helpful to create templates

for data entry if instruments are administered by paper and

pencil. Templates can be designed for entry of responses toeach instrument and for scoring of the instruments. Data can

then be available for extraction from the EMR for utilization

with individual patient feedback or for summary of panels ofpatient outcomes for provider and administrative review.

Additionally for MBC, it will be helpful to build into the

record electronic reminders that will alert providerswhen therepeat administration is due for a particular patient. Collab-

oration with IT departments and health record experts can be

invaluable to make products both patient and administratorfriendly for feedback review. If EMR systems are not

available, clinic administrators may wish to build secure

databases for data entry on standardized screening instru-ments and repeated measures. Sites may wish to build

spreadsheet data entry systems available to providers within

a clinic setting stored on a secured share drive.

Step 4: Establish and Implement StandardOperating Procedures (SOPs) for Screeningand Measurement-Based Care

Clinic administrators would benefit from creating SOPs toguide administration of screening and repeated measures

within their clinics. SOPs should also clearly outline clin-

ical decision points based on clinical practice guidelines forfurther treatment (e.g., Oslin et al., 2006; Trivedi et al.,

2007).In order to speed implementation, it is important to

obtain buy-in by stakeholders. Eliciting stakeholder input isessential and particularly important for the individuals

directly involved in administration of the measures.

Creating a streamlined process for administration will becritical to ensure that the addition of this process does not

disrupt the workflow in the system (Auxier et al., 2011).SOPs for the clinic should include the following:

(1) Training requirements for those administering the

measures,(2) Timelines for initial screening and repeated mea-

sures administration,

(3) Process for administration within the normal work-flow (e.g., when in the clinic appointment it will

occur, what parties will be involved, and wherepaper instruments will be stored or required equip-

ment will be located),

(4) Reference evidence-based guidelines for all criticaldecision-making points of care,

(5) Process for scoring of instrument and data entry (if

not automated),(6) Requirements for managing positive screening results,

(7) Utilization of feedback of results with the individual

patient both for initial scores well as for trackingoutcomes over time to guide evidence-based deci-

sion making related to treatment plans,

(8) Timelines and reporting requirements for paneloutcomes for stakeholders (e.g., regular reviews of

outcomes with staff members, clinic administrators,

and upper management)(9) Methods for monitoring provider/staff compliance

with guidelines of the SOP (e.g., chart reviews, daily

monitors of screening completion, etc.).

Step 5: Engage in Continuous Quality ImprovementProcesses to Evaluate Program Implementation

A large benefit of implementation of MBC is the ability toaggregate program evaluation data to assess outcomes for a

population of patients as well as for individual patients.

When creating new interventions within an integrated pri-mary care setting, it is particularly useful to evaluate whe-

ther programs are effective on a larger scale. For example, a

summary from the Behavioral Health Lab allows forextraction of data for a group of patients enrolled in care

management for depression within a specific period of time,

allowing the ability to evaluate the effectiveness of qualityimprovement efforts. In creating aggregate data summaries

for program evaluation, it will be helpful to allow for

222 J Clin Psychol Med Settings (2015) 22:213–227

123

extraction of data on key patient variables of interest to

stakeholders (e.g., sex, race/ethnicity). In addition to cre-ation of summaries for repeatedmeasures data, summarizing

percentage of positive screens for selected conditions can

assist administrators in determining unrealized needs forcare, which may have been previously unidentified.

Conclusion

In reviewing the above outlined practical steps for imple-mentation of MBC in integrated primary care settings, it is

critical to note that the implementation process involves acultural shift in the delivery of care. Simply mandating or

funding a large scale initiative, such as MBC implemen-

tation, will be insufficient to see the transformation realizedin PCMH (Leykum et al., 2007; Nutting, Miller, Crabtree,

Jaen, Steart, & Stange, 2009). Involving stakeholders at all

levels of the organization, including the patients served inthis model, will be critical to implementation success

(Fortney et al., 2009; Ritchie, Dollar, Kearney, & Kirchner,

2014). With implementation of MBC, integrated primarycare providers and administrators will not only be moving

towards improved outcomes for patients, but also will be

placed at the table to speak the language of primary care byassessing panel change related to quantitative outcomes.

Lambert’s (Lambert et al., 2003, 2005) call to implement

MBC in all mental health care is now over 10 years old. Inorder to progress in the implementation of integrated pri-

mary care, we must bring data-based outcomes to the

table as part of standard practice. While each clinic facesits own unique challenges and barriers with implementa-

tion, the benefits of implementation far outweigh the effort

required.

Compliance with Ethical Standards

Conflict of Interest Lisa Kearney, Laura Wray, Katherine Dollar,and Paul King declare that they have no conflict of interest.

Human and Animal Rights and Informed Consent No animal orhuman studies were carried out by the authors for this article.

References

Arndt, S. A., Schultz, S. K., Turvey, C., & Petersen, A. (2002).Screening for alcoholism in the primary care setting. Journal ofFamily Practice, 51, 41–46. Retrieved from http://www.jfponline.com/.

Auxier, A., Farley, T., & Seifert, K. (2011). Establishing an integratedcare practice in a community health center. ProfessionalPsychology: Research and Practice, 42, 391–397. doi:10.1037/a0024982.

Banks, W. A., & Morley, J. E. (2003). Memories are made of this:Recent advances in understanding cognitive impairment and

dementia. Journal of Gerontology: Series A: Biological Sciencesand Medical Sciences, 58A, 314–321. doi:10.1093/gerona/58.4.M314.

Beck, J. G., & Coffey, S. F. (2007). Assessment and treatment ofPTSD after a motor vehicle collision: Empirical findings andclinical observations. Professional Psychology: Research andPractice, 38, 629–639. doi:10.1037/0735-7028.38.6.629.

Borson, S., Scanlan, J., Brush, M., Vitaliano, P., & Dokmak, A.(2000). The Mini-Cog: A cognitive ‘‘vital signs’’ measure fordementia screening in multi-lingual elderly. International Jour-nal of Geriatric Psychiatry, 15, 1021–1027. doi:10.1002/1099-1166(200011)15:11\1021:AID-GPS234[3.0.CO;2-6.

Bower, P., Gilbody, S., Richards, D., Fletcher, J., & Sutton, A. (2006).Collaborative care for depression in primary care. Making senseof a complex intervention: Systematic review and meta-regres-sion. The British Journal of Psychiatry : The Journal of MentalScience, 189, 484–493. doi:10.1192/bjp.bp.106.023655.

Bradford, S., & Rickwood, D. (2015). Young people’s views onelectronic mental health assessment: Prefer to type than talk?Journal of Child and Family Studies, 24, 1213–1221. doi:10.1007/s10826-014-9929-0.

Bradley, K. A., DeBenedetti, A. F., Volk, R. J., Williams, E. C.,Frank, D., & Kivlahan, D. R. (2007). AUDIT-C as a brief screenfor alcohol misuse in primary care. Alcoholism, Clinical andExperimental Research, 31, 1208–1217. doi:10.1111/j.1530-0277.2007.00403.x.

Brodaty, H., Pond, D., Kemp, N. M., Luscombe, G., Harding, L.,Berman, K., & Huppert, F. A. (2002). The GPCOG: A newscreening test for dementia designed for general practice.Journal of the American Geriatrics Society, 50, 530–534.doi:10.1046/j.1532-5415.2002.50122.x.

Broten, L. A., Naugle, A. E., Kalata, A. H., & Gaynor, S. T. (2011).Depression and a stepped care model. In W. T. O’Donohue & C.Draper (Eds.), WT ODonohue (pp. 17–43)., Stepped care ande-health: Practical applications to behavioral disorders NewYork, NY: Springer. doi:10.1007/978-1-4419-6510-3_2.

Bryan, C. J., Corso, K. A., Neal-Walden, T. A., & Rudd, M. D.(2009). Managing suicide risk in primary care: Practice recom-mendation for behavioral health consultations. ProfessionalPsychology: Research and Practice, 40, 148–155. doi:10.1037/a0011141.

Burns, E., Gray, R., & Smith, L. A. (2010). Brief screeningquestionnaires to identify problem drinking during pregnancy:A systematic review. Addiction, 105, 601–614. doi:10.1111/j.1360-0443.2009.02842.x.

Butler, M., Kane, R. L., McAlpine, D., Kathol, R. G., Fu, S. S.,Hagedorn, H., & Wilt, T. J. (2008). Integration of mentalhealth/substance abuse and primary care. Evidence report/technology assessment. Rockville, MD: Agency for HealthcareResearch and Quality. Retrieved from http://www.ahrq.gov/.

Buysse, D. J., Reynolds, C. F., Monk, T. H., Berman, S. R., & Kupfer,D. J. (1989). The Pittsburgh sleep quality index (PSQI): A newinstrument for psychiatric research and practice. PsychiatryResearch, 28, 193–213. doi:10.1016/0165-1781(89)90047-4.

Chaney, E., Bonner, L., Vivell, S., Cohen, A. N., Young, A. S., &Rubenstein, L. (2011). How behavioral healthcare informaticssystems interface with medical informatics systems: A work inprogress. In N. A. Dewan, J. S. Luo, N. M. Lorenzi, N.A. Dewan, J. S. Luo, & N. M. Lorenzi (Eds.), Informationtechnology essentials for behavioral health clinicians (pp.195–204). New York, NY: Springer. doi:10.1007/978-1-84996-344-2_12.

Clark, M. E., Gironda, R. J., & Young, R. W. (2003). Developmentand validation of the pain outcomes questionnaire-VA. Journalof Rehabilitation Research and Development, 40, 381–396.doi:10.1682/JRRD.2003.09.0381.

J Clin Psychol Med Settings (2015) 22:213–227 223

123

Cleeland, C. S. (2009). The brief pain inventory user guide. Houston,TX: Author. Retrieved from http://www.mdanderson.org/education-and-research/departments-programs-and-labs/departments-and-divisions/symptom-research/symptom-assessment-tools/BPI_UserGuide.pdf.

Croghan, T. W., & Brown, J. D. (2010). Integrating mental healthtreatment into the Patient Centered Medical Home. AHRQPublication No. 10-0084-EF. Rockville, MD: Agency forHealthcare Research and Quality. Retrieved from https://pcmh.ahrq.gov/sites/default/files/attachments/Integrating%20Mental%20Health%20and%20Substance%20Use%20Treatment%20in%20the%20PCMH.pdf.

Department of Veterans Affairs. (2010). Screening and evaluation ofpossible traumatic brain injury in Operation Enduring Freedom(OEF) and Operation Iraqi Freedom (OIF) veterans (VHADirective 2010–2012). Washington, DC: Department ofVeterans Affairs. Retrieved from http://www.va.gov/oig/pubs/VAOIG-12-01702-303.pdf.

Department of Veterans Affairs and Department of Defense (VA/DoD), the Management of MDD Working Group. (2009). VA/DoD Clinical practice guideline for management of majordepressive disorder (MDD). Washington, DC: Department ofVeterans Affairs and Department of Defense. Retrieved fromhttp://www.healthquality.va.gov/guidelines/MH/mdd/MDDFULL053013.pdf.

Department of Veterans Affairs and Department of Defense (VA/DoD), the Management of Post-Traumatic Stress WorkingGroup. (2010). VA/DoD Clinical Practice Guideline for man-agement of post-traumatic stress. Washington, DC: Departmentof Veterans Affairs and Department of Defense. Retrieved fromhttp://www.healthquality.va.gov/guidelines/MH/ptsd/cpg_PTSD-FULL-201011612.pdf.

Department of Veterans Affairs and Department of Defense (VA/DoD), the Management of Substance Use Disorders WorkingGroup. (2009). VA/DoD Clinical practice guideline for man-agement of substance use disorders (SUD). Washington, DC:Department of Veterans Affairs and Department of Defense.Retrieved from http://www.healthquality.va.gov/guidelines/MH/sud/sud_full_601f.pdf.

Derogatis, L. R., & Lynn, L. L. (2000). Screening and monitoringpsychiatric disorder in primary care populations. In M. Maruish(Ed.), Handbook of psychological assessment in primary caresettings (pp. 115–152). Mahwah, NJ: Erlbaum.

DiFranza, J. R., Savageau, J. A., Fletcher, K., Ockene, J. K., Rigotti,N. A., McNeill, A. D., … Wood, C. (2002). Measuring the lossof autonomy over nicotine use in adolescents: The developmentand assessment of nicotine dependence in youths (DANDY)Study. Archives of Pediatric Adolescent Medicine, 156,397–403. doi:10.1001/archpedi.156.4.397.

Doering, L. V., Chen, B., McGuire, A., Bodan, R. C., & Irwin, M. R.(2014). Persistent depressive symptoms and pain after cardiacsurgery. Psychosomatic Medicine, 76, 437–444. doi:10.1097/PSY.0000000000000074.

Donohue, W. T. O., & Draper, C. (2011). The case for evidence-basedstepped care as part of a reformed delivery system. In W.T. O’Donohue & C. Draper (Eds.), Stepped care and e-health:practical applications to behavioral disorders (pp. 1–16). NewYork, NY: Springer. doi:10.1007/978-1-4419-6510-3_3.

Dowrick, C., Leydon, G. M., McBride, A., Howe, A., Burgess, H.,Clarke, P., … Kendrick, T. (2009). Patients’ and doctors’ viewson depression severity questionnaires incentivized in UK qualityand outcomes framework: Qualitative study. BMJ, 338. doi:10.1136/bmj.b663.

Dubowitz, H., Feigelman, S., Lane, W., & Kim, J. (2009). Pediatricprimary care to help prevent child maltreatment: The safe

environment for every kid (SEEK) model. Pediatrics, 123,858–864. doi:10.1542/peds.2008-1376.

Eisen, S. V., Dickey, B., & Sederer, L. I. (2000). A self-reportsymptom and problem rating scale to increase inpatients’involvement in treatment. Psychiatric Services, 51, 349–353.doi:10.1176/appi.ps.51.3.349.

Ewing, J. A. (1984). Detecting alcoholism. The CAGE questionnaire.Journal of the American Medical Association, 252, 1905–1907.doi:10.1001/jama.1984.03350140051025.

Feldhaus, K. M., Koziol-McLain, J., Amsbury, H. L., Norton, I. M.,Lowenstein, S. R., & Abbott, J. T. (1997). Accuracy of 3 briefscreening questions for detecting partner violence in theemergency department. Journal of the American MedicalAssociation, 277, 1357–1361. doi:10.1001/jama.277.17.1357.

Fortney, J. C., Pyne, J. M., Smith, J. L., Curran, G. M., Otero, J. M.,Enderle, M. A., & McDougall, S. (2009). Steps for implementingcollaborative care programs for depression. Population HealthManagement, 12, 69–79. doi:10.1089/pop.2008.0023.

Foster, M. A., Ragsdale, K., Dunne, B., Jones, E., Ihnen, G. H., Lentz,C., & Gilmore, J. (1999). Detection and treatment of depressionin a VA primary care clinic. Psychiatric Services, 50,1494–1495. doi:10.1176/ps.50.11.1494.

Garfield, L. D., Scherrer, J. F., Hauptman, P. J., Freedland, K. E.,Chrusciel, T., Balasubramanian, S., … Lustman, P. J. (2014).Association of anxiety disorders and depression with incidentheart failure. Psychosomatic Medicine, 76, 128–136. doi:10.1097/PSY.0000000000000027.

Gerber, M. R., Iverson, K. M., Dichter, M. E., Klap, R., & Latta, R. E.(2014). Women veterans and intimate partner violence: Currentstate of knowledge and future directions. Journal of Women’sHealth, 23, 302–309. doi:10.1089/jwh.2013.4513.

Gjerdingen, D. K., & Yawn, B. P. (2007). Postpartum depressionscreening: Importance, methods, barriers, and recommendationsfor practice. Journal of the American Board of Family Medicine,20, 280–288. doi:10.3122/jabfm.2007.03.060171.

Greenhalgh, J. (2009). The applications of PROs in clinical practice:What are they, do they work, and why? Quality of Life Research:An International Journal of Quality Of Life Aspects OfTreatment, Care & Rehabilitation, 18, 115–123. doi:10.1007/s11136-008-9430-6.

Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., &Tuttle, K. C. (2004). The therapeutic effects of providing patientprogress information to therapists and patients. PsychotherapyResearch, 14, 308–327. doi:10.1093/ptr/kph027.

Heatherton, T. F., Kozlowski, L. T., Frecker, R. C., & Fagerstrom, K.(1991). The Fagerstrom test for nicotine dependence: Arevision of the Fagerstrom tolerance questionnaire. BritishJournal of Addiction, 86, 1119–1127. doi:10.1111/j.1360-0443.1991.tb01879.x.

Hunter, C. L., & Goodie, J. L. (2012). Behavioral health in theDepartment of Defense patient-centered medical home: History,finance, policy, work force development, and evaluation.Translational Behavioral Medicine, 2, 355–363. doi:10.1007/s13142-012-0142-7.

Hunter, C. L., Goodie, J. L., Oordt, M. S., & Dobmeyer, A. C. (2009).Integrated behavioral health in primary care: Step-by-stepguidance for assessment and intervention. Washington, DC:American Psychological Association. doi:10.1037/11871-000.

Johns, M.W. (1991). A new method for measuring daytime sleepi-ness: The Epworth sleepiness scale. Sleep, 14, 540–545.Retrieved from http://journalsleep.org/Default.aspx.

Katon, W., Lin, E. B., & Kroenke, K. (2007). The association ofdepression and anxiety with medical symptom burden in patientswith chronic medical illness. General Hospital Psychiatry, 29,147–155. doi:10.1016/j.genhosppsych.2006.11.005.

224 J Clin Psychol Med Settings (2015) 22:213–227

123

Katzman, R., Brown, T., Fuld, P., Peck, A., Schecter, R., &Schimmel, H. (1983). Validation of a short orientation-mem-ory-concentration test of cognitive impairment. American Jour-nal of Psychiatry, 140, 734–739. doi:10.1176/ajp.140.6.734.

Kearney, L. K., Post, E. P., Pomerantz, A. S., & Zeiss, A. M. (2014a).Applying the interprofessional patient aligned care team in theDepartment of Veterans Affairs. American Psychologist, 69,399–408. doi:10.1037/a0035909.

Kearney, L. K., Wray, L. O., Dollar, K., King, P. R., & Vair, C. L.(2014b). Psychological assessment of Veterans in primary care-mental health integration contexts. In S. S. Bush (Ed.),Psychological assessment of veterans (pp. 88–126). New York:Oxford University Press.

Kessler, R. (2009). Identifying and screening for psychological andcomorbid medical and psychological disorders in medicalsettings. Journal of Clinical Psychology, 65, 253–267. doi:10.1002/jclp.20546.

Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi,E.,…Walters, E. E. (2005). The World Health Organization adultADHD self-report scale (ASRS): A short screening scale for usein the general population. Psychological Medicine, 35, 245–256.doi:10.1017/S0033291704002892.

Kessler, R., & Stafford, D. (2008). Primary care is the de facto mentalhealth system. In R. Kessler, D. Stafford, R. Kessler, & D.Stafford (Eds.), Collaborative medicine case studies: Evidencein practice (pp. 9–21). New York, NY: Springer. doi:10.1007/978-0-387-76894-6_2.

Kimerling, R. S., Gima, K., Smith, M. W., Street, A., & Frayne, S.(2007). The veterans health administration and military sexualtrauma. American Journal of Public Health, 97, 2160–2166.doi:10.2105/AJPH.2006.092999.

Kneebone, I. L., Neffgen, L. M., & Pettyfer, S. L. (2012). Screeningfor depression and anxiety after stroke: Developing protocols foruse in the community. Disability and Rehabilitation, 34,1114–1120. doi:10.3109/09638288.2011.636137.

Kokmen, E., Naessens, J. M., & Offord, K. P. (1987). A short test ofmental status: Description and preliminary results. Mayo ClinicProceedings, 62, 281–288. doi:10.1016/S0025-6196(12)61905-3.

Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9:Validity of a brief depression severity measure. Journal ofGeneral Internal Medicine, 16, 606–613. doi:10.1046/j.1525-1497.2001.016009606.x.

Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B.,Vermeersch, D. A., Clouse, G. C., & Yanchar, S. C. (1996). Thereliability and validity of the outcome questionnaire. ClinicalPsychology & Psychotherapy, 3, 249–258. doi:10.1002/(SICI)1099-0879(199612)3:4\249:AID-CPP106[3.0.CO;2-S.

Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E.J. (2005). Providing feedback to psychotherapists on theirpatients’ progress: Clinical results and practice suggestions.Journal of Clinical Psychology, 61, 165–174. doi:10.1002/jclp.20113.

Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A.,Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians toroutinely track patient outcome? A meta-analysis. ClinicalPsychology: Science and Practice, 10, 288–301. doi:10.1093/clipsy/bpg025.

Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A.,Nielsen, S. L., & Hawkins, E. J. (2001). The effects of providingtherapists with feedback on patient progress during psychother-apy: Are outcomes enhanced? Psychotherapy Research, 11,49–68. doi:10.1080/713663852.

Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W.,Hawkins, E. J., Nielsen, S. L., & Goates, M. (2002). Enhancingpsychotherapy outcomes via providing feedback on client

progress: A replication. Clinical Psychology & Psychotherapy,9, 91–103. doi:10.1002/cpp.324.

LeFevre, M. L., on behalf of the USPSTF. (2014). Screening forsuicide risk in adolescents, adults, and older adults in primarycare: U.S. Preventive Services Task Force recommendationstatement. Annals of Internal Medicine, 160, 719–726. doi:10.7326/M14-0589.

Leykum, L. K., Pugh, J., Lawrence, V., Parchman, M., Noel, P. H.,Cornell, J., & McDaniel, R. R, Jr. (2007). Organizationalinterventions employing principles of complexity science haveimproved outcomes for patients with Type II diabetes. Imple-mentation Science, 2, 28. doi:10.1186/1748-5908-2-28.

Linton, J. C. (2004). Psychological assessment in primary care. In L.J. Haas & L. J. Haas (Eds.), Handbook of primary carepsychology (pp. 35–45). New York, NY: Oxford UniversityPress. Retrieved from http://psycnet.apa.org/psycinfo/2005-03335-001.

Liu, C.-F., Hedrick, S. C., Chaney, E. F., Heagerty, P., Felker, B.,Hasenberg, N., … Katon, W. (2003). Cost-effectiveness ofcollaborative care for depression in a primary care veteranpopulation. Psychiatric Services, 54, 698–704. Retrieved fromhttp://www.ncbi.nlm.nih.gov/pubmed/12719501.

Martin, L. R., Williams, S. L., Haskard, K. B., & DiMatteo, M. R.(2005). The challenge of patient adherence. Therapeutics andClinical Risk Management, 1, 189–199. Retrieved from https://www.dovepress.com/therapeutics-and-clinical-risk-management-journal.

Mattocks, K. (2015). Mental health care screening and care coordi-nation during pregnancy and postpartum for women veterans.Journal of Women’s Health, 24, 1–2. doi:10.1089/jwh.2014.1517.

Melzak, R. (1987). The short-form McGill Pain questionnaire. Pain,30, 191–197. doi:10.1016/0304-3959(87)91074-8.

Mokdad, A. H., Marks, J. S., Stroup, D. F., & Gerberding, J. L.(2004). Actual causes of death in the United States, 2000.Journal of the American Medical Association, 291, 1238–1245.doi:10.1001/jama.291.10.1238.

Morin, C. M. (1993). Insomnia: Psychological assessment andmanagement. New York: Guilford Press. doi:10.1002/smi.2460100113.

Moyer, V. A., on behalf of the USPSTF. (2012). Screening for andmanagement of obesity in adults: U.S. Preventive Services TaskForce recommendation statement. Annals of Internal Medicine,157, 373–378. doi:10.7326/0003-4819-157-5-201209040-00475.

Moyer, V. A., on behalf of the USPSTF. (2013a). Primary careinterventions to prevent child maltreatment: U.S. PreventiveServices Task Force recommendation statement. Annals ofInternal Medicine, 159, 289–295. doi:10.7326/0003-4819-159-4-201308200-00667.

Moyer, V. A., on behalf of the USPSTF. (2013b). Primary careinterventions to prevent tobacco use in children and adolescents:U.S. Preventive Services Task Force recommendation statement.Annals of Internal Medicine, 159, 552–557. doi:10.7326/0003-4819-159-8-201310150-00697.

Moyer, V. A., on behalf of the USPSTF. (2013c). Screening andbehavioral counseling interventions in primary care to reducealcohol misuse: U.S. Preventive Services Task Force recom-mendation statement. Annals of Internal Medicine, 159,210–218. doi:10.7326/0003-4819-159-3-201308060-00652.

Moyer, V. A., on behalf of the USPSTF. (2013d). Screening forintimate partner violence and abuse of elderly and vulnerableadults: U.S. Preventive Services Task Force recommendationstatement. Annals of Internal Medicine, 158, 478–486. doi:10.7326/0003-4819-158-6-201303190-00588.

Moyer, V. A., on behalf of the USPSTF. (2014). Screening forcognitive impairment in older adults: U.S. Preventive Services

J Clin Psychol Med Settings (2015) 22:213–227 225

123

Task Force recommendation statement. Annals of InternalMedicine, 160, 791–797. doi:10.7326/M14-0496.

Nasreddine, Z. S., Phillips, N. A., Bedirian, V., Charbonneau, S.,Whitehead, V., Collin, I., … Chertkow, H. (2005). The montrealcognitive assessment (MoCA): A brief screening tool for mildcognitive impairment. Journal of the American GeriatricsSociety, 53, 695–699. doi:10.1111/j.1532-5415.2005.53221.x.

Nutting, P. A., Miller, W. L., Crabtree, B. F., Jaen, C. R., Steart, E. E.,& Stange, K. C. (2009). Initial lessons from the first nationaldemonstration project on practice transformation to a patient-centered medical home. Annals of Family Medicine, 7, 254–260.doi:10.1370/afm.1002.

Oslin, D. W., Ross, J., Sayers, S., Murphy, J., Kane, V., & Katz, I.(2006). Screening, assessment, and management of depression inVA primary care clinics: The behavioral health laboratory.Journal of General Internal Medicine, 21, 46–50. doi:10.1111/j.1525-1497.2005.0267.x.

Paykel, E. S., Myers, J. K., Lindenthal, J. J., & Tanner, J. (1974).Suicidal feelings in the general population: A prevalence study.British Journal of Psychiatry, 124, 460–469. doi:10.1192/bjp.124.5.460.

Petterson, S. M., Phillips, R. L., Bazemore, A. W., Dodoo, M. S.,Zhang, X., & Green, L. A. (2008). Why there must be room formental health in the medical home. American Family Physicians,77, 757. Retrieved from http://www.aafp.org/journals/afp.html.

Pignone, M. P., Gaynes, B. N., Rushton, J. L., Burchell, C. M.,Orleans, C. T., Mulrow, C. D., & Lorh, K. N. (2002). Screeningfor depression in adults: A summary of the evidence for the U.S.preventive services task force. Annals of Internal Medicine, 136,765–776. doi:10.7326/0003-4819-136-10-200205210-00013.

Posner, K., Brown, G. K., Stanley, B., Brent, D. A., Yershova, K. V.,Oquendo, M. A., … & Mann, J. J. (2011). The Columbia-suicideseverity rating scale: Initial validity and internal consistencyfindings from three multisite studies with adolescents and adults.American Journal of Psychiatry, 168, 1266–1277. doi:10.1176/appi.ajp.2011.10111704.

Prins, A., Ouimette, P., Kimerling, R., Cameron, R. P., Hugelshofer,D. S., Shaw-Hewger, J., …Thrailkill, A. (2004). The primarycare PTSD screen (PC-PTSD): Development and operatingcharacteristics. Primary Care Psychiatry, 9, 9–14. doi: 10.1185/135525703125002360.

Ritchie, M. J., Dollar, K. M., Kearney, L. K., & Kirchner, J. E.(2014). Research and services partnerships: Responding to needsof clinical operations partners: Transferring implementationfacilitation knowledge and skills. Psychiatric Services, 65,141–143. doi:10.1176/appi.ps.201300468.

Robinson, P. J., & Reiter, J. T. (2007). Behavioral consultation andprimary care: A guide to integrating services. New York:Springer. doi:10.1007/978-0-387-32973-4.

Saunders, J. B., Aasland, O. G., Babor, T. F., de la Fuente, J. R., &Grant, M. (1993). Development of the alcohol use disordersidentification test (AUDIT): WHO collaborative project on earlydetection of persons with harmful alcohol consumption: II.Addiction, 88, 791–804. doi:10.1111/j.1360-0443.1993.tb02093.x.

Scott, K., & Lewis, C. C. (2015). Using measurement-based care toenhance any treatment. Cognitive and Behavioral Practice, 22,49–59. doi:10.1016/j.cbpra.2014.01.010.

Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janvas, J.,Weiller, E., … Dunbar, G. C. (1998). The mini-internationalneuropsychiatric interview (M.I.N.I.): The development andvalidation of a structured diagnostic psychiatric interview forDSM-IV and ICD-10. Journal of Clinical Psychiatry, 59 (Suppl20), 22–33. Retrieved from http://www.psychiatrist.com/jcp/pages/home.aspx.

Sherin, K. M., Sinacore, J. M., Li, X. Q., Zitter, R. E., & Shakil, A.(1998). HITS: A short domestic violence screening tool for use

in a family practice setting. Family Medicine, 30, 508–512.Retrieved form http://www.stfm.org/NewsJournals/FamilyMedicine.

Skinner, H. (1982). The drug abuse screening test. AddictiveBehaviors, 7, 363–371. doi:10.1016/0306-4603(82)90005-3.

Sohal, H., Eldridge, S., & Feder, G. (2007). The sensitivity andspecificity of four questions (HARK) to identify intimate partnerviolence: A diagnostic accuracy study in general practice. BMCFamily Practice, 8, 1–9. doi:10.1186/1471-2296-8-49.

Trivedi, M. H., Rush, A. J., Gaynes, B. N., Stewart, J. W.,Wisniewski, S. R., Warden, D., … Howland, R. (2007).Maximizing the adequacy of medication treatment in controlledtrials and clinical practice: STARD measurement-based care.Neuropsychopharmacology, 32, 2479–2489. doi:10.1038/sj.npp.1301390.

U.S. Preventive Services Task Force (USPSTF). (2015). Recommen-dations for primary care practice. Retrieved June 29, 2015 fromhttp://www.uspreventiveservicestaskforce.org/BrowseRec/Index.

USPSTF. (2009a). Counseling and interventions to prevent tobaccouse and tobacco-caused disease in adults and pregnant women:U.S. preventive services task force reaffirmation recommenda-tion statement. Annals of Internal Medicine, 150, 551–555.doi:10.7326/0003-4819-150-8-200904210-00009.

USPSTF. (2009b). Screening and treatment for major depressivedisorder in children and adolescents: US preventive services taskforce recommendation statement. Pediatrics, 123, 1223–1228.doi:10.1542/peds.2008-2381.

USPSTF. (2009c). Screening for depression in adults: U.S. preventiveservices task force recommendation statement. Annals of Inter-nal Medicine, 151, 784–792. doi:10.7326/0003-4819-151-11-200912010-00006.

USPSTF. (2010). Screening for obesity in children and adolescents:US preventive services task force recommendation statement.Pediatrics, 125, 361–367. doi:10.1542/peds.2009-2037.

USPSTF. (2014). Final recommendation statement: Drug use, illicit:Screening. Retrieved from http://www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/drug-use-illicit-screening.

Valenstein, M., Vijan, S., Zeber, J. E., Boehm, K., & Buttar, A.(2001). The cost-utility of screening for depression in primarycare. Annals of Internal Medicine, 134, 345–360. doi:10.7326/0003-4819-134-5-200103060-00007.

VISN 4 MIRECC (n.d.) Resources and tools: BHL software.Retrieved October 27, 2015, from http://www.mirecc.va.gov/visn4/BHL/BHLresources3.asp.

Ward, M. F., Wender, P. H., & Reimherr, F. W. (1993). The WenderUtah Rating Scale: An aid in the retrospective diagnosis ofchildhood attention deficit hyperactivity disorder. AmericanJournal of Psychiatry, 150, 885–890. doi:10.1176/ajp.150.6.885.

Weathers, F. W., Litz, B. T., Keane, T. M., Palmieri, P. A., Marx, B.P., & Schnurr, P. P. (2013). The PTSD checklist for DSM-5(PCL-5). Retrieved from www.ptsd.va.gov.

Weaver, T. S., Gilbert, L., El-Bassel, N., Resnick, H. S., & Noursi, S.(2015). Identifying and intervening with substance-using womenexposed to intimate partner violence: Phenomenology, comor-bidities, and integrated approaches within primary care and otheragency settings. Journal of Women’s Health, 24, 51–56. doi:10.1089/jwh.2014.4866.

Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W.,Nielsen, S. L., & Hawkins, E. J. (2003). Improving the effects ofpsychotherapy: The use of early identification of treatment andproblem-solving strategies in routine practice. Journal ofCounseling Psychology, 50, 59. doi:10.1037/0022-0167.50.1.59.

Wray, L. O., Szymanski, B. R., Kearney, L. K., & McCarthy, J. F.(2012). Implementation of primary care-mental health integra-tion services in the veterans health administration: Program

226 J Clin Psychol Med Settings (2015) 22:213–227

123

activity and associations with engagement in specialty mentalhealth services. Journal of Clinical Psychology in MedicalSettings, 19, 105–116. doi:10.1007/s10880-011-9285-9.

Young, A. S., Klap, R., Sherbourne, C. D., & Wells, K. B. (2001).The quality of care for depressive and anxiety disorders in theUnited States. Archives of General Psychiatry, 58, 55–61.doi:10.1001/archpsyc.58.1.55.

Zanjani, F., Miller, B., Turiano, N., Ross, J., & Oslin, D. (2008).Effectiveness of telephone- based referral care management: Abrief intervention to improve psychiatric treatment engagement.Psychiatric Services, 59, 776–781. doi:10.1176/appi.ps.59.7.776.

J Clin Psychol Med Settings (2015) 22:213–227 227

123