Cricket Mitchell, PhD

CIMH Evaluation Consultant

CiMH Palette of Measures CiMH Palette of Measures Evaluation Training: Center for Evaluation Training: Center for

Epidemiological Studies – Epidemiological Studies – Depression Scale (CES-D)Depression Scale (CES-D)

Palette of Measures Evaluation: Palette of Measures Evaluation: What You Will Need What You Will Need (slide 1 of 2)(slide 1 of 2)

• Palette of Measures Data Entry Shell v2 (Excel file)

– Developed by CiMH and customized for each participating agency

– Holds all data for clients served thru the Palette of Measures project

– Demographics, service delivery information, pre- and post- outcome measure data

• Palette of Measures Data Dictionary v2 (Word document)

– A guide for using the associated data entry shell– Defines each column in the excel file

2

Palette of Measures Evaluation: Palette of Measures Evaluation: What You Will Need What You Will Need (slide 2 of 2)(slide 2 of 2)

• Outcome measures from the two-pronged approach– General Outcome Measure– Target-Specific Outcome Measure(s)– For example…

• Center for Epidemiological Studies – Depression Scale (CES-D)– Public domain, no fee for use

3

Overview of Training Overview of Training • Brief Overview of Palette of Measures evaluation

protocol• Center for Epidemiological Studies – Depression

Scale (CES-D): Target-specific measure when the focus of treatment is depression– Administration– Scoring– Clinical Utility

• Instructions for Palette of Measures data entry and data submissions– Data entry: CES-D– Data entry: Demographics & Services– Data submissions to CiMH

4

5

Brief Overview of Palette of Brief Overview of Palette of Measures Evaluation ProtocolMeasures Evaluation Protocol

Outcome AssessmentOutcome Assessment• Palette of Measures providers will track outcomes

using data from pre- and post- administrations of standardized measures of functioning

• Pre- and Post- a “dose” of treatment / an intervention interval– General measure of youth mental health functioning

(e.g., YOQ/YOQ-SR, CANS, Ohio Scales)– Target-specific measure linked to focus of

treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS)

• Providers may choose to administer mid-course assessments as well– e.g., at 3-month intervals in usual care

6

A note about the use of standardized A note about the use of standardized assessment measures… assessment measures… (slide 1 of 2)(slide 1 of 2)

• Assessment is the beginning of developing a relationship with the child and family– Demonstrates a desire to know what the child

and family are experiencing– By incorporating standardized assessment

measures of functioning, the efficiency and thoroughness of assessment is enhanced

7

A note about the use of standardized A note about the use of standardized assessment measures… assessment measures… (slide 2 of 2)(slide 2 of 2)

• Using standardized assessment measures of functioning…– Assists in initial clinical impressions– Provides valuable information to guide

treatment/interventions– Assesses sufficiency of treatment delivered– Demonstrates treatment-related improvements

in child functioning

8

9

Center for Epidemiological Studies Center for Epidemiological Studies - Depression Scale (CES-D)- Depression Scale (CES-D)

CiMH CES-D TrainingCiMH CES-D Training• Information on the administration, scoring,

and clinical utility of the CES-D was obtained from on-line resources published by the Oregon Health & Science University and by the University of North Carolina at Chapel Hill, LONGSCAN studies

10

CES-D DescriptionCES-D Description

• Target specific measure for depression

• Measures depressive symptoms experienced in the previous week– Self-report for ages 12 and older

• 5 minutes to complete

• Available in English and Spanish

11

CES-D DescriptionCES-D Description

• Valid and reliable

• Sensitive to clinical change

• Available in the public domain, provided to partnering agencies by CiMH

12

CES-D DescriptionCES-D Description• 20 items• 4-point Likert scale response

options*– Rarely or none of the time (0)– Some or a little of the time (1)– Occasionally or a moderate

amount of time (2)– Most or all of the time (3)*All positive items are reverse-scored

• Total Score

13

CES-D DescriptionCES-D Description

• The 20 items on the CES-D reflect six major dimensions of depression– Depressed mood – Feelings of guilt and worthlessness – Feelings of helplessness and hopelessness – Psychomotor retardation – Loss of appetite– Sleep disturbance

14

Example: Items from the CES-DExample: Items from the CES-D

• I was bothered by things that usually don’t bother me.

• I felt depressed.

• I felt hopeful about the future.

• My sleep was restless.

• I talked less than usual.

• I was lonely.

15

CES-D AdministrationCES-D Administration

• Administer pre- and post- a dose of treatment, or an intervention interval, focused on depressive symptomotology– CES-D completed by client– Some agencies may choose to administer

mid-treatment assessments as well

16

Let’s take a look at the Let’s take a look at the CES-D... CES-D...

17

CES-D AdministrationCES-D Administration• Less than 5 minutes to complete• When in doubt, administer the measure aloud

– Readability is age 12

• Ensure that the respondent is clear about the timeframe– “... how you have been feeling during the past

week.”

• Ensure all items are completed– Leave no blanks

18

CES-D ScoringCES-D Scoring

• Four of the 20 items are phrased positively to control for response bias4. I felt that I was just as good as other

people.

8. I felt hopeful about the future.

12. I was happy.

16. I enjoyed life.

19

CES-D ScoringCES-D Scoring

• Response values for each item response are on slide 13

• Re-code/reverse-score the positively phrased items: 4, 8, 12, and 16– For these four items only:

• 3=0, 2=1, 1=2, and 0=3

• Once the four positively phrased items have been reverse-scored, sum up all of the item responses (items 1-20)

20

CES-D ScoringCES-D Scoring

• Possible scores on the CES-D range from 0 to 60

• A score > 16 indicates mild to moderate depressive symptomotology

• A score > 21 indicates significant depressive symptomotology

21

Clinical Utility of the CES-DClinical Utility of the CES-D

• Screening tool– Can be used to identify those in need of further

assessment and/or treatment targeting depression

• Treatment planning– Total Score indicates the severity of current

depressive symptomotology– Items scored with a 2 or a 3 indicate specific

symptoms of distress to be targeted in treatment

• Monitoring clinical progress– Can be administered weekly, if desired

22

Clinical Utility of the CES-DClinical Utility of the CES-D

• Comparisons of pre/post scores reveal areas of clinical improvement– e.g.,

• Does the youth’s severity of depressive symptomotology decrease substantially?

• Is there improvement across all specific symptoms of significant distress at the time of enrollment in treatment?

23

24

Palette of Measures Data Entry and Palette of Measures Data Entry and Data SubmissionsData Submissions

Palette of Measures Data Entry Palette of Measures Data Entry ShellShell• There is a separate spreadsheet in the excel

workbook (aka database) for each type of data:– Demographics & Services– Pre- General outcome measure– Post- General outcome measure– Pre- Target-specific measure(s)– Post- Target-specific outcome measure(s)

• Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants

25

Palette of Measures Data Entry Palette of Measures Data Entry ShellShell• In addition to the spreadsheets that hold

data...– There is an Instructions spreadsheet

• Basic data entry instructions• Contact information for T.A. (Cricket Mitchell)

– There is a Data Lists spreadsheet at the end of the workbook that you will not use

• Data Lists populate the pull-down menus in other spreadsheets

26

Palette of Measures Data Entry: Palette of Measures Data Entry: CES-DCES-D

27

•There is a separate spreadsheet for Pre- CES-D data, Post- CES-D data, and Mid- CES-D data

•In each spreadsheet, there is a field for the CES-D Total Score

•In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields.

Pre-CES-D   Self-Report (ages 12+)

  Date of  

Client ID# Assessment CESDTot1

     

     

Palette of Measures Data Entry: Palette of Measures Data Entry: CES-DCES-D

28

•For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-CES-D spreadsheet by selecting from the available pull-down menu

•e.g., 1st mid-treatment assessment, 2nd mid-treatment assessment

Mid-CES-D     Self-Report (ages 12+)

  Date of Assessment  

Client ID# Assessment Interval CESDTotM

       

       

But, before you enter any But, before you enter any outcome data, you’ll enter outcome data, you’ll enter Demographics & Service Demographics & Service Delivery Information...Delivery Information...

29

Palette of Measures Data Entry: Palette of Measures Data Entry: Demographics & Services Demographics & Services (1 of 2)(1 of 2)

30

•Use a unique identifier for Client ID#•Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language)

•Dates should be entered as xx/xx/xxxx•Axis I diagnoses s/b the numeric DSM-IV code•Therapist ID is optional

Client Information:                   Primary (DSM-IV code) (DSM-IV code)  

Client ID# DOB Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID

               

               

Palette of Measures Data Entry: Palette of Measures Data Entry: Demographics & Services Demographics & Services (2 of 2)(2 of 2)

31

•Select Focus from pull-down menu (e.g., anxiety, depression)•The shell will hold data for up to 4 foci, or treatment targets

•Enter Date of First Session•The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions)

•2 levels of “Completed?”•Treatment targeting this particular focus•Overall service delivery

Focus 1 of Treatment  

  Date of Date of Total # Completed(if Focus 1 not completed) Completed

(if Services not completed)

Focus1First 

SessionLast 

Sessionof 

Sessions Focus 1? Reason Services? Reason

               

Palette of Measures Data Palette of Measures Data Submissions Submissions (slide 1 of 3) (slide 1 of 3)

• Data submissions to CiMH will occur twice a year throughout the duration of the project– The end of each May (reflecting all clients served from the

initiation of the project through the end of that April)– The end of each December (reflecting all clients served

from the initiation of the project through the end of that November)

– Note that this is the anticipated schedule; actual data submission dates may vary slightly

• An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline

32

Data Submissions Data Submissions (slide 2 of 3) (slide 2 of 3)

• Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH:– Use YouSendIt, or another secure web-based transfer

site, to submit data electronically• YouSendIt (www.YouSendIt.com) is a vendor that supports the

secure transfer of electronic data (encrypted and password-protected)

– Mail a password-protected CD to CiMH and submit the password separately (via email or phone)

– Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone)

33

Data Submissions Data Submissions (slide 3 of 3) (slide 3 of 3)

• After data are submitted, sites continue to enter new data into the same database – Always reflects an ongoing, historical record of

clients served through the Palette of Measures project

• Every effort is made to distribute reports within two months of each data submission– Aggregate and site/agency-specific reports

34

35

QuestionsQuestions

36

The EndThe End

Contact Information•Cricket Mitchell, PhD

•Email: cmitchell@cimh.org•Cell phone: 858-220-6355