Cricket Mitchell, PhD

CIMH Evaluation Consultant

CiMH Palette of Measures CiMH Palette of Measures Evaluation Training: How I Think Evaluation Training: How I Think

Questionnaire (HIT)Questionnaire (HIT)

Palette of Measures Evaluation: Palette of Measures Evaluation: What You Will Need What You Will Need (slide 1 of 2)(slide 1 of 2)

• Palette of Measures Data Entry Shell v2 (Excel file)

– Developed by CiMH and customized for each participating agency

– Holds all data for clients served thru the Palette of Measures project

– Demographics, service delivery information, pre- and post- outcome measure data

• Palette of Measures Data Dictionary v2 (Word document)

– A guide for using the associated data entry shell– Defines each column in the excel file

2

Palette of Measures Evaluation: Palette of Measures Evaluation: What You Will Need What You Will Need (slide 2 of 2)(slide 2 of 2)

• Outcome measures from the two-pronged approach– General Outcome Measure– Target-Specific Outcome Measure(s)– For example…

• How I Think Questionnaire © (HIT)– Copyrighted and published by Research Press

• Must be purchased by interested Palette of Measures project participating agencies

3

Overview of Training Overview of Training • Brief Overview of Palette of Measures evaluation

protocol• How I Think Questionnaire©: Target-specific

measure for disruptive behavior– Administration– Scoring– Clinical Utility

• Instructions for Palette of Measures data entry and data submissions– Data entry: HIT– Data entry: Demographics & Services– Data submissions to CiMH

4

5

Brief Overview of Palette of Brief Overview of Palette of Measures Evaluation ProtocolMeasures Evaluation Protocol

Outcome AssessmentOutcome Assessment• Palette of Measures providers will track outcomes

using data from pre- and post- administrations of standardized measures of functioning

• Pre- and Post- a “dose” of treatment / an intervention interval– General measure of youth mental health functioning

(e.g., YOQ/YOQ-SR, CANS, Ohio Scales)– Target-specific measure linked to focus of

treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS)

• Providers may choose to administer mid-course assessments as well– e.g., at 3-month intervals in usual care

6

A note about the use of standardized A note about the use of standardized assessment measures… assessment measures… (slide 1 of 2)(slide 1 of 2)

• Assessment is the beginning of developing a relationship with the child and family– Demonstrates a desire to know what the child

and family are experiencing– By incorporating standardized assessment

measures of functioning, the efficiency and thoroughness of assessment is enhanced

7

A note about the use of standardized A note about the use of standardized assessment measures… assessment measures… (slide 2 of 2)(slide 2 of 2)

• Using standardized assessment measures of functioning…– Assists in initial clinical impressions– Provides valuable information to guide

treatment/interventions– Assesses sufficiency of treatment delivered– Demonstrates treatment-related improvements

in child functioning

8

9

How I Think Questionnaire© (HIT)

CiMH HIT TrainingCiMH HIT Training• Information on the administration, scoring,

and clinical utility of the HIT was obtained from the measure’s Manual, written by Alvaro Q. Barriga, John C. Gibbs, Granville Bud Potter and Albert K. Liau, and published by Research Press

10

How I Think Questionnaire© (HIT) Description

• Target-specific measure for disruptive behavior

• Measures youth tendencies to engage in self-serving cognitive distortions, or thinking errors– Youth self-report for ages 12-19

• 5-15 minutes to complete

11

How I Think Questionnaire© (HIT) Description

• Written at a 4th grade reading level

• Valid and reliable

• Copyrighted and published by Research Press– After purchase of the manual (~$2) the cost per

use ~$1.20• **20% discount if purchased through CIMH

– http://www.researchpress.com/product/item/5136/

12

HIT© Description• 54 items• 6-point Likert scale

response options– Disagree strongly (1)– Disagree (2)– Disagree slightly (3)– Agree slightly (4)– Agree (5)– Agree strongly (6)

• Eight Subscale Scores– Self-Centered (SC)– Blaming Others (BO)– Minimizing/Mislabeling (MM)– Assuming the Worst (AW)– Opposition-Defiance (OD)– Physical Aggression (PA)– Lying (L)– Stealing (S)

• Two Summary Scale Scores– Overt– Covert

• HIT Total Score

13

Example: Items from the HIT©

• People should try to work on their problems.

• If I see something I like, I take it.• When I get mad, I don’t care who gets

hurt.• Everybody lies, it’s no big deal.• You should get what you need, even if it

means someone has to get hurt.

14

HIT AdministrationHIT Administration• Administer pre- and post- a dose of

treatment, or an intervention interval, focused on disruptive behavior– HIT completed by client– Some agencies may choose to administer

mid-treatment assessments as well

15

Let’s take a look at the Let’s take a look at the HIT... HIT... (not distributed due to copyright laws)(not distributed due to copyright laws)

16

How I Think Questionnaire© Scoring• Use the HIT Questionnaire Computational Form

– Only one is provided with each HIT manual, this form will need to be copied to score each HIT

• For each item, transfer the value corresponding to the circled response into box under the appropriate scale (item # in parentheses)– Values for each item response are on slide 13– Each item loads onto one of the nine scales

• Determine each scale’s Mean score– Sum all responses and divide by the number of items in

that scale• Determine the Mean Summary Scores (e.g. Overt

Scale, Covert Scale, Overall HIT Score)

17

HIT Scoring• Missing Data (items that are left blank)

– If 5 or more items are missing, consider the questionnaire invalid

– If an item is left blank, it is not coded and will not contribute to the scale score in which it falls

• Note that this will change the denominator (divisor) in determining the scale score, which is the mean, or average of all scale items

• Two Responses to One Question– If a youth has circled two responses to an item, take

the higher value of the two

18

HIT Scoring• Use the HIT Questionnaire Profile Form to

plot a youth’s scores– As with the HIT Questionnaire Computational

Form, only one is provided with each HIT manual, this form will need to be copied to score each HIT

• The Profile Form shows youth scores that are above the clinical cutpoint, are borderline-clinical, or are in the nonclinical range

19

Clinical Utility of theClinical Utility of theHITHIT• HIT Total Score/Overall HIT Score

– Possible scores range from 1.0 to 6.0– Clinical cutpoint

• 2.98 or higher– Total Scores are reflective of self-serving cognitive

distortions, or thinking errors, reported by youth– High scores reflect more cognitive distortions that are

likely related to antisocial behaviors; whereas, low scores reflect more normative, non-clinical thought processes

• It is important to examine the profile of elevations across all summary scales and subscales

20

Clinical Utility of the Clinical Utility of the HITHIT• HIT Subscale Scores

– Possible scores range from 1.0 to 6.0– Clinical cutpoints vary (see slide 26)– Each subscale is named for its item content

• e.g., the Blaming Others subscale contains items that assess youth tendencies to blame others for their actions, the Lying subscale contains items that assess youth condoning and engaging in lying

– Higher scores reflect a higher endorsement of that thought process or behavior

21

Clinical Utility of the Clinical Utility of the HITHIT• HIT Subscale Scores

– Examining the profile of cognitive distortion scales should provide a method for directly assessing the particular cognitive style displayed by a youth

– Permits clinicians to individualize treatment approaches more effectively

• e.g., youth with elevated Self-Centered scores are likely to require cognitive interventions that address a strong egocentric bias

22

Clinical Utility of the Clinical Utility of the HITHIT• Overt Summary Scale

• Opposition-Defiance (OD) and• Physical Aggression (PA)

– The Overt Scale reflects a predilection for antisocial behavior that typically involves confrontation of a victim

• Covert Summary Scale• Lying (L) and• Stealing (S)

– The Covert Scale reflects a preference for antisocial behavior that is primarily nonconfrontational

23

Clinical Utility of the Clinical Utility of the HITHIT• Treatment planning will likely vary for

youth with high Overt Scale scores vs. youth with high Covert Scale scores– Numerous studies have found differences in

the etiological factors, symptom presentation, and long-term prognosis between youth who display overt aggression and youth who engage in delinquency that is more covert

24

Clinical Utility of the How I Think Questionnaire©

• Assesses a variety of specific aspects of self-serving youth thinking errors that are likely to contribute to aggression and delinquent behavior– e.g., blaming others, assuming the worst, minimizing

and mislabeling• Informs which aspects of thinking errors and

associated antisocial behavior youth have the greatest need for intervention

• Comparisons of pre/post scores demonstrate treatment-related outcomes in maladaptive thought processes

25

Summary of HIT Score Ranges Summary of HIT Score Ranges and Clinical Cutpointsand Clinical Cutpoints

26

HIT Score(range 1-6)

Clinical Cutpoint

HIT Score(range 1-6)

Clinical Cutpoint

Self-Centered (SC) 3.16 Lying (L) 3.39

Blaming Others (BO) 3.08 Stealing (S) 2.57

Minimizing/Mislabeling (MM) 2.96

Assuming the Worst (AW) 2.92 Overt Summary Score 3.06

Opposition-Defiance (OD) 3.20 Covert Summary Score 2.96

Physical Aggression (PA) 3.01 HIT Total Score 2.98

27

Palette of Measures Data Entry and Palette of Measures Data Entry and Data SubmissionsData Submissions

Palette of Measures Data Entry Palette of Measures Data Entry ShellShell• There is a separate spreadsheet in the excel

workbook (aka database) for each type of data:– Demographics & Services– Pre- General outcome measure– Post- General outcome measure– Pre- Target-specific measure(s)– Post- Target-specific outcome measure(s)

• Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants

28

Palette of Measures Data Entry Palette of Measures Data Entry ShellShell• In addition to the spreadsheets that hold

data...– There is an Instructions spreadsheet

• Basic data entry instructions• Contact information for T.A. (Cricket Mitchell)

– There is a Data Lists spreadsheet at the end of the workbook that you will not use

• Data Lists populate the pull-down menus in other spreadsheets

29

Palette of Measures Data Entry: Palette of Measures Data Entry: HITHIT

30

•There is a separate spreadsheet for Pre-HIT data, Post-HIT data, and Mid-HIT data

•In each spreadsheet, there is a separate field for each of the HIT Scale Scores, Subscale Scores, and Total Scores

•In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields.

Mid-HIT Date ofAssessm

ent

Self-Report (ages 12-19)

How I Think Questionnaire Mean Total, Scale, and Subscale Scores

Client ID#Assessm

ent IntervalHITTotM

HIT-OVM

HIT-COVM

HIT-SCM

HIT-BOM

HIT-MMM

HIT-AWM

HIT-ODM

HIT-PAM

HIT-LYM

HIT-STM

HIT-ARM

   

Palette of Measures Data Entry: Palette of Measures Data Entry: HITHIT

31

•For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-HIT spreadsheet by selecting from the available pull-down menu

•e.g., 1st mid-treatment assessment, 2nd mid-treatment assessment

Mid-HIT Date of

Assess-ment

Self-Report (ages 12-19)

How I Think Questionnaire Mean Total, Scale, and Subscale Scores

Client ID#Assess-ment

IntervalHITTotM

HIT-OVM

HIT-COVM

HIT-SCM

HIT-BOM

HIT-MMM

HIT-AWM

HIT-ODM

HIT-PAM

HIT-LYM

HIT-STM

HIT-ARM

   

But, before you enter any But, before you enter any outcome data, you’ll enter outcome data, you’ll enter Demographics & Service Demographics & Service Delivery Information...Delivery Information...

32

Palette of Measures Data Entry: Palette of Measures Data Entry: Demographics & Services Demographics & Services (1 of 2)(1 of 2)

33

•Use a unique identifier for Client ID#•Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language)

•Dates should be entered as xx/xx/xxxx•Axis I diagnoses s/b the numeric DSM-IV code•Therapist ID is optional

Client Information: Primary (DSM-IV code) (DSM-IV code)

Client ID# DOB Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID

Palette of Measures Data Entry: Palette of Measures Data Entry: Demographics & Services Demographics & Services (2 of 2)(2 of 2)

34

•Select Focus from pull-down menu (e.g., anxiety, depression)•The shell will hold data for up to 4 foci, or treatment targets

•Enter Date of First Session•The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions)

•2 levels of “Completed?”•Treatment targeting this particular focus•Overall service delivery

Focus 1 of Treatment

Date of Date of Total # Completed(if Focus 1 not

completed) Completed(if Services not

completed)

Focus1First

SessionLast

Sessionof

Sessions Focus 1? Reason Services? Reason

Palette of Measures Data Palette of Measures Data Submissions Submissions (slide 1 of 3) (slide 1 of 3)

• Data submissions to CiMH will occur twice a year throughout the duration of the project– The end of each May (reflecting all clients served from the

initiation of the project through the end of that April)– The end of each December (reflecting all clients served

from the initiation of the project through the end of that November)

– Note that this is the anticipated schedule; actual data submission dates may vary slightly

• An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline

35

Data Submissions Data Submissions (slide 2 of 3) (slide 2 of 3)

• Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH:– Use YouSendIt, or another secure web-based transfer

site, to submit data electronically• YouSendIt (www.YouSendIt.com) is a vendor that supports the

secure transfer of electronic data (encrypted and password-protected)

– Mail a password-protected CD to CiMH and submit the password separately (via email or phone)

– Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone)

36

Data Submissions Data Submissions (slide 3 of 3) (slide 3 of 3)

• After data are submitted, sites continue to enter new data into the same database – Always reflects an ongoing, historical record of

clients served through the Palette of Measures project

• Every effort is made to distribute reports within two months of each data submission– Aggregate and site/agency-specific reports

37

38

QuestionsQuestions

39

The EndThe End

Contact Information•Cricket Mitchell, PhD

•Email: cmitchell@cimh.org•Cell phone: 858-220-6355