3_13_2003

8/3/2019 3_13_2003

1/55

Behavior RatingBehavior RatingScalesScales

Definition Types

Construction Issues

Weaknesses Strengths

Selection Considerations

Specific Scales: Conners;CBCL; BASC; others

8/3/2019 3_13_2003

2/55

DefinitionDefinition Rating Scale: any paper and

pencil device where by one

(usually a care taker such asa parent or teacher, though

not excluding peers)

assesses the behavior of thatindividual based on his or her

observations of the child or

adolescent over an extended

period of time (usually morethan a month)

Martin, Hooper & Snow, 1986

8/3/2019 3_13_2003

3/55

Types of Rating ScalesTypes of Rating Scales

Range of constructs from generalfunctioning to concrete behaviors

Personality: Personality Inventory forChildren-Revised (PIC-2); MinnesotaMultiphasic Personality Inventory-

Adolescent (MMPI-A)

Behavior Checklists: Child Behavior

Checklist (CBCL); Conners RatingScales-Revised; Behavior

Assessment System for Children(BASC); Devereux Scales of MentalDisorders

Specific Disorders- ChildrensManifest Anxiety Scales; BeckDepression Inventory; ChildrensDepression Inventory

8/3/2019 3_13_2003

4/55

Summary ofSummary ofConstruction IssuesConstruction Issues

Checklist vs Dichotomy vsContinuum

Item choice

Ability to Sum Scores

Anchors

Description of

Behavior/Construct

8/3/2019 3_13_2003

5/55

Checklist vs DichotomyChecklist vs Dichotomyvs Continuumvs Continuum

Checklists: rater checks the item

of the behavior exists; can beused in screening for specificDSM-IV disorders

Dichotomy: rater indicates of the

behavior exists or does not exist;forced dichotomy; Yes/No

Continuum: 1 2 3 4 5

Increases reliability with more

steps (plateau after 11 steps withlittle gain); Odd number allowsfor a neutral, middle step, but cancreate a response set

8/3/2019 3_13_2003

6/55

Item ChoiceItem Choice

Subjectivity of instrument is a

function of the level of analysis;type of item; manner scaled

Sufficient number of items tosample the construct

Face validity of items

Specificity of behavior: Isdelinquent vs Lies; steals;violates curfew

Too specific may lead to trivialinformation, excessive length

Time frame identified, e.g. Withinthe last two weeks

8/3/2019 3_13_2003

7/55

Various strategiesVarious strategies

used to develop itemsused to develop itemsand scalesand scales

Factor analysis: placing in a

factor items that cluster together Empirical keying: using selected

items to distinguish one groupfrom another

Theoretical constructs: usingselected items to measure thetheoretical constructs underlyingthe construction of the test

Content analysis: using expertsto select items to measure thetrait or diagnostic category ofinterest

8/3/2019 3_13_2003

8/55

Ability to Sum ScoresAbility to Sum Scores

Construction of some testsallows for sum scores acrossscales which increases thereliability of the instrument

Broad band factors havehigher reliability than narrowband, e.g. Internalizing &Externalizing have higher

reliabilities than individualscales such as SocialWithdrawal or Aggression

8/3/2019 3_13_2003

9/55

AnchorsAnchors

End points on a scale

Numerical (Likert scale) Degrees of agree/disagree

Adjectives such as good/bad;carefree/anxious; impulsive/reflective

Actual behavior to typify a type ofattitude such as religion: attendschurch 1 time per months; 2 times permonths; weekly; biweeklyThis maybe specific to the construct; may not

represent equal intervals; may bedifficult to find discreet specificbehaviors

Comparison to norm or product scales

8/3/2019 3_13_2003

10/55

Description ofDescription ofBehavior/ConstructBehavior/Construct

Scales need to be defined

Based on theory

Behaviors which fall under

one construct on one test,

may be utilized on another

construct in another test

8/3/2019 3_13_2003

11/55

Summary ofSummary ofWeaknessesWeaknesses

Disadvantages

Considerations for Misuse

Safeguards

8/3/2019 3_13_2003

12/55

DisadvantagesDisadvantages

Four areas of variation on

assessment data which

summarize the

disadvantages of rating

scales: source variance,setting variance, temporal

variance, and instrument

variance (Martin, Hooper andSnow)

8/3/2019 3_13_2003

13/55

Source VarianceSource Variance

Primary source of error in rating scale data is the

informant

Knowledge of subject for at least 2 months Perceptions of rater

Tolerance of behavior

Stress level of respondent

Choice of informant may slant results

Internalizing behaviors or low rate behaviors

may not be observed

May not recognize the usefulness of the scale

Reading level of informant (30-40% of the

population does not read at a fifth grade level)

Response Bias

8/3/2019 3_13_2003

14/55

Response BiasResponse Bias

Science identifies truth as theconvergence of data

Respondents may differ in

perception, normative lifeexperiences (e.g.urban/suburban; poverty/wealth),response style, and desired

outcome: teacher may want thechild in a program;teacher/parent may not haveobjective view in relation tonormal peers; parent may haveulterior motive such as custody,monetary benefits

Respondents sometimes arebiased without awareness

8/3/2019 3_13_2003

15/55

Reasons for Inadvertent BiasReasons for Inadvertent Bias

Complexity of the mental processesrequired for response lead tobias (Cooper, 1981)

1. Observation of the action

2. Observation encoding,aggregation, & storage in short-term memory

3. Short-term memory decay

4. Transfer to long-term storageand aggregation

5. Long-term memory decay

Above can be influenced byexpectation of respondent

8/3/2019 3_13_2003

16/55

Reasons for Inadvertent BiasReasons for Inadvertent Bias(cont.)(cont.)

6. Presentation of categories to be

rated7. Observation and impression

retrieval from long-term storage

8. Recognition of observations and

impressions relevant to ratingcategory.

9. Comparison of observations andimpressions to raters

standards10.Incorporation of extraneous

considerations

11.Making the rating-weighing the

behavior

8/3/2019 3_13_2003

17/55

Types of Response BiasTypes of Response Bias

May be due to respondents

intentions or characteristic way ofresponding to an item regardless

of content

Halo Effect Leniency or severity

Central tendency or range

restriction

Response acquiescence

Response deviance

Social desirability

8/3/2019 3_13_2003

18/55

Halo EffectHalo Effect

A raters failure to discriminate

among distinct and independentaspects of a ratees behavior(Saal, 1980)

Cognition: rate child positively in

emotional or behavioral issuesbecause they are smart

Socially adept: child must beemotionally or cognitively adept

because of positive socialbehaviors (always helpful,smiles)

Other raters may report

conflicting information

8/3/2019 3_13_2003

19/55

Leniency or severityLeniency or severity

Occurs when ratings are

consistently higher or lower thanare warranted

Inferred when a rater uses

predominantly one extreme or

the other on the scale

Cannot be verified unless an

independent observation or other

party disagrees, e.g. parent seeschild as hyperactive while few

others see him as such

8/3/2019 3_13_2003

20/55

Central tendency orCentral tendency orrange restrictionrange restriction

Rater restricts range of all ratings

to average or above or below(may revert to leniency orseverity bias)

Rater may choose middle

response since they feel they donot know all the universe ofpossible occurrences of thebehavior (e.g. I dont know how

he is with his friends; I only seehim at school/home) thereforecannot rate as AlwaysTrue/False, etc.

8/3/2019 3_13_2003

21/55

ResponseResponse

acquiescence &acquiescence &response devianceresponse deviance

Response acquiescence

tends to agree with each item

Response deviance tends to

respond in a deviant,

unfavorable, uncommon, orunusual way

8/3/2019 3_13_2003

22/55

Social desirabilitySocial desirability

Interpret the test responses

to provide the most favorableview of the child

Rater may not be aware of

the tendency to underrateproblematic responses

Rater may hesitant toendorse items that suggest

the presence of a particulardisorder (e.g. BeckDepression Inventory)

8/3/2019 3_13_2003

23/55

Methods to minimizeMethods to minimize

biasbias Use a lie scale or faking good scale

Switch left and right for positive

responses

Use bipolar adjectives

Response scaling: many problem

behaviors occur in all children,

dichotomy is not adequate (most

children yell, cry, hit at leastsometimes)

Provide clear instructions

Limit number of response categories to

reduce confusion, lack of focus, length Identify at the beginning what the

scales mean and time frame for rating

8/3/2019 3_13_2003

24/55

Setting VarianceSetting Variance

Interaction with the

environment can affect

results, i.e. home/school/

clinic

Interventions used

Consider if instrument is

sensitive across settings or

specific to one setting

8/3/2019 3_13_2003

25/55

Temporal VarianceTemporal Variance

Change in behavior over time

Medication issues

Intervention

Maturation

Significant events: deaths,

divorce, illness, trauma

8/3/2019 3_13_2003

26/55

Instrument VarianceInstrument Variance

Sloppy construction

Definition of construct

Qualitative technical aspects

Quantitative: depth of

information as well as

breadth

8/3/2019 3_13_2003

27/55

Considerations forConsiderations for

MisuseMisuse

May be convenient and efficient

for assessor, but may not be forthe informant

Provide feedback and explain the

instrument

Inappropriate use of instrument

for screening, diagnosis,

intervention development,

program evaluation Choice of an instrument to sway

identification of a specific

condition

8/3/2019 3_13_2003

28/55

SafeguardsSafeguards

Aggregate principle: collect data on same

construct over varied settings withvaried instruments to increase

reliability by controlling the sources of

variance

Test over several time periods Use several instruments

Use several raters

Multi-setting, Multi-source, Multi-

instrument Design Variations in responses may be due to

setting, activity, or rater

Can lead to hypothesis development

8/3/2019 3_13_2003

29/55

StrengthsStrengths

Rating scale is a derivative of theunstructured interview, an evolution ofthe interview in the direction ofincreasing structure

The interview has more variability ininterviewers; does not cover all areas;problems may be missed; clients arenot always willing and articulateinaccurate reporting; reliability and

validity may be poor Rating scale can identify strengths and

weaknesses

Validate referents concern

Evaluate the severity and range of the

concern

Assess atypical patterns

Part of multi-source, multi-methodevaluation

8/3/2019 3_13_2003

30/55

Strengths (cont.)Strengths (cont.)

Several assumptions allow for

the comparison of ratersresponses:

1) Informants can describe or ratethe child

2) Items have the same or similarmeaning for all respondents

3) Respondents report theirthoughts, feelings, & behaviorsopenly and honestly

4) Measures have adequatereliability and validity

8/3/2019 3_13_2003

31/55

Strengths (cont.)Strengths (cont.) Rating scales can tap behaviors

you may not be able to quantifyin other tests

Convenience: time-and cost-efficient for assessor, multipleviewpoints

Comprehensive scales canensure touching range ofproblem areas unlike interviewswhich may delve into oneproblem but miss others

Structured response format andoperationalizing behavior canreduce subjectivity

Increase ecological validity of the

assessment, normal environment

8/3/2019 3_13_2003

32/55

Strengths (cont.)Strengths (cont.) Teacher ratings have high

predictive power; teacher hasformal training, structure setting,comparison to other children

Biases evidenced betweensettings or individuals can beused in assessment andintervention, identify the real

problem (child or referent),parenting style differences,influence of setting

Some rating scales ask

informant to identify the mostproblematic/concerning problem

Child may not be able tointeract/respond to assessment,

e.g. infants, severely impaired

8/3/2019 3_13_2003

33/55

Strengths (cont.)Strengths (cont.)

Use of caretaker as informant is

strength in parents haveobserved child since birth;

parents are motivated; part of

natural environment

More objective and reliable than

projective and interview; can be

less biased than self-report

Can provide information onstrengths as well as concerns

S l tiS l ti

8/3/2019 3_13_2003

34/55

SelectionSelection

ConsiderationsConsiderations Technical considerations:

Norms, validity, reliability,constructs sampled, test

construction Informant, situation, time, client

Scope of instrument: Narrowand/or broad category of

behaviors; Choose for what youneed and want; strengths(competencies) and weaknesses

Purpose or use: screening,

diagnosis, placement,intervention; program evaluation

Clinical Utility: ease ofadministration; useful clinical

information; sensitive to effectsof intervention

8/3/2019 3_13_2003

35/55

Specific ScalesSpecific Scales

BASC

CBCL

Conners

others

BASCBASC BehaviorBehavior

8/3/2019 3_13_2003

36/55

BASCBASC--BehaviorBehavior

Assessment SystemAssessment Systemfor Childrenfor ChildrenTeacher Rating Scale

Preschool 4-5 yrs (109 items)

Child 6-11 yrs (148 items)

Adolescent 12-18(138 items)

Parent Rating Scale

Preschool 4-5 yrs (105 items)Child 6-11 yrs (138 items)

Adolescent 12-13 (126 items)

Self-Report Scale

Child 8-11 yrs (152 items)Adolescent 12-18 yrs (186 items)

Each takes about 30 minutes tocomplete

8/3/2019 3_13_2003

37/55

BASC (cont.)BASC (cont.)

Scores

Teacher and Parent have 4-

point response (never,

sometimes, often, almost

always)Self-Report has true/false

T scores and %ile ranks

Scored by hand on carbonlessforms or computer

8/3/2019 3_13_2003

38/55


Standardization

2,084 Children ages 6-11 and 1,090adolescents 12-18 for parent scale

1,259 children ages 6-11 and 809adolescents 12-18 for teacher scale

5,413 children ages 8-11 and 4,448

adolescents ages 12-18 for Self-Report

Collected 1988-1991, matching 1986U.S. Census

Separate norms for males, females, andclinical samples

About 70% of clinical samples weremales with dx of conduct or behaviordisorder

8/3/2019 3_13_2003

39/55


Reliability

Internal consistency reliabilities for the 3scales in the school age sample range

from .62 to .95 for TRS; .58 to .94 for

PRS, and .61 to .89 for Self-Report

Interrater reliabilities: PRS are generallylow, .35 to .73; TRS from .29 to .70 for

preschool and .44 to .93 for school-

age; none available for adolescents

Test-retest: PRS for 2 to 8 week interval

range from .41 to .94; TRS .59 to .95;Self-Report .57 to .81 for children and

.67 to .81 for adolescents

8/3/2019 3_13_2003

40/55


Validity

Construct validity for internalizingand externalizing dimensions of

the BASC scales is supported by

factor and structural equation

analyses

Criterion-related validity is

satisfactory for the 3 scales, as

show by acceptable correlationswith other similar measures

8/3/2019 3_13_2003

41/55


Integrative approach across

multiple informants Strength is in assessment of

children ages 6 to 11 years,particularly in externalizing

behaviors Separation of Attention &

Hyperactivity; Depression &Anxiety

Limited psychopathology andpersonality domains

Comparison across child andadult forms is difficult

Readability of Self-Report maybe too high

Child B h i Ch kli tChild B h i Ch kli t

8/3/2019 3_13_2003

42/55

Child BehaviorChecklistChild BehaviorChecklist

(CBCL)(CBCL)

Teachers Report Form (TRF)Teachers Report Form (TRF)

& Youth Self& Youth Self--Report (YSR)Report (YSR)

Parent Rating

Preschool 2-3 yrs (99 items)School-age 4-18 yrs (120 items)

Teacher Rating Form

Caregiver/Teacher 2-5 yrs (99items)

School Age 6-18 yrs (120 items)

Youth Self-Report

Ages 11-18 yrs (119 items)Requires 5th grade reading level

about 30 minutes to complete

Parent and Teacher form take

10-15 minutes to complete

8/3/2019 3_13_2003

43/55

CBCL, TRF & YSR (Cont.)CBCL, TRF & YSR (Cont.)

Scores

3-point response (not true,

somewhat true or sometimes

true & , often true)

T scores and %ile ranks

Scored by templates,

scannable answer sheets, or

computer

8/3/2019 3_13_2003

44/55


Standardization

1,200 males and females ages 4-11 and 1,168 adolescents 12-18for parent scale

713 children ages 5-11 and 678

adolescents 12-18 for teacherscale

637 Males and 678 females forSelf-Report

Collected 1989, matching 1990U.S. Census

Separate norms for males&females

8/3/2019 3_13_2003

45/55


Reliability

Internal consistency reliabilities for theparent from .56 to .92; for teacher.63

to .96; and .59 to .90 (males) & .59 to

.89 (females)for Self-Report

Interrater reliabilities: Parent.26 to .86;Teacher from -.05 to .81; none

available for adolescents

Test-retest: Parent for 1 week interval

range from .63 to .97; Teacher .82 to

.95for males & .43 to .99 for females;Self-Report .47 to .81 for 50 children

ages 11 to 18

8/3/2019 3_13_2003

46/55


Validity

Concurrent validity for parent,teacher, and YSR forms is

satisfactory, acceptable

correlations with Conner

Discriminant validity for parent and

teacher forms is acceptable and

satisfactory for YSR shown by

significant differences in scores

between referred and

nonreferred samples

8/3/2019 3_13_2003

47/55


Does not provide validity

scales

Support cross-informant

assessment

Low levels of reliability,suggesting caution in their

interpretation and application

Broad-based screeningmeasure rather than a

precise measure of disorder

8/3/2019 3_13_2003

48/55

Conners RatingConners Rating

ScalesScales--RevisedRevised

Parent and teacher versions

are designed for ages 3-17

Self-report is for ages 12-17

years

Short forms (@ 27 items) and

long forms (59-87 items) are

available

8/3/2019 3_13_2003

49/55



Scores

4-point response (not true at

all, just a little true, pretty

much true, & very much true)

T scores

Scored by self-scoring sheet or

computer scored with

interpretive report

8/3/2019 3_13_2003

50/55



Standardization

8,000 individuals drawn from 1993to 1996 from 45 U.S. states and

10 Canadian provinces.

Norms are provided separately for

males and females by age levels

Does not match U.S. Census as

there are more Euro-Americans

than in general population

8/3/2019 3_13_2003

51/55



Reliability

Internal consistency reliabilities forthe parent and teacher from .73

to .96; for adolescent .75 to .92

Test-retest: Parent and teacher

forms are variable for long and

short forms, with better

reliabilities for the short form over

a 6-8 week retest; self-report

form ranges from .72 to .89

between the two forms.

8/3/2019 3_13_2003

52/55



Validity

Construct validity is satisfactorybased on factor analysis used toconstruct the scales

Convergent validity is good, high

correlations between long andshort forms

Criterion validity is good, highcorrelations between various

versions of the scalesDiscriminant validity for parent and

teacher forms is good significantdifferences in scores betweenreferred and nonreferredsamples

8/3/2019 3_13_2003

53/55



Improvement over previous

scales Standardization samples are

small for any age group or

gender

Adequate to good reliability and

adequate validity, with informant

versions strong in evaluating

externalizing problems

Self-report is useful for

measuring general distress

8/3/2019 3_13_2003

54/55

OthersOthers

Devereux Scales of Mental

Disorders: Good reliability butlimited validity; limited in its

evaluation of psychopathology;

some items include content that

is difficult for parents andteachers to evaluate; not clearly

aligned to DSM-IV, although this

was an objective

Scales specific to ADHD or otherdiagnosis: many have limited

sample size and limited utility

8/3/2019 3_13_2003

55/55

ReferencesReferences

Knoff, H. M. (2002). Best practices inpersonality assessment. In A. Thomas& J. Grimes (eds) Best practices inschool psychology IV, Vol. 2.Bethesda, MD: National Association ofSchool Psychologists

Martin, R., Hooper, S., & Snow,J.(1986). Behavior rating scaleapproaches to personality assessmentin children and adolescents. In H.Knoff (Ed.) The assessment of childand adolescent personality. New York:Guilford Press.

Sattler, J.M. (2002). Assessment ofchildren: Behavioral and clinicalapplications, (4th ed.). San Diego:

Jerome M. Sattler, Publisher, Inc.

Date post:	06-Apr-2018
Category:	Documents
Upload:	neelam-upadhyay
View:	213 times
Download:	0 times

3_13_2003

Documents