Vocational AssessmentsVocational Assessments

Diana ZitelliDiana Zitelli&&

Erin RichardErin Richard

OutlineOutline

IntroductionIntroduction Analysis of 2 vocational Analysis of 2 vocational

assessmentsassessments Enderle-Severson Transition Enderle-Severson Transition

Rating Scales (ESTR-J & ESTR-Rating Scales (ESTR-J & ESTR-III)III)

Vocational Adaptation Rating Vocational Adaptation Rating Scale (VARS)Scale (VARS)

Comparison & conclusionComparison & conclusion

OverviewOverview

Several assessment tools related to Several assessment tools related to vocational curriculum, none vocational curriculum, none specifically created for individuals specifically created for individuals with autismwith autism

““There are not, in my opinion, any There are not, in my opinion, any good vocational assessments for the good vocational assessments for the ASD population.”ASD population.” Peter Gerhardt, Ed.D., President of the Peter Gerhardt, Ed.D., President of the

Organization for Autism ResearchOrganization for Autism Research

ESTR: AuthorsESTR: Authors

Jon Enderle - former teacher of Jon Enderle - former teacher of secondary school students with secondary school students with disabilities and elementary inclusion disabilities and elementary inclusion teacher in Minnesotateacher in Minnesota

Dr. Susan Severson - former teacher Dr. Susan Severson - former teacher of students with severe disabilities, of students with severe disabilities, currently the coordinator of the currently the coordinator of the moderate/severe mentally moderate/severe mentally handicapped teacher training handicapped teacher training program at Minnesota State program at Minnesota State University MoorheadUniversity Moorhead

ESTR: A Brief HistoryESTR: A Brief History

ESTR – 1991, a transition rating ESTR – 1991, a transition rating scale designed for use with scale designed for use with individuals with all disabilities individuals with all disabilities

ESTR-R – revised in 1995 based on ESTR-R – revised in 1995 based on continuing research and feedback continuing research and feedback from users (revisions included more from users (revisions included more comprehensive subscales within comprehensive subscales within several categories, and a more several categories, and a more cohesive sequence of items in order cohesive sequence of items in order of difficulty level)of difficulty level)

ESTR: Current Versions ESTR: Current Versions AvailableAvailable ESTR-J – a revised version ESTR-J – a revised version

developed in 2003 intended for use developed in 2003 intended for use with individuals with MILD disabilities with individuals with MILD disabilities (revisions include removal of items (revisions include removal of items that do not pertain to this population that do not pertain to this population as well as rewording of several items as well as rewording of several items to better reflect skill level of to better reflect skill level of population)population)

ESTR-III – the current version ESTR-III – the current version adapted from ESTR-R in 2003 adapted from ESTR-R in 2003 intended for use with individuals with intended for use with individuals with MODERATE to SEVERE disabilitiesMODERATE to SEVERE disabilities

ESTR: Goals of AssessmentESTR: Goals of Assessment

The transition assessment process should The transition assessment process should provide the student, parent, and/or provide the student, parent, and/or educator with answers to these four major educator with answers to these four major questions:questions: ““What are the student’s desired future What are the student’s desired future

outcomes/goals?”outcomes/goals?” ““What skills does the student possess?”What skills does the student possess?” ““What skills must the student acquire to achieve What skills must the student acquire to achieve

their goals?”their goals?” ““What planning issues need to be addressed to What planning issues need to be addressed to

enhance the student’s opportunities to enhance the student’s opportunities to experience success in vocational, residential experience success in vocational, residential and community environments?”and community environments?”

ESTR: Goals of Assessment ESTR: Goals of Assessment (cont.)(cont.) After completing the ESTR, the student, After completing the ESTR, the student,

parents, and educators should have a parents, and educators should have a comprehensive anecdotal description of the comprehensive anecdotal description of the students current strengths and possible students current strengths and possible areas of concernareas of concern

Responses are divided into three Responses are divided into three categories indicating degree of categories indicating degree of participation; asks whether a student participation; asks whether a student performs the described behaviors: performs the described behaviors: “independently and consistently”, “with “independently and consistently”, “with assistance”, or “does not participate at this assistance”, or “does not participate at this time” time”

Raw “score” of assessment is not of major Raw “score” of assessment is not of major concern, rather the descriptive analysis concern, rather the descriptive analysis generated is considered valuable to generated is considered valuable to develop long term goals and should aid in develop long term goals and should aid in developing IEPsdeveloping IEPs

ESTR: Assessment ESTR: Assessment CategoriesCategories

EmploymentEmployment Recreation and LeisureRecreation and Leisure Home LivingHome Living Community ParticipationCommunity Participation Post Secondary EducationPost Secondary Education

ESTR: Research Supporting ESTR: Research Supporting UseUse No experimental research articles No experimental research articles

foundfound Dr. Severson’s dissertation: Dr. Severson’s dissertation: An An

investigation of the reliability and investigation of the reliability and validity of the ‘Enderle-Severson validity of the ‘Enderle-Severson Transition Rating Scale’Transition Rating Scale’

““Google” search for Google” search for Enderle-Enderle-SeversonSeverson generates about 250 hits, generates about 250 hits, many of which recommend the many of which recommend the ESTR as one of several useful ESTR as one of several useful transition assessmentstransition assessments

ESTR: ProsESTR: Pros

Result are descriptive and narrative, Result are descriptive and narrative, and therefore can be directly and therefore can be directly translated into IEP goalstranslated into IEP goals

Assessment is comprehensive, Assessment is comprehensive, addressing areas across all relevant addressing areas across all relevant skill domains as well as future goalsskill domains as well as future goals

Publishers provide an online Publishers provide an online assessment generator for assessment generator for convenienceconvenience

ESTR: ConsESTR: Cons

Many of the items include “skills” Many of the items include “skills” that we know cannot be that we know cannot be observed or measured observed or measured E.g., developing an understanding E.g., developing an understanding

of insurance, understanding of insurance, understanding concepts of sexual awareness, concepts of sexual awareness, having confidence in oneselfhaving confidence in oneself

ESTR: Suggested ESTR: Suggested ModificationsModifications

A behavior analyst might modify A behavior analyst might modify the ESTR by rewording certain the ESTR by rewording certain items to address skills that are items to address skills that are observable and measurable observable and measurable

VARS: IntroductionVARS: Introduction

Developed to quantify Developed to quantify maladaptive behaviors that maladaptive behaviors that could jeopardize the individuals could jeopardize the individuals ability to gain and maintain ability to gain and maintain employment employment

Looks at both behavioral Looks at both behavioral excesses and deficitsexcesses and deficits

VARS: PurposeVARS: Purpose

Aide in curriculum developmentAide in curriculum development Assist in making placement Assist in making placement

decisionsdecisions Give additional data for the Give additional data for the

evaluation of workersevaluation of workers

VARS: ItemsVARS: Items

133 items organized into 6 133 items organized into 6 categoriescategories Verbal mannersVerbal manners

Calls others derogatory namesCalls others derogatory names Communication skillsCommunication skills

Cannot carry on conversationCannot carry on conversation Attendance & punctualityAttendance & punctuality

Makes unnecessary or questionable Makes unnecessary or questionable trips to the restroomtrips to the restroom

VARS: ItemsVARS: Items

133 items organized into 6 133 items organized into 6 categoriescategories Interpersonal behaviorInterpersonal behavior

Laughs inappropriatelyLaughs inappropriately Respect for property, rules, & Respect for property, rules, &

regulationsregulations Takes food from snack areaTakes food from snack area

Grooming & personal hygieneGrooming & personal hygiene Has body odorHas body odor

VARS: The LearnerVARS: The Learner

Standardized on 606 mentally Standardized on 606 mentally retarded workers in New Yorkretarded workers in New York Majority were adolescents & Majority were adolescents &

young adults (13-30 years)young adults (13-30 years) Mild to moderate mental Mild to moderate mental

retardationretardation

VARS: The AdministratorVARS: The Administrator

No special education required to No special education required to administeradminister TeachersTeachers Teachers’ aidesTeachers’ aides ParentsParents Vocational specialistsVocational specialists

Rater rates behavior after Rater rates behavior after having a minimum of 70 hours having a minimum of 70 hours of contact with the learnerof contact with the learner

VARS: Rating ProcedureVARS: Rating Procedure

Each item rated on a 4 point Each item rated on a 4 point scalescale 4-Never4-Never 3-Sometimes3-Sometimes 2-Often2-Often 1-Regularly1-Regularly

VARS: ScoringVARS: Scoring

FrequencyFrequency Frequency of occurrence as rated Frequency of occurrence as rated

by administratorby administrator SeveritySeverity

Reflects the probability that a Reflects the probability that a given frequency of each problem given frequency of each problem behavior would cause termination behavior would cause termination of employmentof employment

VARS: Tabulation of RatingsVARS: Tabulation of Ratings

Add totals within each category Add totals within each category to obtain raw score for each to obtain raw score for each scalescale

Raw scores converted to deciles Raw scores converted to deciles using a table in the manualusing a table in the manual

Can be crudely graphed on Can be crudely graphed on chartchart

VARS: The researchVARS: The research

Malgady, Barcher, Davis, & Malgady, Barcher, Davis, & Towner (1980). Towner (1980). Conducted at a 6 level sheltered Conducted at a 6 level sheltered

workshop. workshop. Scores on VARS domains Scores on VARS domains

predicted “workers’ placement in predicted “workers’ placement in sheltered workshop significantly sheltered workshop significantly better than a regression model better than a regression model involving only IQ, age, and sex”involving only IQ, age, and sex”

VARS: ProsVARS: Pros

Even if test is not fully Even if test is not fully administered or scored, items administered or scored, items serve as a good checklist from serve as a good checklist from which IEP goals may be which IEP goals may be developeddeveloped

Authors suggest second person Authors suggest second person administers the test to gain IOAadministers the test to gain IOA

Easy to administerEasy to administer

VARS: ConsVARS: Cons

Out of printOut of print Scoring based on administers Scoring based on administers

memory of eventsmemory of events Scoring is subjectiveScoring is subjective Items written in negative, which Items written in negative, which

can be confusingcan be confusing The results, while informative, The results, while informative,

do not help professionals do not help professionals choose a suitable placementchoose a suitable placement

VARS: Cons (cont.)VARS: Cons (cont.)

Some items are unlikely to be Some items are unlikely to be displayed by individuals with displayed by individuals with autism (although perhaps autism (although perhaps individuals with Asperger individuals with Asperger Syndrome)Syndrome) ““Heckles other workers”Heckles other workers”

Some items are not observableSome items are not observable ““Fails to take pride in finished Fails to take pride in finished

work”work”

VARS: Suggested VARS: Suggested ModificationsModifications

Collect data over a specified Collect data over a specified period (e.g., 2 weeks) on period (e.g., 2 weeks) on specific items instead of relying specific items instead of relying on administrator’s memoryon administrator’s memory Define rating scale more preciselyDefine rating scale more precisely

Ignore items that aren’t Ignore items that aren’t observable (although scores observable (although scores could not be tabulated)could not be tabulated)

SummarySummaryESTRESTR VARSVARS

AssessesAssesses Current strengths & possible Current strengths & possible areas of concernareas of concern

Behaviors that may Behaviors that may influence ability to influence ability to obtain and maintain obtain and maintain employmentemployment

ResearchResearch 1 dissertation1 dissertation 2 published articles2 published articles

ProsPros •Result can be directly Result can be directly translated into IEP goalstranslated into IEP goals• Assessment is Assessment is comprehensivecomprehensive•Publishers provide an online Publishers provide an online assessment generator for assessment generator for convenienceconvenience

•Items serve as ides for Items serve as ides for developing IEP goals developing IEP goals •Realizes that problem Realizes that problem behavior is important in behavior is important in determining vocational determining vocational potentialpotential•Easy to administerEasy to administer

ConsCons Many of the items include Many of the items include “skills” cannot be observed or “skills” cannot be observed or measured measured

•Out of printOut of print•Scoring subjectiveScoring subjective•Does not yield Does not yield recommended jobsrecommended jobs

ConclusionsConclusions

An autism specific vocational An autism specific vocational assessment is needed to target assessment is needed to target the unique challenges the unique challenges individuals with autism face in individuals with autism face in the workplace, including the workplace, including assessment of both skill level assessment of both skill level and behavior challengesand behavior challenges

ReferencesReferences

Malagady, R.G, Barcher, P.R., Davis, J., & Malagady, R.G, Barcher, P.R., Davis, J., & Towner, G. (1980). Validy of the vocational Towner, G. (1980). Validy of the vocational adaptation rating scale: Prediction of mentally adaptation rating scale: Prediction of mentally retarded workers’ placement in sheltered retarded workers’ placement in sheltered workshops. workshops. American Journal of Mental DeficiencyAmerican Journal of Mental Deficiency 66, 633-640., 633-640.

Towner, G., Malagady, R.G., Barcher, P.R. Towner, G., Malagady, R.G., Barcher, P.R. & Davis, J. (1980). & Davis, J. (1980). Vocational Adaptation Rating Vocational Adaptation Rating Scale.Scale. Los Angeles: Western Psychological Los Angeles: Western Psychological Services.Services.

ESTR Publications. ESTR Publications. http://www.estr.nethttp://www.estr.net