Improving the Oklahoma Modified Alternate Assessment Program General Supervision Enhancement Grant (GSEG) 2007-2010Renée Cameto, Principal Scientist and Katherine Nagle, Project Director

Contact Us: Renée Cameto - renee.cameto@sri.com Katherine Nagle - katherine.nagle@sri.com

Technical assistance in three focus areas:

Accessibility of the AA-MAS (OMAAP) reading assessment

Cognitive interview studyDesigned professional development module for teachers to instruct and improve students test taking strategies

Students from most disability categories took the AA-MASStudents most likely to take the AA-MAS - Learning disability - Other health impairments - Intellectual disability - Emotional disturbance

Trouble with organizing and keeping track of their work Required frequent clarification of instruction and one on one supportDifficulty with memorizationDifficulty finishing assignments and were easily distracted

Academic learning characteristics of students taking the AA-MAS, 2008

Reading - Read slowly - Difficulty drawing inferences from grade level text - Difficulty answering comprehension questions on long passages - Limited awareness of narrative or expository text structures - Difficulty identifying the main ideaMathematics - Difficulty with problems requiring multi-step solutions - Difficulty understanding and applying mathematics procedures - Slow or inaccurate retrieval of basic math facts

Professional development needs of teachers of AA-MAS eligible students, 2008

Teachers had many professional development needsMore than half of the teachers wanted training in: - Teaching students organization and study skills - Improving student’s problem-solving skills - Increasing student’s persistence on tasks - Strategies for teaching comprehension of narrative text

Two stage sampling procedure:

First stage stratified district sampling of small, medium, and large districts Second stage random selection of special education teachers from each selected district - Small districts: 2 teachers - Medium districts: 4 teachers - Large districts: 22 teachers

The total number of teachers selected:

404 in 2007-08 422 in 2008-09

Response rates:

66% in 2007-08 74% in 2008-09

Technical quality of data and documentation of the AA-MAS (OMAAP)

Expert review of technical documentation of OMAAPRecommendations for improvement of documentationRecommendations for studies to be conducted to support validity and alignment - Comparison of scores on AA-MAS (OMAAP) and regular assessment (OCCT) for same student

State guidelines for assessment eligibility decisions and standards-based IEPs (OMAAP)

Revise eligibility guidelines to reflect - Federal regulations - State requirementsRevise guidelines for developing standards-based IEPsConduct annual monitoring survey Oklahoma Special Education Teacher Survey (OSETS) that assessed teachers’ knowledge of - Eligibility guidelines - Guidelines for developing and implementing standards-based IEPs - Student characteristics - Professional development needs

What are the professional development needs of teachers of AA-MAS eligible students?What are the disability and behavioral and learning characteristics of students who take AA-MAS?

Background

Data Sources and Sampling

Research Questions

Results

Percentage of teachers reporting that they had at least one student participating in the AA-MAS in the following disability categories:

Percent100806040200

52%

9%

7%

12%

91%

44%

0%

11%

16%

70%

Intellectual disability

29%Autism

Hearing impairment/deafness

Traumatic brain injury

31%Speech and language impairment

Visual impairment/blindness

Learning disability

Emotional disturbance

Deaf-blindness

Orthopedic impairment

Multiple disabilities

Other health impairment

Disability categories of AA-MAS test takers, 2008

Percentage of respondents indicating that one or more students assessed on the AA-MAS in their classrooms exhibited the following behavioral characteristics all or most of the time:

47%Do not follow directions or instructions

35%Avoid participating in class discussions

74%Require frequent clarification ofinstructions and one on one support

72%Have difficulty with memorization

40%Do not consider the consequencesof their behavior

66%Have difficulty finishing assignments and are easily distracted

50%Struggled to sit still and fidgeted

75%Have trouble with organization andkeeping track of work

32%Have trouble following school rules

Behavioral characteristics of students taking the AA-MAS, 2008

Percent100806040200

Percentage of respondents indicating that one or more students assessed on the AA-MAS exhibited the following academic learning characteristics all or most of the time:

66%Limited vocabulary compared with same age peers

90%Read slowly

60%Lose spot when reading

88%Difficulty drawing inferences from grade level text

56%Difficulty predicting what may happen next in a story

88%Difficulty answering comprehensionquestions on long passages

80%Difficulty identifying the main idea of grade level texts

82%Limited awareness of text structures

59%Poor sight word recognition skills

84%Difficulty with problems requiring multi-step solutions

63%Problems forming mental representationsof mathematical concepts

70%Slow or inaccurate retrieval of basic math facts

56%Difficulty comparing/classifying/sortingitems according to a specific criteria

63%Difficulty with number concepts

73%Difficulty understanding/applying math procedures

45%Math anxiety

Percent 100806040200

Percentage of teachers indicating additional professional development needs for 2008-09:

44%

50%

57%

50%

49%

43%

70%

60%

78%

Increase reading oral reading fluency

Improve vocabulary acquisition

Improve comprehension of narrative text

Improve expository text

Improve retrieval of basic math facts

Improve computational skills

Improve problem-solving skills

Increase task persistence

Improve organization and study skills

Percent 100806040200