SECONDARY LITERACY:UNIVERSAL SCREENING AND PROGRESS MONITORING

David Putnam, Jr., Ph.D.Associate Director, C & ITigard Tualatin School District

OBJECTIVES OF THE PRESENTATION

Provide a rationale and framework for literacy intervention at the secondary level

Examine the Maze and its relationship to OAKS Detail the process for using Maze for Universal

Screening, program evaluation, and identifying students in need of additional support

Describe progress monitoring effectiveness, and procedures for analyzing performance and instructional decision making

WHAT’S THE PROBLEM?

Teaching reading is often considered an elementary school task despite….. More than 8 million students in grades 4 – 12 are

struggling readers (USDoE 2003). 40% of HS students cannot read well enough to

benefit from their textbooks (NAEP, 2005). In Oregon in 2006-07, 33% of 8th graders and 35%

of 10th graders (35%) did not meet OAKS reading.

The problem is more severe when we disaggregate data by racial and special program subgroups.

• High Expectations for Student Achievement--And Always Increasing

• Students w Moderate to Severe Educational and/or Behavioral Needs--Big Prerequisite Skill Deficits

• Students with a Long History of Failure--Poor Motivation and Lots of Escape Driven Behavior

• General Education Teachers with Limited Support Skills and Instructional Materials

• Students’ Programs Being Driven by Graduation Requirements Rather Than Instructional Needs

Mark R. Shinn, Ph.D. & Madi Phillips, Ph.D.NASP, 2007

SECONDARY EDUCATION FOR MANY STUDENTS

TTSD SECONDARY LITERACY PLAN: Focus resources on teaching

literacy strategies proven to increase achievement for all students across all content areas

Execute a comprehensive literacy intervention model to address students in need of strategic and intensive interventions

Use a Three Tier Model morphed to secondary schools

EFFECTIVE SECONDARY INSTRUCTION:A THREE TIER MODEL

All students, IN EVERY TIER, have access to embedded literacy strategies across content areas

Strategies: Frayer Model Anticipation Guide Word Sorts DR/TA or KWL Group Summarizing Definition Word

Chart Differentiated

Assessment

TIER I: WHAT DO ALL STUDENTS RECEIVE?

Core Curriculum Access to Content

Literacy Strategies A limited number of

students are monitored by the Literacy Specialist

Target = 80% of student population

TIER II: WHAT DO STUDENTS RECEIVE IN ADDITION TO THE CORE?

Content Literacy Strategies Across the Content Areas

Strategic Intervention Middle School: Soar

to Success High School: Literacy

Strategies Classes

Target = >15%

Student Population

TIER III: WHAT DO STUDENTS RECEIVE IN ADDITION TO THE CORE?

Content Literacy Strategies Across the Content Areas

Comprehensive reading and writing support LANGUAGE! (High

School) LANGUAGE! (Middle

School)

WHAT IS UNIVERSAL SCREENING?

Universal screening is the process of efficiently assessing ALL students in a timely manner to analyze: The effectiveness of

curriculum, instruction, and school organization

Students’ level of proficiency in essential academic areas

Identify student that MAY need additional help

WHY UNIVERSAL SCREENING?DETERMINE PROGRAM EFFECTIVENESS

Are 80% of our students meeting the benchmark? 80% by ethnicity? By program sub-group? By subject?

WHY UNIVERSAL SCREENING? PROGRAM EVALUATION

Helps you to determine if the core curriculum needs to be addressed Intensity Fidelity Targeted Group size Instructional skills

WHY UNIVERSAL SCREENING?IDENTIFY STUDENTS IN NEED OF SUPPORT

Periodic and universal screening ensures that no students “fall through the cracks”

Strategic support: Students are placed in a program that provides moderate intervention and progress monitored every 2 weeks

Intensive support: Students are placed in an intervention that is intense and progress monitored every 2 weeks

WHAT UNIVERSAL SCREENERS ARE USED WITH SECONDARY STUDENTS IN TTSD?

MAZE OAKS Grades Attendance Office Discipline Referrals

(ODRs)

WHAT IS MAZE? Multiple-choice cloze task

Grade-level passage w/ every 7th word replaced by 3 word choices in parenthesis

Student reads silently and selects as many correct words as they can in 3 minutes

Curriculum-Based Measurement test that is “INDICATOR” of overall reading health Combines fluency, comprehension, and

all other subsumed reading skills Can be administered to a group;

scored later Easy & quick to administer, multiple

forms

EXAMPLE OF MAZE PASSAGE

WHY MAZE?

Allows for screening/assessing ALL students, ALL groups of Students, and School-wide literacy in time for intervention

Can use same test to monitor progress

Frequent progress monitoring increases academic achievement

Maze scores are a predictor of performance on OAKS AND NOW HS graduation

TABLE OF PROBABLE SUCCESS (MN)Maze Correct

Choices3 minutes

Writing: CWS minus IWS7 minutes

Probability of Passing

Minnesota Basic Skills Test

4 9 10%

7 33 20%

10 53 30%

12 70 40%

14 83 50%

16 100 60%

19 116 70%

22 137 80%

26 162 90%

37 210 100%Critical values corresponding to likelihood of passing 8th grade

Minnesota Basic Skills Test – Doug Marston, et al.

MAZE/OAKS CORRELATIONS IN TTSD:SPRING MAZE OAKS BEST SCORE

Grade Median Score

Passage 1 Passage 2 Passage 3

6 .660 .607 .668 .636

7 .689 .615 .649 .706

8 .684 .634 .701 .661

• All correlations moderate to high• Relatively consistent across passages• Median correlations “in the middle”

MAZE AND PROBABILITY OF SUCCESS ON OAKS (TTSD)

Grade

Fall Maze Score Needed

for 85% Probability of Passing OAKS

Reading in Spring

Spring Maze Score Needed for 85%

Probability of Passing OAKS

Reading in Spring

6 20 33

7 20 33

8 21 37

VARIABILITY ACROSS PASSAGES

6th Grade 7th Grade 8th Grade

P1 20.275935828877 22.1125265392781 23.7958287596048

P2 25.7754010695187 23.6082802547771 20.3075221238938

P3 23.5080213903743 26.009564293305 24.9328932893289

Median 23.4903640256959 23.855472901169 22.9835345773875

2.50

7.50

12.50

17.50

22.50

27.50

Single Maze Passages vs Median Scores, Fall, 2008

# o

f C

orr

ect

Wo

rds

MAZE SCREENING IN MS IN TTSD

All students screened 3 times per year Three, 3 minute tests will be given each time

Screening assessment will occur in (Matrix/Trek/LA class)

Tests will be scored and data entered by (Classified Staff/Parent volunteers/Electronically)

Data will be used for program evaluation and to place students in support

Students in support will be monitored

HOW DO YOU ANALYZE EFFECTIVENESS OF ACADEMIC PROGRAMS?

Focused on MAZE, OAKS and Grades Queried ESIS for a demographic file

with student name, ID #, ethnicity, program subgroup

Merged demographic file with data file for each measure

Created an Excel template organized by all subgroups

HOW DO YOU ANALYZE PROGRAM EFFECTIVENESS USING OAKS AND MAZE?

Core Data Analysis MAZE, OAKS, Grades blank template MAZE, OAKS, Grades Data Example

HOW TO IDENTIFY STUDENTS IN NEED OF INTERVENTION?

Initial Screening: Screening process initiated when

academic skills fall at or below the 35% on OAKS, AND/OR

In Middle Schools: Bottom 20% of students on the MAZE-CBM/Maze Benchmark scores

Screen further with San Diego Quick, SRAI, and curriculum placement tests, when appropriate

HOW TO IDENTIFY STUDENTS IN NEED OF INTERVENTION?

Post Screening Diagnostics and Placement: 6-Minute Solution--check for fluency &

accuracy; then, San Diego Quick to determine level of SRAI

to use; then, SRAI to gauge comprehension skills; then, Language! placement tests are

administered for students with the most significant reading needs

HOW DO YOU IDENTIFY THE LOWEST 20% OF STUDENTS WITH MAZE DATA?

Example Excel file

Example of IPAS School Student list

PROGRESS MONITORING

What is progress monitoring? What are the effects of progress

monitoring? How do you conduct progress

monitoring at the secondary level? How do you decide if the intervention

is working?

WHAT IS PROGRESS MONITORING?

An on-going, systematic approach to gathering academic and behavioral data to evaluate response to intervention, thereby

allowing data-based decision-making regarding instruction and learning outcomes on a frequent basis.

help schools establish more effective programs for children who have not benefited from previous programming.

In other words, it tells us if our interventions are working

EFFECTS OF PROGRESS MONITORING

• Progress monitoring has been extensively researched in Special Education (Fuchs & Fuchs, 1986)

• Students showed improved reading scores when teachers:• monitored their progress (+.70 effect size; ≈ 25th

50th %ile. Like it!)• graphed their reading scores (+.80 effect size. Love

it!)• used decisions rules to determine whether to make a

change in instruction (+.90 effect size. Gotta have it!)

EFFECTS OF PROGRESS MONITORING

CBM with decision rules (Fletcher, et.al., 2006) “goal raising rule” for students responding well:

effect size .52 (≈ 25th 40th %ile) “change the program rule” for students not

responding well: effect size .72 (≈ 25th 50th %ile) Results in teachers planning more comprehensive

reading programs

Additional support for effectiveness in General Education(Fuchs, et al., 1994)

PROGRESS MONITORING IN SECONDARY SCHOOLS

Select assessment tools Maze

Determine how often to progress monitor Every 2 weeks

Identify & Train staff to: Administer & score Reading Teacher Input & Analyze data Instructional Coordinator

Use the data Intervention planning at “20%” monthly

meetings Student feedback

Sanford & Putnam (2007)

32

MEET MONTHLY TO CONSIDER…

1. Continuing (Student is making progress, but, continues to need support)

2. Intensifying (Intervention is not working and should be revised), or

3. Referring for Special Education Evaluation (Intensive intervention is proving unsuccessful)

4. Exiting (Intervention no longer needed)

EBIS Intervention Progress Monitoring

2

45

6

25

0

5

10

15

20

25

30

9/15

/ 200

7

10/ 1

5/20

07

11/ 1

4/20

07

12/ 1

4/20

07

1/13

/ 200

8

2/12

/ 200

8

3/13

/ 200

8

4/12

/ 200

8

5/12

/ 200

8

6/11

/ 200

8

Monitoring/ Correct

Baseline/ Benchmark

Aimline

Trendline

Grade: 8

Probe Name: Maze Liz Ryan

Teacher Name: Goldman

School: The School of Rock

Cor

rect

Res

pons

es/M

in.

Intervention: Soar

Intervention Change:

Language C

3-4 Data Points

Below the Aimline!

EBIS Intervention Progress Monitoring

2

45

6

12

16

20

26 25

0

5

10

15

20

25

30

35

9/15

/ 200

7

10/ 1

5/20

07

11/ 1

4/20

07

12/ 1

4/20

07

1/13

/ 200

8

2/12

/ 200

8

3/13

/ 200

8

4/12

/ 200

8

5/12

/ 200

8

6/11

/ 200

8

Monitoring/ Correct

Baseline/ Benchmark

Aimline

Trendline

Grade: 8

Probe Name: Maze Liz Ryan

Teacher Name: Armeli

School: The School of Rock

Cor

rect

Res

pons

es/M

in.

Intervention: Language C

Intervention Change:

Language C

Now that’s WORKIN’!

EBIS Intervention Progress Monitoring

2

45

67

6

8

25

0

5

10

15

20

25

30

9/15

/ 200

7

10/ 1

5/20

07

11/ 1

4/20

07

12/ 1

4/20

07

1/13

/ 200

8

2/12

/ 200

8

3/13

/ 200

8

4/12

/ 200

8

5/12

/ 200

8

6/11

/ 200

8

Monitoring/ Correct

Baseline/ Benchmark

Aimline

Trendline

Grade: 8

Probe Name: Maze Rachael Cifuentes

Teacher Name: Helton

School: The School of Rock

Cor

rect

Res

pons

es/M

in.

Intervention: Language C

Intervention Change:

Language C

3-4 Data Points Below the Aimline!

Consider SPED

Referral

EXIT FROM INTERVENTION WHEN:

Maze scores indicate 4 or more data points above the aimline AND are at or above the 50th percentile; AND

Grade+ scores are at or above the 5th stanine; AND

OAKS scores are at or above the 35th percentile

INTENSIFY READING INTERVENTIONS WHEN:

Progress monitoring indicates 4 data points below the aimline (maze).

Slope is flat or decreasing AND scores are below 50th percentile (maze).

Grade+ scores at or below 3rd stanine.

MAKE A PLAN

Select Measures Decide

Who will assess students? Who will record & graph the information? Who will make instructional decisions?

Get Training Establish

Decision rules Team Process Schedule for assessment

REFERENCES FOR MAZE

AIMSweb www.aimsweb.org

Easy CBMhttp://easycbm.com/

National Center on Student Progress http://www.studentprogress.org/

Intervention Central www.interventioncentral.org

David Putnam, Jr., Ph.D.dputnam@ttsd.k12.or.us

QUESTIONS?