Making Use of DIBELS dataIn Effective New Mexico Reading

First schools

Education Development Center:

Naomi Hupert & Lauren Bates

New Mexico Public Education Department,

Director of Reading First, Dr. Lana Paolillio

New Mexico Reading FirstThis presentation draws on findings from the 3.5 year evaluation of Reading First schools in New Mexico. During the 2006-07 school year researchers collected qualitative data from schools demonstrating positive impact on students as measured by the DIBELS.

Districts Visited

Schools Visited

Observations conducted during . . .

School Staff Interviewed

11 22 38 75

Reading Block Administrative Staff * Teachers

27 49 26

3 Kindergarten 8 District Coordinators

8 1st grade 17 Principals

8 2nd grade 26 Reading Coaches, Interventionists

4 3rd grade 2 Other

4 Mixed grades

Interventions

11 Conducted by Reading Coach, Teacher or Interventionist

Site visits between September 2006 and January, 2007.

NM DIBELS performance

All students at Benchmark from 03 - 07

0

10

20

30

40

50

60

70

80

Year1a

Year 1c Year2a

Year 2c Year3a

Year 3c Year4a

Year 4c

BOY and EOY for 03 - 07

Percentage at Benchmark

Kinder

First

Second

Third

(17,233 students assessed in September of 2007)

NM DIBELS performance

All students at Intensive from 03-07

0

5

10

15

20

25

30

35

40

Year1a

Year1c

Year2a

Year2c

Year3a

Year3c

Year4a

Year4c

BOY and EOY for 03 - 07

Percentage at Intensive

Kinder

First

Second

Third

(17,233 students assessed in September of 2007)

School level • State level

• What is happening at the school level - are schools showing greater growth doing something different than those showing less growth?– Schools making use of data in novel ways– Administrators, coaches, interventionists engaged in work with assessment data

• At the state level - are there recommendations the state could be making based on state-wide trends?– Using data to address challenges

Our Criteria

• 70% or more of students attain Benchmark support status in April, 2006

• 70% or more of third grade students reach Benchmark status in April, 2006

• Students at Benchmark status increase by 20% or more between September 2005 and April 2006

• Students at Intensive status decrease by 15% or more between September 2005 and April 2006

• 90% or more of students at Benchmark status in September 2005 maintain their Benchmark status in April 2006

Effective Schools Found

Three NMRF schools met all five criteria:• Alcalde Elementary (Española)• Rio Costilla Elementary (Questa)• Gallina Elementary (Jemez Mountain)These schools had several traits in common:

• Rural settings• Small student populations• Small class sizes

Alcalde ElementaryEspañola Public Schools

• District geography: The town of Española (~10,000 residents) and nearby smaller towns, including some within Native American reservations.

• School K-3 Student population: 91• Student:teacher ratio: 15.4 students per teacher (district wide)

• The main school building has been condemned and all classrooms and offices are now portables.

Rio Costilla ElementaryQuesta Independent Schools

• District geography: The town of Questa (~2,000 residents) and surrounding villages.

• School K-3 Student population: 26• Student:teacher ratio: 12.7 students per teacher (district wide)

• In this small district, the district RF coordinator doubles as a reading coach in one of the schools. Teachers are residents of NM and of nearby Colorado.

Gallina ElementaryJemez Mountain Public Schools

• District geography: The village of Gallina (~500 residents) and other nearby villages.

• School K-3 Student population: 27• Student:teacher ratio: 10.3 students per teacher (district wide)

• Although none of the district’s schools are on reservations, Jemez Mountain has one of the state’s highest percentages of Native American students.

Another Effective SchoolSince NM also has large, urban schools, we decided to find the state’s most effective large school. Lavaland Elementary (Albuquerque Public Schools) met three of our criteria:

• Students at Benchmark status increased by 20% or more between September 2005 and April 2006

• Students at Intensive status decreased by 15% or more between September 2005 and April 2006

• 90% or more of students at Benchmark status in September 2005 maintained their Benchmark status in April 2006

Lavaland ElementaryAlbuquerque Public Schools

• District geography: The largest city in NM (~500,000 residents).

• School K-3 Student population: 431• Student:teacher ratio: 14.6 students per teacher (district wide)

• This is the second largest RF school in the state. 2005/2006 was the principal’s second year and the reading coach’s first.

Range of common themes among schools

Commonalities between the sites revealed a sophisticated implementation of the program, as well as several other unexpected factors:– All adopted their core programs at least one year prior to

joining Reading First;– All had Spanish bilingual programs for the entire student

body;– All engaged in district-wide communication between reading

coaches and coordinators;– All shared data not only with teachers and administrators, but

also with students and parents;– All exhibited constant analysis and adjustment regardless of

previous successes;– All used the “walk to intervention” model, during which at-

risk students met with a reading coach or interventionist in ability level groups;

– All had close relationships with their PED regional specialists.

District-Wide Communication

• In Albuquerque, the district coordinator and reading coaches from all RF schools met twice every month. Principals also attended the meeting on a monthly basis.

• At the Questa district, the district coordinator is also a reading coach based out of one of the two RF schools. The coordinator/coach traveled between the schools every week, teaching at each site and meeting with the other reading coach and interventionist.

• Española’s coordinator and reading coaches met monthly to present ideas and address RF issues.

• In Jemez Mountain, the reading coach, district coordinator, regional specialist, and entire K-3 staff met monthly at Gallina Elementary to discuss data and individual student progress.

Sharing data with students and parents

• At all 4 schools DIBELS assessment results are posted either in a public area or in the reading resource room.

• Teachers at these schools keep folders or binders of their class’ DIBELS data, which includes DIBELS data and “academic profiles” of students.

• Reading coach, principal and interventionist have accountability binders that house data, testing schedules, and other relevant assessment information.

• Students at all four schools have a degree of ownership over their own data. Whether they literally move a data point on the school’s DIBELS display (Rio Costilla), track their scores in data folders (Alcalde and Lavaland) or complete graphs of their data (Alcalde, Lavaland and Gallina), they become familiar with their own results. All interviewees stated that student familiarity with data is important and helpful. One interventionist said that the data has “impacted the students because for the first time, I’m seeing kids who are really involved in their learning because they’re part of the process. They’re understanding that your goal is to read, to read fluently and to understand the reading.” She also speculated that data use has “caused a level of involvement and investment among teachers and students I’ve never seen before.”

Students and Parents (continued)

• Parents are included in discussion about assessment data.

• Teachers in all schools discuss DIBELS scores during parent/teacher conferences. Parents at Rio Costilla also receive a demonstration of the DIBELS test on a handheld. At both Alcalde and Gallina, they send home DIBELS data reports for all students. At Alcalde, they also give a presentation on the DIBELS at the school’s Open House held during the beginning of the year. One reading coach shared, “The whole language at our school, the parents understand it. If we say progress monitoring, they know what that is. Our community language is different.” According to one teacher, keeping parents aware of student progress assists him to “get help from home” for struggling readers.

Analysis and adjustmentData collection and presentation, though important, are but the first steps in implementing data-driven instruction. At all of the schools visited, the staff discussed and modeled a committed to analyzing data, reflecting on the results, and adjusting instructional practice accordingly. – Alcalde implemented the Walk to Read model in the second

grade and administrators added a replacement core program and increased intervention times,

– Lavaland tested a Kindergarten-Plus class and extended the reading block to 120 minutes for second and third grades. An administrator at explained that the school plans to “push the envelope a little more. We are looking at grouping and how we can do appropriate activities based on data.”

– Gallina’s reading coach credited Reading First’s impact with the elimination of SPED referrals this year, but noted that she is still adjusting intervention groups to foster student success.

A Final Interesting Case: Lake Arthur

Lake Arthur Elementary has staged one of the most dramatic turnarounds in NM:– At the end of its first year of Reading First implementation (2004-05), Lake Arthur was identified as one of the poorest performing schools in the state.

– In April 2005, the school/district was one of three districts in the state that documented an increased the percentage of students needing Intensive support, and a decrease the percentage of students at Benchmark.

– By January 2006, Lake Arthur was one of seven districts where the percentage of students at Benchmark increased by more than 20% during one semester. It was also one of five districts where the percentage of Intensive students decreased by more than 15%.

Lake Arthur Data

By April 2006, Lake Arthur was one of the highest performing schools in the state:

• 70% or more of third graders at Benchmark• 20% or more increase in the percentage of students at Benchmark during 2005/2006

• 15% or more decrease in the percentage of students at Intensive during 2005/2006

• Moved at least 75% of students out of Strategic (100% of those went to Benchmark)

• 90% or more of students at Benchmark in September remained at Benchmark in April

Frequent Data Meetings

Lake Arthur used data in a few unique ways:

• The school has a Literacy Instructional Strategy Team meeting every day, including teachers K-5, during which the team discussed data and strategies.

• The school also held in-depth data meetings three times a year.

“Backwards Testing”

• The reading coach did “backwards testing” on the DIBELS and TPRI: “If [students] show up Strategic in third grade, I’ll assess them at the second grade level until I find where they are showing a Benchmark performance.” This served to “determine our initial concentration for Intensive kids, then on kids who aren’t progressing to aim line, or are progressing sideways.”

Integrating Several Data Sources

• In addition to DIBELS and TPRI data, the reading coach used data from a phonics inventory he developed to determine the specific areas of need for at-risk students.

• At-risk students must “demonstrate proficiency a number of times and in a number of ways” before moving to new skills or intervention groups.

Student Directed Conferences

• Students led parent/teacher conferences and had to explain their DIBELS results to their parents.

• Students rated themselves on dimensions such as “I complete my assignments every day” and “I attend school regularly.”

• The student, parents, and teacher complete a “Learning Expectation Form.” The student and the parents write their plans for the child’s success, the teacher responds to them, and then adds her own strategies for helping the student.

District Level Trends

What the data show: a 1st grade “drop-off”Many students who are identified at Benchmark in September of 1st grade are identified as needing either Intensive or Strategic support by January of 1st grade.

What we have found: fragile literacy skillsIf a student enters Kindergarten reading below Benchmark, then makes great gains over the course of the year and is identified as Benchmark in September of 1st grade, he or she has a very high chance of performing below Benchmark by January of 1st grade.

% at Benchmark in Sept of Kindergarten

% at Strategic in Sept of Kindergarten

% at Intensive in Sept of Kindergarten

First graders at Benchmark in Sept and Jan

42.6

38.8

18.6

First graders at Benchmark in Sept but not Jan

13.5

53.7

32.8

• This suggests that students entering Kindergarten below Benchmark are receiving a high level of intervention, and many of them are moving quickly to Benchmark.

• However, these students appear unable to sustain their performance at this support recommendation.

• We suggest that these students have “fragile” reading skills. They have a command of the skills necessary to be at grade level at the beginning of first grade, but because they have acquired these skills over a short time they have not had adequate time to practice and consolidate them and make use of them in the context of reading a passage, as required by the ORF test.

Implications

Implications (continued)

What does this mean for schools?• We suggest that ongoing intervention be provided to Kindergarten students who entered K below Benchmark, even if they have achieved Benchmark during the year.

• We suggest that first grade students who start the year at Benchmark, but who arrived in K below Benchmark, be provided with intervention instruction between September and January that focuses on using discrete skills in the context of reading text, and have multiple opportunities for reading and practicing oral reading fluency.

Conclusions• Schools where students are demonstrating positive growth in reading, as measured by the DIBELS, employ a common set of expectations towards the role that assessment data will and should play in the planning and provision of instruction. These include sharing assessment information widely, inviting all members of the education community to view and interpret data (parents, teachers, students, administrators), and anticipate that changes will be made as a result of findings from data.

• Similar expectations at the state level can help to foster this approach to the use of assessment data across all schools.

Ongoing questions we would love to explore and discuss:

• Why are smaller schools more able to demonstrate positive change?

• Why are single grade schools more able to do so?

• Are these trends evident in other states, and/or other programs?

• Does who administers the DIBELS make a difference in teachers’ use of the data?

• Why do schools with universal dual language programs appear to be doing so well?

Thank you

For more information or questions about this

presentation, please contact:

Naomi Hupert

Nhupert@edc.org

www.edc.org/CCT/