Date post: | 17-Jan-2016 |
Category: |
Documents |
Upload: | adelia-shaw |
View: | 217 times |
Download: | 0 times |
New Hampshire’s 2007 AYP Status Reports
& Follow The Child Growth
Reports August 20, 2007
August 20, 2007 NH Department of Education 2
Table of Contents What is an accountability
system? 2007 AYP Status Results
p.4 release schedule p.5 2006/2007 Comparison p.6 Basic Calculation P.7 Gr. 3-8 NECAP Achievement Levels P.8 Taskforce Recommendations P.9 Minimum n definition P.10-11 Index Definition and example P.12-13 NECAP and NH-Alt conversions P.14-15 Starting Points and AMOs P16-18 Confidence Intervals and chart P.19 Safe Harbor and example P.20-22 Sample Report Shells P.23 Special Notes on Data P.24 Results and Resources P.25 Future work
NH Follow The Child Growth Reports P.26 History of the NH Growth Model P.27 Why did NH submit a proposal for a
growth model? P.28 What is the NH Follow The Child
Growth Model? P.29 NH Follow The Child Growth
Expectations P.30 Growth Groups by NECAP Scaled
Score P.31 Reading Growth Targets P.32-33 Sample Report Shells P.34 What has changed now that the
Growth Model is New Hampshire’s? P. 35 Why Use Two Models? P.36 So What Does All This Mean? P.37 Who created NH Model? P.38 Timeline for AYP and Growth Release P.39 Contact Information
August 20, 2007 NH Department of Education 3
What is an Accountability System? State Standards
- Curriculum Frameworks
- Standards for School Approval Valid & Reliable Assessment
System- state assessment
- local formative, benchmark & competency based assessments
Data Analysis Tools- Performance Pathways
Accountability Reporting - Assessment Reports
- NCLB AYP Status Reports
- NH Growth Reports
Statewide System of Support-School Improvement Coaches (content, data, special education, leadership, NH-Alt)- Leadership Institute- Literacy Action Plan- Numeracy Plan- Ongoing PD - DINI support- High School Vision Statement- High School Redesign- PD Master Plans- Special Education Focused Monitoring- Root Cause Analysis program
August 20, 2007 NH Department of Education 4
2007 AYP Status Results – release schedule(impacts school year 2007-2008)
AYP Definition Grades Testing Data with dates
Report Issued
School
District
State
Index based on NECAP and NH-Alt
(since 2006)
Elementary and Middle Grades
Gr. 3-8 NECAP (Oct 2006)
Gr. 2-7 NH-Alt (2005-2006)
Aug. 28
2007
10:00 AM
District % Basic or above based on NHEIAP
and NH-Alt (no information included In this ppt)
High School Gr. 10 NHEIAP (May 2006)
Gr. 10 NH-Alt (2005-2006)
Aug. 28
2007
10:00 AM
School
State
High School Graduation Rate Only
Aug. 28
2007
10:00 AM
August 20, 2007 NH Department of Education 5
2007 AYP Status Results – 2006/2007 Comparison
What’s the Same as in 2006 ?
Factors included (participation, other indicator, performance)
Index values (20, 40, 60, 80, 100)
AMOs (Annual Measurable Objective)
Report shell (mostly)
What’s Different?
Calculating Safe Harbor Cleaning up the
demographic data Report Shell (2%
calculation & overall determination)
August 20, 2007 NH Department of Education 6
•Calculate “Other Indicator” for the school (attendance rate at 3-8; graduation rate at high school)
•Calculate Participation rate for the school and each subgroup (based on testing year)
•Calculate Performance for the school and each subgroup (based on teaching year)
–Calculate Index (Compare to AMO target)
–If not OK, check confidence interval (99%)
–If still not OK, check safe harbor
–Check 2% in case of Special Education subgroup
2007 AYP Status Results – Basic Calculation
August 20, 2007 NH Department of Education 7
Grades 3-8 NECAPAchievement LevelsNECAP Cut points Commissioners from VT, RI, and NH adopted cut points on January 20, 2006 Proficiency at the student level includes performance at achievement levels 3 and 4.
Achievement Levels: Level 1: Substantially Below Proficient Level 2: Partially Proficient Level 3: Proficient Level 4: Proficient with Distinction
Scaled Scores will be reported as a 3-digit number where the first digit is the grade level and the other part will be a score 00 to 80 (reported as whole numbers at the student level). X40 is the reported cut point for Proficient.
Grade 3: 300-380Grade 4: 400-480Grade 5: 500-580Grade 6: 600-680Grade 7: 700-780Grade 8: 800-880
August 20, 2007 NH Department of Education 8
2007 AYP Status Results – Taskforce Recommendations
An External AYP Task Force met to make recommendations to the Department. After review by the Internal AYP Task Force, we have arrived at the following decisions: Racial categories (no change)
Defined by Beginning of Year (BOY) submissions Other Indicator (no change)
Attendance Rate (90% or improvement over previous year) Calculated from End of Year (EOY) submissions
Participation assigned to testing school Performance assigned to teaching school Full Academic Year (FAY) for performance calculations
Continuous enrollment from October 1 to the end of the teaching year
August 20, 2007 NH Department of Education 9
2007 AYP Status Results – Minimum “n” definition(Minimum number of students in a group required in order to perform calculation) 95% Participation rate: 40 for each group within
testing grades in school or district 75% Graduation rate: 40 within school or district 90% Attendance rate: 40 within school or district
Includes all grades 1-8 in the school or district Performance targets:11 for each group within
testing grades in school or district
August 20, 2007 NH Department of Education 10
2007 AYP Status Results - Index DefinitionIndex System provides partial credit for scores below
Proficient. A school’s index score will be the average of all student index points assigned to the school.
Proficiency Level Index Points
Level 1: Substantially below Proficient
X00 0
1a 20
1b 40
Level 2: Partially Proficient2a 60
2b 80
Level 3: Proficient
Level 4: Proficient with Distinction100
August 20, 2007 NH Department of Education 11
An Index Example: Two schools
Two schools, both with 180 students tested and 61.1% of their students scoring proficient or better in reading.
READING Maple Street CS Pine Street MS
Level points # of students
total # of students
total
X00 0 30 0 5 0
Level 1a 20 20 400 5 100
Level 1b 40 10 400 10 400
Level 2a 60 5 300 20 1200
Level 2b 80 5 400 30 2400
Level 3 100 60 6000 60 6000
Level 4 100 50 5000 50 5000
Sum 180 12500 180 15100
Index 69.4 83.9
August 20, 2007 NH Department of Education 12
Scaled Scores for each Index NECAP Reading
0 1a 1b 2a 2b 3 and 4
300 301 to 321 322 to 330 331 to 335 336 to 339 340 to 380
400 401 to 421 422 to 430 431 to 435 436 to 439 440 to 480
500 501 to 519 520 to 529 530 to 534 535 to 539 540 to 580
600 601 to 617 618 to 628 629 to 634 635 to 639 640 to 680
700 701 to 717 718 to 728 729 to 734 735 to 739 740 to 780
800 801 to 815 816 to 827 828 to 833 834 to 839 840 to 880
Mathematics
0 1a 1b 2a 2b 3 and 4
300 301 to 323 324 to 331 332 to 335 336 to 339 340 to 380
400 401 to 421 422 to 430 431 to 435 436 to 439 440 to 480
500 501 to 525 526 to 532 533 to 536 537 to 539 540 to 580
600 601 to 625 626 to 632 633 to 636 637 to 639 640 to 680
700 701 to 727 728 to 733 734 to 736 737 to 739 740 to 780
800 801 to 827 828 to 833 834 to 836 837 to 839 840 to 880
August 20, 2007 NH Department of Education 13
Raw Score to Performance IndexConversion table for NH-Alt
* The lowest score any scoreable portfolio can earn is 13 raw score points.
** A raw score of 0 is only possible if the portfolio submitted was judged to be unscoreable by two trained and independent scorers.
Portfolio Raw Score Points Earned
Proficiency Level Performance Index Level
Performance Index Points Assigned
47-52 Proficient with Distinction 4 100
38-46 Proficient 3 100
34-37 Partially Proficient 2b 80
29-33 Partially Proficient 2a 60
21-28 Substantially Below Proficient 1b 40
13*-20 Substantially Below Proficient 1a 20
0** Un-scoreable (SBP) 0 0
August 20, 2007 NH Department of Education 14
AYP Status Def: Starting PointsStarting Points: For each content area separately, a
baseline was created via the 20% method outlined in NCLB: Determine index for each school Rank schools by each index Identify at “20th percentile” school
“20th percentile” school: the school where 20% of the students in the whole list attend that school or a school with a lower index.
That school’s index is the starting point
Starting points: Reading 82, Mathematics 76
August 20, 2007 NH Department of Education 15
AYP Status Def: AMOsAnnual Measurable Objectives
2 year intervals, equally spaced to 100
Grades 3-8 Index
Reading Mathematics
Starting Point (2005-2006) 82 76
2006 – 2007 82 76
2007 – 2008 86 82
2008 – 2009 86 82
2009 – 2010 91 88
2010 – 2011 91 88
2011 – 2012 95 94
2012 – 2013 95 94
2013 – 2014 100 100
So, Pine St. MS met the AMO since their index (83.9) is greater than or equal to 82 in reading, but Maple St. CS did not.
August 20, 2007 NH Department of Education 16
AYP Status Def: Confidence Intervals
1. Calculate the within-school variance for each school.
2. Calculate the average within-school variance for the state.
3. Calculate the standard error for each group
4. Calculate the 99% confidence interval for each group
1)_(
var2
studentindex
schoolstudentschool n
indexmeanindex
schools
school
naverage var
var
groupnaverage
groupSE var
groupgroup SECI *333.2
Where:indexstudent = index score for the student
mean_indexschool = average index score for the school to which the student belongs
nindexstudent= the number of student index scores for the school of interest
nschools = the number of schools, and ngroup = number of students in the group
For each subject separately:
For NHReading: 603.38615Math: 800.72625
The smallest index allowed with the CI is: AMO – CIgroup
August 20, 2007 NH Department of Education 17
Same as last year since AMOs are the same.
August 20, 2007 NH Department of Education 18
Maple St. CS still did not meet AMO with the confidence interval since their index (69.4) is not greater than or equal to 77.7
August 20, 2007 NH Department of Education 19
AYP Status Def: Safe Harbor (with an example)Even though a group does not make performance expectations, it
may have improved enough to be okay. To make Safe Harbor a group must meet the 10% rule and the additional indicator. The 10% rule requires that the “complement” of the group’s Index (100-Index) be at least 10% lower than the previous year. In addition, the percent of students scoring proficient or better must increase.
October 2005 NECAP &
2004-05 NH-Alt
October 2006 NECAP &
2005-06 NH-Alt
Reduction SH Goal
(’05 100-Index) x (.10)
Safe Harbor?
Is reduction
> or =
(SH Goal)?
Index 100-Index Index 100-Index
Maple Street CS
61 39 69.4 30.6 39-30.6 = 8.4
39 x .10
= 3.9
Is 8.4>3.9
Yes
Gould School
61 39 63 37 39-37 = 2 39 x .10
= 3.9
Is 2>3.9
NoMaple St. CS, however, improved enough to meet performance expectations by meeting safe harbor requirements.
August 20, 2007 NH Department of Education 20
August 20, 2007 NH Department of Education 21
August 20, 2007 NH Department of Education 22
August 20, 2007 NH Department of Education 23
2007AYP Status Results - Special Notes on Data
Reports are based on student demographic and program participation data reported by districts EOY (End of Year) files BOY (Beginning of Year) files SPEDIS/NHSEIS, ESL, F&R systems
Assessment reports were released in January 2007 and districts/schools had about a month to review and report and correct discrepancies in student demographic data.
August 20, 2007 NH Department of Education 24
2007 AYP Status Results and Resources www.ed.state.nh.us/education/ayp
AYP Status Reports at grades 3-8 AYP Status support material CSV file of all results New Hampshire’s AYP Status Reports
& Follow The Child Growth Reports (ppt) Separate HS District Reports (old AYP def) Separate HS School Reports (graduation rate
only)
August 20, 2007 NH Department of Education 25
AYP Status – Future work
Out of District Students in Private Placements How to include students placed in out-of-district
private placements?
Writing and Science Neither Writing nor Science is scheduled to be a
part of AYP at this time
August 20, 2007 NH Department of Education 26
History of the NH Growth Model Growth Model being considered by Accountability Task Force Lyonel Tracy becomes NH Commissioner Follow The Child Initiative Begins Spellings invitation to states to submit growth model proposals NH submits Growth proposal in February 2006 USED defers NH proposal until two years of assessment data NH revises and resubmits in November 2006 USED asks for clarifications in December 2006 NH submits revisions in January 2007 Peer Review #1 in March 2007 Peers ask for compilation of all revisions and clarifications (Task Force
reviews and refines proposal) Peer Review #2 in May 2007 May 22 – conference call informing us that the peers voted to not approve
our model – but, we could come to DC in June to work toward approval June 4, 2007 – AYP Task Force advises Commissioner to run the NH
Growth Model as our own pilot this year
August 20, 2007 NH Department of Education 27
Why did NH submit a proposal for a growth model?
The Accountability Task Force had been considering growth for a while and the idea reflected the goals of Follow The Child
To build on the NH philosophy of continuous improvement and longitudinal student growth
To allow us to determine individual growth targets for students that accelerate progress & close the achievement gap
To include individual targets as part of a larger system that includes interim testing, personalization, and remediation, if necessary
August 20, 2007 NH Department of Education 28
What is the NH Follow The Child Growth Model? (www.ed.state.nh.us/education/FTC/growth_model.pdf) We calculate growth targets for EVERY student, and tally the number of students meeting or exceeding their target
The targets are based on the previous year’s NECAP score – and if no NECAP score is available, the target is proficiency
Targets are based on the distance to proficiency – closing the gap (as measured by the number of standard deviations below proficiency) This ensures comparability between grades
If students are already proficient, the target is designed to encourage a level that exceeds proficiency
August 20, 2007 NH Department of Education 29
NH Follow The Child Growth Expectations
>1 SD below
½ to 1 SD below
0 to ½ SD below
0 to ¼ SD above
¼ to 1¼ SD above
> 1 ¼ SD above
Narrow gap to prof. by one third of the # of SD below
Narrow gap to prof. by one half of the # of SD below
Proficiency
Drop by no more than ¼ SD
Stay at least 1 SD above prof.
Previous NECAP scaled score.
Targets for next testing cycle.
August 20, 2007 NH Department of Education 30
Growth Groups by NECAP Scaled Score
Group 1 2 3 4 5
Math >1SDbelow
1SD – ½SDBelow
Proficient-½SD to Proficient + ¼SD
¼SD – 1¼SDabove
>1¼SDabove
Grade 3 300-329 330-334 335-342 343-353 354-380
Grade 4 400-428 429-434 435-442 443-454 455-480
Grade 5 500-528 529-534 535-542 543-554 555-580
Grade 6 600-628 629-634 635-642 643-654 655-680
Grade 7 700-729 730-734 735-742 743-753 754-780
August 20, 2007 NH Department of Education 31
Reading Growth Targets
Grades 3 to 4 Grades 4 to 5 Grades 5 to 6 Grades 6 to 7 Grades 7 to 8
Score Target Score Target Score Target Score Target Score Target
300 416 400 513 500 612 600 713 700 814
301 416 401 514 501 612 601 714 701 814
302 417 402 514 502 613 602 714 702 815
303 417 403 515 503 614 603 715 703 816
304 418 404 516 504 614 604 716 704 816
305 419 405 516 505 615 605 716 705 817
306 419 406 517 506 616 606 717 706 818
307 420 407 518 507 617 607 718 707 818
308 420 408 518 508 617 608 718 708 819
309 421 409 519 509 618 609 719 709 820
310 422 410 520 510 619 610 720 710 820
Complete reading and mathematics growth targets charts available at:www.ed.state.nh.us\education\FTC\growth\reading_growth_targets.pdf www.ed.state.nh.us\education\FTC\growth\math_growth_targets.pdf
August 20, 2007 NH Department of Education 32
Confidential list of students with scores from 2005 and 2006, and targets for 2006 and 2007.
August 20, 2007 NH Department of Education 33
Public report. Growth targets have been set at the school level (listed next to each content area heading). The school and each group may meet the school growth targets directly or with the use of confidence interval.
August 20, 2007 NH Department of Education 34
What has changed now that the Growth Model is New Hampshire’s? Confidence Interval based on growth targets
and group size (99%) to identify groups not reaching growth targets with more confidence
AMO Targets for future years to be determined after reviewing pilot results
Can we/do we include growth reports for high school?
August 20, 2007 NH Department of Education 35
Why Use Two Models?
The FTC growth model encourages schools to focus on all students, not just the students that scored just below the performance level cuts.
In contrast, the status/index model rewards schools for improvement that crosses achievement levels, regardless of the amount of growth.
New Hampshire feels that a valid accountability system should incorporate both status and growth and public reporting of other assessments throughout the year.
August 20, 2007 NH Department of Education 36
So What Does All This Mean?
NCLB Accountability is based on the Index The FTC Growth Report is a NH Pilot The school growth reports are for
informational purposes to inform teachers, administrators, and the public
The student roster information is for teacher instructional use and student goal setting
Growth targets give us “one more picture” of student performance
August 20, 2007 NH Department of Education 37
Who created NH Model?
The New Hampshire Accountability Task Force made up of: District and school personnel NHDOE staff Representatives from university system,
school boards, parents Technical advisors from National Center for
the Improvement of Educational Assessment (NCIEA) and Measured Progress
August 20, 2007 NH Department of Education 38
Timeline for AYP and Growth Release
Tuesday, August 28 at 10:00 AM – public release of index reports and school, district, & state growth reports
2007-08 school year: review & work with districts on growth targets and growth reports
Use growth results to determine AMOs and future use in NH Accountability System
August 20, 2007 NH Department of Education 39
Contact Information
Deb Wiswell (Accountability)
603-271-3828
Tim Kurtz (Curriculum and Assessment)
603-271-3846
Gary Guzouskas (School Improvement and Appeals)
603-271-5873