accountability 2013
Interpreting Your 2013 Accountability ReportIt’s Like Learning To Read All Over Again
Ervin [email protected]
John [email protected]
TEPSA Webinar | September 5, 2013
Welcome!
goals2013 accountability
Construction
Clarity Of calculation . . . and application
Connection
Of indexes What measures/data are included?
To instructional processes and systems
TEPSA Webinars (May 10 and 17)
Today
Ongoing
the vibe
YourQuestion
s
5
accountability
2013the BIG picture
p. 21
p. 22
Accountability | Overall Design
• Four Performance Indexes
• Designed to meet 4 statutory goals1. Improving student achievement at all levels in core subjects
2. Ensuring progress of all students toward Advanced Academic Performance (STAAR Level III)
3. Closing Performance Gaps among groups in Advanced Academic Performance (STAAR Level III)
4. Closing gaps among groups in % of students graduating under Recommended or Advanced high school programs
Index 1
Index 2
Index 3
Index 4
p. 22
2013 AccountabilityAchievement
All Students Only Single percentage
combined over ALL Subject Areas
≥ Level II (Phase 1) STAAR
≥ Met Std Grade 11 TAKS
Progress• 10 student groups
– All Students– 7 Race/Ethnicity– SpEd– ELL
• Minimum size– 10 for All Students– 25 for other groups
• By Subject Area– Rdg & Math in 2013*
• Based on PROGRESS– 1 point for % Met– 2 points for %
Exceeded
Closing Gaps• 3 student groups
– Eco Dis and – 2 lowest
Race/Ethnicity groups from prior year
• By Subject Area (all subjects)
• 1 point for % Level II (Phase 1)
• Level III to be added (2014)
Postsecondary• 4-year or 5-year
Graduation Rate (or Annual Dropout Rate)– 10 student groups
• % RHSP/DAP– 8 student groups
(not SpEd or ELL)
• STAAR Postsecondary Readiness – Level II Final (2014)
Index 1Student
Achievement
Index 2Student Progress
Index 3Closing
Performance Gaps
Index 4Postsecondary
Readiness
Target = 50 Target = ≈ 5th %ile
Target = 55Target = 75
Target =30 29 17 21
ES MS HS Dist
p. 21
accountability
2013Interpreting our ReportsIt’s like learning to read all over again!
2013 accountability . . .
Part Section # Pages1 Accountability Summary 12 Index Calculations and Data Tables 123 Distinction Designations (Campuses Only) 4 (8/29/13)
4 System Safeguards 3 (8/29/13)
Total Pages 20 (8/29/13)
. . . a report in 4 parts
p. 1p. 2-13 p. 14-
17p. 18-
20
Index 1
Index 1Student
Achievement
Index 1Student
Achievement
p. 2
p. 3-4
Index 1: Student AchievementIndex 1Student
Achievement
p. 22
10 Additional Student Groups reported 7 Race/
Ethnicity SpEd Eco Dis ELL
Used for System Safeguards
p. 2
• Calculate Overall Pass Rate For All Students1. Add # Passed (≥ Phase-in 1 Level II) each subject area (R + M + W + S + SS)
• 277 + 284 + 65 + 60 + 34 = 720
2. Add # Tests taken each subject area (R + M + W + S + SS)• 485 + 473 + 174 + 127 + 127 = 1,386
3. Divide # Passed by # Tests to determine % Passed• 720 ÷ 1,386 = 51.9%, which ROUNDS to 52%
4. Assign 1 point for each % Passed = Index 1 Score• 52% = 52 POINTS
• Overall pass rate IS NOT average pass rate– Average pass rate would equal: (57% + 60% + 37% + 47% + 27%) ÷ 5 = 47%
Index 1 Score57% 60% 37% 47% 27%
p. 3-4
p. 2
• Where did these numbers come from?• Pages 3-4!
% at Final Level II (13%)% at Level III (3%)
[Instructional Planning]
Disaggregated Data There are a LOT of numbers on pages 3 and 4!
“I thought Index 1 ONLY evaluated the All Students group!”
True!
BUT . . . the data are included for System Safeguards
p. 3
• System Safeguard– Low Performance in a single cell for a particular student group does NOT
automatically cause the district/campus to miss Index 1– However, ANY student group meeting MSC with a pass rate below 50% MUST
be addressed in the district/campus improvement plan• MSC = 0 for All Students | 25 for all other Student Groups
• So how do we “find” System Safeguards?1. In each subject area, determine which student groups meet MSC
• MSC = 0 for ALL Students | 25 for all other student groups
2. Then determine if % Passing < 50% (at Phase-in Level II or above) for any group meeting MSC
3. If % Passing < 50% AND Student Group meets MSC, then Student Group’s Performance MUST be addressed in improvement plan
p. 3
Others?
# GroupsMissing Safeguard
8
# GroupsMeeting Safeguard
9
p. 3
# GroupsMeeting MSC
Reading, Math, Writing 17
p. 4
Science?Social Studies?
# GroupsMeeting MSC
Reading, Math, Writing 17
# GroupsMissing Safeguard
8
# GroupsMeeting Safeguard
9Science, Social Studies 8 8 0
TOTAL 25 16 9
p. 3
v
p. 4
System Safeguards• 25 student groups met
MSC (across 5 subject areas)
• ONLY 9 Met Safeguard target of 50%
• 16 Missed Safeguard target (including EVERY group in Writing, Science, Social Studies)
• BUT campus MET Index 1 Target!
All 16 Student Groups that Missed Safeguard target of 50% must be
addressed in CIP
System Safeguards• Are summarized with Y/N indicators in
Part 4 of the Campus’ 2013 Accountability Report (p. 18) p.
18
p. 3
p. 19
System Safeguards• Performance Rate data from p. 3-4
are duplicated on p. 19 (slightly reorganized)
p. 19
System Safeguards• Performance Rate data from p. 3-4
are duplicated on p. 19 (slightly reorganized)
• Participation Rate data are also provided
Index 1is summarized in
the campus’ 2013 Accountability
Summary(p. 1)
2013
Acc
ount
abilit
y Su
mm
ary
p. 1
27
Your Question
s
Index 1Student Groups?
MSC?Safeguards?
Index 2
Index 2Student Progress
Index 2Student Progress
p. 5
p. 6-7
Index 2: Student Progress Index 2Student Progress
p. 22
Index 2: Student Progress
Based on actual student progress (2012 to 2013)
Index 2Student Progress
STAAR Level IIIfocuses on progress toward . . .
Each student is assigned a progress category based on his/her change in scale score in relation to progress expectations
–Did Not Meet
–Met
– Exceeded
Index 2: Student ProgressIndex 2Student Progress
Index 2: Student Progress
• Subject areas evaluated for 2013– Reading• Grade 4 through English II
– Math • Grade 4 through Algebra I
– Writing • English I to English II
Index 2Student Progress
Performance Standard Met Progress Exceeded Progress
Index 2 is based on a fraction where Numerator = # of points based on % of students who Met or Exceeded Progress
1 point for % Met 2 points for % Exceeded
Denominator = Maximum Points Possible based on number of student groups meeting minimum size criteria (MSC) Maximum Points Possible = 200 points for each student group meeting MSC
Index 2: Student ProgressIndex 2Student Progress
What is “progress”?
Index 2: Student ProgressIndex 2Student Progress
• With exceptions for high-performing (“almost perfect”) and low-performing (“lower than guessing”) students, the general rule is
MetProgress = Level II Final
(Higher Grade) – Level II Final(Lower Grade)
ExceededProgress = Level III
(Higher Grade) – Level II Final(Lower Grade)
p. 23
Subjectand
GradeLevel/Course
ItemsPoints
Level II Standard
(FinalRecom-
mended)
Level III Standard
Grade 4 (Eng. & Span.) 48 1599 1677
Grade 5 (Eng. & Span.) 50 1627 1710
Met Progress?Level II 5th Grade – Level II 4th Grade1627 – 1599 = 28
p. 23
Exceeded Progress?
Level III 5th Grade – Level II 4th Grade1710 – 1599 = 111
p. 23
Subjectand
GradeLevel/Course
ItemsPoints
Level II Standard
(FinalRecom-
mended)
Level III Standard
Grade 4 (Eng. & Span.) 48 1599 1677
Grade 5 (Eng. & Span.) 50 1627 1710
p. 23
Index 2Simple
Example
All Students
# Tests 500
# Met 300
# Exceeded 100
% Met 60%
% Exceeded 20%
1 point for % Met OR Exceeded 80
1 point for % Exceeded 20
Weighted Progress Points 100
Maximum Possible Points 200
Index 2 Score(Weighted Progress Points ÷ Maximum Possible Points)
50
1 point for % Met 60
2 points for % Exceeded 40
Weighted Progress Points 100
Index 2 Data Reports
Build “back to front”
Start with page 7
Then page 6
Then page 5
p. 7
For each student group meeting MSC in each subject, determine: % Met or Exceeded Progress % Exceeded Progress
From Index 1 (p. 2)
REMINDERS (Why Index 2 numbers will not match Index 1): Only students with 2 years of “matched” data are included Progress starts at 4th grade (no 3rd graders receive a Progress Measure) ELL students are “mostly excluded” from Index 2 in 2013 STAAR Modified and STAAR Alternate are not included in Index 2 in 2013 Writing is ONLY evaluated for English I to English II
MSC = 10 tests for All Students | 25 tests for other Student Groups
p. 6
Data from page 7 are reproduced on page 6 In each subject area, for each Student Group meeting MSC
Assign 1 point for each % Met OR Exceeded Assign another 1 point for each % Exceeded
Total = Weighted Progress Rate for each Student Group meeting MSC Effectively 1 point for each % Met and 2 points for each % Exceeded
Add Weighted Progress Rate in each subject for each student group meeting MSC Maximum Points = # Student Groups meeting MSC x 200
+ +
+ +
p. 5
Data from page 6 are summarized on page 5 (remember: read back to front!) Weighted Progress Rate for each Student Group meeting MSC in each subject
area is reported and summed Reading: 75 + 64 + 77 = 216 points Math: 70 + 67 + 72 = 209 points
Sum the Total Points in Reading and the Total Points in Math 216 + 209 = 425
Sum the Maximum Points in Reading and the Maximum Points in Math 600 + 600 = 1200
425 ÷ 1200 = 35.4, which rounds to 35 . . . SO Index 2 Score = 35 points
Index 2 Score
+ ++ + + +
Index 2is summarized in
the campus’ 2013 Accountability
Summary(p. 1)
2013
Acc
ount
abilit
y Su
mm
ary
p. 1
47
Your Question
s
Index 2Progress Categories
PointsData
Index 3
Index 3Closing
PerformanceGaps
p. 8
p. 9-11
Index 3: Closing Performance Gaps
Index 3Closing
Performance Gaps
Impacts every student group of which the ELL student is a member
p. 24
Index 3: Closing Performance Gaps
Index 3 emphasizes Level III (Advanced Academic Performance) for “Performance Gap Groups”
BUT . . . STAAR Level III cannot be included in accountability until 2014, so for 2013 . . .
Measure = % of students in Performance Gap Groups “passing” state assessment Evaluated in EACH subject area (Reading, Math, Writing, Science, Social
Studies) Then cumulated to a single score
Index 3Closing
Performance Gaps
Index 3: Closing Performance Gaps
Index 3: Closing Performance Gaps?
Index 3Closing
Performance Gaps
All
Eco Dis
Eco Dis
% Achieving Level III
Index 3: Closing Performance Gaps
Student Groups Identified for Evaluation in 2013
Economically Disadvantaged (no MSC)
2 lowest performing Race/Ethnicity groups meeting MSC (≥ 25 tests) across all subjects from Index 1 in 2012 The 2 Race/Ethnicity Groups are determined based on combined
performance across all subjects (Index 1) So the Race/Ethnicity Groups analyzed in Index 3 are the same in all subjects For Index 3 evaluation, MSC for each Race/Ethnicity Group in each subject
area = 25 tests taken by students in that Race/Ethnicity Group in that subject area
If a Race/Ethnicity Group does not meet MSC in a subject area, then it is NOT evaluated in that subject area
Index 3Closing
Performance Gaps
p. 8
• Summary Data – page 8• How were the 2 Race/Ethnicity groups
selected?• From 2012 performance data
p. 10
• Summary Chart of 2012 Performance for the 7 Race/Ethnicity Groups• Which Race/Ethnicity groups met MSC in 2012 (≥ 25 tests taken by students in that
Student Group across all subject areas)?– African American, Hispanic, White
• If ≥ 3 Race/Ethnicity groups meet MSC, then 2 lowest performing are used for Index 3– African American (44%) and Hispanic (51%)
• If only 2 Race/Ethnicity groups meet MSC, then only 1 Race/Ethnicity group is used for Index 3 (the lower performing group)
• If only 1 (or 0) Race/Ethnicity groups meet MSC, then Index 3 is based solely on Economically Disadvantaged
Index 3 Data Reports
Build “back to front”
Start with pages 10 and 11
Then pages 8 and 9
Then top of page 8
p. 10
For each subject and each student group, build from the bottom row up Total Tests # at Level II Phase 1 or above % at Level II Phase 1 or above
p. 11
Repeat for Science and Social Studies Total Tests # at Level II Phase 1 or above % at Level II Phase 1 or above
Notice that there are fewer than 25 tests taken by African American students in Science and Social Studies
A Student Group is NOT EVALUATED in a subject area UNLESS it meets MSC IN THE SUBJECT AREA (≥ 25 tests taken by students in that student group in that subject area)
p. 8
These data come from pages 10-11
p. 10
p. 8
In each subject area, for each Student Group meeting MSC: Assign 1 point for each % at Phase-in 1 Level II or above Then total Weighted Points Maximum Points = 100 x Number of Groups meeting MSC
+ +
+ +
p. 9
Determine Weighted Performance Rate for Writing, Science, Social Studies <25 tests taken by Af. Am. students in Sci and SS Af. Am. student group is NOT evaluated for Index 3 (Max. Pts. = 200)
+ +
+ +
+ +
p. 8
• Index 3 Summarized Data – top of page 8• Total Weighted Performance for each Student Group meeting MSC in each
subject area (from bottom of page 8 and page 9)• Total Points per subject and Maximum Points per subject• Summed across all subjects• Total points (750) ÷ Maximum Points (1300) = 57.6, which rounds to 58• Index 3 Score = 58 points
Index 3 Score
p. 2
What will be the Race/Ethnicity groups in Index 3 for this campus in 2014? Look at Index 1 data table for 2013 3 Race/Ethnicity groups meet MSC (≥ 25 tests taken by students in that group) 2 lowest performing are: African American and Hispanic In 2014, Index 3 will be weighted:
1 Point for each % at Level II 2 points for each % at Level III
Index 3is summarized in
the campus’ 2013 Accountability
Summary(p. 1)
2013
Acc
ount
abilit
y Su
mm
ary
p. 1
65
Your Question
s
Index 3Closing Gaps
Future – % at Level III
Index 4
Index 4Postsecondary
ReadinessIndex 4
Postsecondary Readiness
p. 12
p. 13
Index 4Postsecondary
Readiness
Index 4: Postsecondary Readiness
For 2013 Accountability, Index 4 is based on Graduation Rate (Cohort Measure)
4-year rate for Class of 2012 Cohort (entered 9th grade in 2008-09), OR 5-year rate for Class of 2011 Cohort (entered 9th grade in 2007-08)
RHSP/DAP Diploma Rate (Annual Measure) %age of students who were awarded diplomas in 2011-12 school year
REGARDLESS of when they entered 9th grade
For high schools without a Graduation Rate, the Grade 9-12 Annual Dropout Rate is used
Coming Soon to Index 4 . . .
Postsecondary Readiness in 2014 . . .
Scheduled to include for all campuses % of students passing one or more STAAR
assessments at Level II – Final
Index 4is summarized in
the campus’ 2013 Accountability
Summary(p. 1)
2013
Acc
ount
abilit
y Su
mm
ary
p. 1
71
Your Question
s
Index 4Graduation RateRHSP/DAP Rate
2013 AccountabilityAchievement
52points
Progress
35points
Closing Gaps
58points
Postsecondary
N/A
Index 1Student
Achievement
Index 2Student Progress
Index 3Closing
Performance Gaps
Index 4Postsecondary
Readiness
Target = 50
Target = 55Target = 75
Our Sample Campus
Target =30 29 17 21
ES MS HS DistMet Standard
Campus Distinction
Designations
Distinction Designations Campuses that earn Met Standard are eligible
for 3 campus distinction designations:
p. 14-17
1. Top 25% Student Progress
2. Academic Achievement Distinction Designation (AADD) in Reading/ELA
3. AADD in Math
Campus Distinction Designation
p. 16
Campus must rank in top quartile (top 10 scores) of Campus Comparison Group
Top 25% Student Progress
Determining Academic Achievement Distinction Designations (AADDs)
Academic Achievement Distinction Designations (AADDs)
AADDs recognize outstanding campus achievement in ELA and Math
Each campus is compared to its campus comparison group on a number of indicators in ELA and in Math that are specific to its campus type
To earn AADD in a subject area, a campus must be in the top quartile of its campus comparison group on the following percentages of the indicators applicable to the campus group type in that subject area: Elementary and Middle Schools: ≥ 50% of the indicators High Schools and K 12 campuses: ≥ 33% of the indicators‐
AADDsp. 24
Reading/ELA Indicators1. Attendance Rate2. Exceeded Progress
Reading3. Grade 7 Writing (Level
III)4. Grade 8 Reading (Level
III)
Math Indicators1. Attendance Rate2. Exceeded Progress Math3. Grade 5 Math (Level III)4. Algebra I by Grade 8
Participation5. Algebra I by Grade 8
Performance (Level III)
p. 14
p. 15
Distinction Designations
But how far off are we?
p. 14
Score NeededTo Get
IntoQ1
97.1%17%
2%16%
p. 15
Score NeededTo Get
IntoQ1
97.1%11%
13%50%
Distinction Designations
How do we know this?
p. 19
NEW!Released 8/29/2013
My Score
Where I WANT To Be
Distinction Designationsare summarized in
the campus’ 2013 Accountability
Summary(p. 1)
2013
Acc
ount
abilit
y Su
mm
ary
p. 1
AADDs(for Elementary
Campuses)p. 37
Reading/ELA Indicators1. Attendance Rate2. Exceeded Progress
Reading3. Grade 3 Reading
(Level III)4. Grade 4 Writing
(Level III)
Math Indicators1. Attendance Rate2. Exceeded Progress
Math3. Grade 5 Math (Level
III)
Connecting to
Instruction
2013 AccountabilityAchievement
How did we do?
Progress
Are our interventions
working?
Closing Gaps
Are we meeting the
needs of under-
performing groups?
Postsecondary
Are students ready for college or
career?
Index 1Student
Achievement
Index 2Student Progress
Index 3Closing
Performance Gaps
Index 4Postsecondary
Readiness
Are we meeting the needs of under-performing groups?
Are our interventions working?
Are students ready for college or career?
How did we do?
Appealing Ratings• Deadline: September 9, 2013
• Appeals are only accepted for a change in rating
• Appeals are not accepted for:– System Safeguard Measures– Distinction Designations
• Appeals are not an opportunity to correct data submissions
accountability connect
Year-long support for district leaders Webinars, tools,
resources, templates Telephone/email support
District and campus-level support through district contact Interpretation of data Appeals Planning for 2014