Unbridled LearningUnbridled Learning
Kentucky’s New Assessment and Accountability Model
Presented by Michele Reynolds, Ed.D. Associate Director of Assessment and Data Services
Fayette County Public [email protected]
A Balanced ApproachA Balanced ApproachNext-
Generation Learners
Next-Generation
Instructional Programs and
Support
Next-Generation
Professionals
Next-Generation
Schools/Districts
(reporting mechanism)
Achievement (Proficiency)
Gap
Growth
Readiness for College/Career
Graduation Rate
Program Reviews
Percent Effective Teachers
Percent Effective Leaders
= Revised Report Card
Proposed Overall Accountability Score (using data from the preceding columns)2011-2012 2012-2013 2014-2015
Overall Score Phase-InOverall Score Phase-In2011-2012Next Generation Learners – 100%
2012-2014Next Generation Learners – 77%Next Generation Inst. Prog. – 23%
2014-2015Next Generation Learners – 70%Next Generation Inst. Prog. – 20%Next Generation Professionals – 10%
What tests will students What tests will students take?take?
NRT vs. CRTNRT vs. CRTNationally norm-referenced section
embedded in the test for reading, math, science and social studies
All in one booklet – Part A
Timed test
You can’t prepare for an NRT in the same way you prepare for a CRT
Each student will receive 2 scores – N, A, P, D and percentile
Number of Items and Testing Number of Items and Testing TimesTimes(see handout in packet)(see handout in packet)Something to think about . . .
Each extended response question is worth 4 points, each short answer question is worth 2 points, and each multiple choice question is worth 1 point.
6th Grade Reading Example 66 MC – each worth one point for a total of 66 points 3 SA – each worth two points for a total of 6 points 2 ER – each worth four points for a total of 8 points 66 + 6 + 8 = 80 total points possible
Using the example above, let’s now think about the weight of each type of question on the 6th grade reading test:
MC is worth 66 points out of a possible 80 – 82.5% SA is worth 6 points out of a possible 80 – 7.5% ER is worth 8 points out of a possible 80 – 10%
Cautio
n: Th
ere
is not
an e
xact
scien
ce to
this.
It v
aries
by gra
de lev
el an
d subjec
t are
a. S
ome
item
s are
field
test
ing it
ems a
nd are
not
inclu
ded in
the
over
all sc
ore,
and o
nly ce
rtain
item
s in th
e NRT
sect
ion co
unt in th
e
CRT sc
ore.
This
is ju
st m
eant t
o be
a ball
park
calcu
lation
.
Next-Generation Learner Next-Generation Learner Accountability WeightsAccountability Weights
Grade Range
Achievement
(Proficiency)
Gap Growth
College/ Career
Readiness
Graduation
Rate
Total
Elementary
30 30 40 N/A N/A 100
Middle 28 28 28 16 N/A 100
High 20 20 20 20 20 100
Achievement (Proficiency)Achievement (Proficiency)
Achievement = proficiencyAchievement for elementary and
middle schools is based on performance on the reading, math, science, social studies and on-demand writing tests
Achievement for high school is based on performance on the end-of-course exams and the on-demand writing test
Achievement CalculationAchievement CalculationFor each content area . . .1 point awarded for each %
scoring proficient or distinguished0.5 points awarded for each %
scoring apprentice0 points awarded for novice
There are no more low, medium, high levels!
Bonus!!!Bonus!!!
Additional 0.5 awarded for each % distinguished combined with -0.5 for each % novice
Bonus points awarded in the amount that distinguished offsets novice
(NOTE: No points will be taken away if novice offsets distinguished).
Maximum 100 points in any subject area
Bonus ScenariosBonus ScenariosIf a school had . . . 20% Novice and 40% Distinguished20(-0.5) + 40(0.5) = -10 + 20 = 10 10 bonus points would be awarded
But what if it was just the reverse?If a school had . . .40% Novice and 20% Distinguished40(-0.5) + 20(0.5) = -20 + 10 = -10No bonus points would be awarded, but
none would be taken away either.
An Example . . . An Example . . .
Let’s say your school had the following results in reading:
Novice – 15%Apprentice – 20%Proficient – 40%Distinguished – 25%
Put Your Math Hats On!Put Your Math Hats On!You get 0 points for every % Novice:
0 x 15% = 0You get .5 points for every % Apprentice:
.5 x 20% = 10You get 1 point for every %
Proficient/Distinguished:1 x 40% = 401 x 25% = 25
Total: 0 + 10 + 40 + 25 = 75 points
But Wait! There’s More!But Wait! There’s More!Don’t forget to see if you qualify for the bonus
points . . .
You had15% Novice and 25% Distinguished15(-0.5) + 25(0.5) = -7.5 + 12.5 = 5
Good job! Your students earned 5 bonus points for your school!
75 reading achievment points + 5 bonus points = 80
Your Reading Achievement score would be an 80.
Overall Achievement Overall Achievement ScoreScoreFollow this same process for math, science, social studies and writing to get achievement scores in each of the 5 subject areas and average the scores.
For example, let’s say you earned 80 in reading, 71 in science, 82 in math, 93 in social studies, and 89 in writing.
80 + 71+ 82 + 93 + 89 = 415415 / 5 = 83
Your Overall Achievement score would be an 83.
Apply the WeightsApply the Weights
Multiple the overall achievement score by the achievement weight to figure out how many points out of the total score you have earned for achievement.
Middle School Example: 83 x .28 = 23.2 weighted points toward your overall Next Generation Learners accountability score
Grade Range
Achievement
(Proficiency)
Gap Growth
College/ Career
Readiness
Graduation
Rate
Total
Elementary
30% 30% 40% N/A N/A 100
Middle 28% 28% 28% 16% N/A 100
High 20% 20% 20% 20% 20% 100
GapGapGap = distance of the gap group
from the goal of 100% proficiencyGap calculation for elementary and
middle schools is based on performance on the reading, math, science, social studies and on-demand writing tests
Gap calculation for high school is based on performance on the end-of-course exams and the on-demand writing test
Non-duplicated Gap GroupNon-duplicated Gap GroupThe Student Gap Group is an aggregate
count of the following student groups:Ethnicity/race (Af. Am., Hisp., Nat.
Am.)Special educationPoverty (free/reduced lunch)Limited English Proficiency
NOTE: These groups are included because they are groups who have historically demonstrated achievement gaps nation-wide.
Yes or No? Yes or No? A student either IS or IS NOT in the Gap Group.
If the student meets any one (or more) of the criteria, then he/she is included in the gap group.
Criteria: Af Am, Hisp, Nat Am, SpEd, FRL, LEPDonatello – Af. Am, FRL Enrique – White, LEP Jo – Asian
Ricky – White, FRL, Sp. Ed. Michelle – White Marco – Hisp, LEP, FRL, SpEd
Gap GroupDonatello
RickyEnriqueMarco
Not Gap GroupMichelle
Jo
Gap CalculationGap CalculationThis yields a single gap number of
proficient or higher students with no student counting more than one time for each subject area.
For each subject area, the % of students scoring proficient or distinguished in the Non-duplicated Gap Group is reported.
These are added up and averaged for an overall Gap score.
More about the Gap More about the Gap CategoryCategoryThe achievement of all individual
groups will still be disaggregated and reported as they always have been, however . . .
Gap reporting for accountability purposes will include only the % of student in the combined non-duplicated Gap Group scoring at proficient and distinguished level.
Disaggregated Gap ReportDisaggregated Gap Report
Accountability of Individual Accountability of Individual GroupsGroupsSchools can be flagged for the state
consequences for underperforming individual gap groups.
All schools with gap groups underperforming in the third standard deviation (3 Sigma Model) will face state consequences.
KDE will use the 3 Sigma model to eliminate the masking of low-scoring groups.
An Example . . . An Example . . .
Let’s say your school had the following results in reading:
Novice – 25%Apprentice – 20%Proficient – 45%Distinguished – 10%
Calculating the Gap ScoreCalculating the Gap ScoreThe % of students in the Gap Group scoring
Proficient or Distinguished is reported as the Gap score. Novice and Apprentice scores are not included in the gap calculation.
45% Proficient + 10% Distinguished = 55%
Your Reading Gap Score would be a 55.
Overall Gap ScoreOverall Gap ScoreFollow this same process for math, science, social
studies and writing to get gap scores in each of the 5 subject areas and average the scores.
For example, let’s say you earned 55 in reading, 46 in science, 78 in math, 64 in social studies, and 91 in writing.
55 + 46+ 78 + 64 + 91 = 334334 / 5 = 66.8
Your Overall Gap score would be a 67.
Apply the WeightsApply the Weights
Multiple the overall gap score by the gap weight to figure out how many points out of the total score you have earned for achievement.
Elementary School Example: 67 x .30 = 20.1 weighted points toward your overall Next Generation Learners accountability score
Grade Range
Achievement
(Proficiency)
Gap Growth
College/ Career
Readiness
Graduation
Rate
Total
Elementary
30 30 40 N/A N/A 100
Middle 28 28 28 16 N/A 100
High 20 20 20 20 20 100
GrowthGrowth
Growth = comparison of an individual student’s score from one year to the next
Growth for elementary and middle schools is based on performance on the reading and math tests
Growth for high schools is based on PLAN to ACT in reading and math
Growth ScalesGrowth ScalesGrowth comparisons are made to
similar performing “academic peers.”Low, typical and high growth scales
will be determined based on actual student performance state-wide.
These are equal-interval, percentile-based scales.
Based on model currently used by Massachusetts Department of Education
Growth CalculationGrowth Calculation
For reading and math . . . 1 point for each % of students showing typical or high growth
0 points for low growth
An Example . . . An Example . . .
Let’s say your school had the following results in reading:
Low Growth – 40%Typical Growth – 45%High Growth – 15%
Calculating the Growth Calculating the Growth ScoreScore
You get 0 points for every % of students with low growth:0 x 40% = 0
You get 1 point for every % with typical or high growth:1 x 45% = 451 x 15% = 15
Total: 0 + 45 + 15 = 60 points
Overall Growth ScoreOverall Growth ScoreFollow this same process for math and then
average the reading and math scores.
For example, let’s say you earned 60 in reading and 86 in math.
60 + 86 = 146146 / 2 = 73
Your Overall Growth score would be a 73.
College/Career ReadinessCollege/Career Readiness
College/Career Readiness = number of graduates successfully meeting an indicator of readiness
Readiness for middle schools is based on performance on EXPLORE test
Readiness for high schools is based on ACT, college placement tests and career measures
College/Career Readiness College/Career Readiness CalculationCalculationMiddle Schools% of students meeting the ACT-
established benchmarks for EXPLORE in reading, English and math is averaged to generate an overall readiness percentage
EXPLORE BenchmarksReading – 15English – 13Math – 17
37
College/Career Readiness RateCollege/Career Readiness RateCollege Ready:
Must meet benchmarks on one of the
following:
College Ready
ACT
COMPASS
KYOTE
Career Ready: Must meet benchmarks for one
requirement in Career Academic area and
must meet one requirement in
Career Technical area
Career Ready Academic
Career Ready Technical
Armed Services Vocational
Aptitude Battery (ASVAB)
ACT Work Keys (Applied Math,
Locating information, and Reading
for Information)
Kentucky Occupational
Skills Standards Assessment
(KOSSA)
Industry Certificates
Bonus: College AND Career Ready must meet at least one from each
area
College Ready AcademicCareer Ready
Technical
ACT or COMPASS or KYOTE
KOSSA
Industry Certificates
NOTES: (1) By meeting the College Ready Academic definition, the student does not have to take the additional tests of ASVAB or Work Keys for the bonus area. (2) For accountability purposes, the bonus shall not allow the readiness percentageto exceed 100 percent.
37
College ReadyCollege Ready
KDE:OAA:DSR: 2/3/2012
38
College Placement Tests
College Entrance Exam
1 point
Career ReadyCareer Ready
ACADEMIC
WorkKeys = Silver or Above
◦ Applied Math
◦ Locating Information
◦ Reading for Information
---OR--- ASVAB = 55
Armed Services Vocational Aptitude Battery
TECHNICAL
KOSSA
◦ Multiple-Choice = 70
◦ Constructed-Response = 75
---OR--- Industry Certificate
Certification based on industry standards
39
1 point
College College && Career Ready Career Ready
40
1.5 points
Graduation RateGraduation Rate
Graduation Rate = % of students graduating
Average Freshman Graduation Rate (AFGR) is current transitional reporting model
Federally-mandated Graduation Cohort Model goes into effect in three years
Elementary Accountability Elementary Accountability DesignDesign
Achievement•Reading, Math, Science, Social Studies, Writing—all equally weighted•One point for each % of students proficient or higher and ½ point for each % of students at apprentice. Bonus ½ point for % distinguished students (offset by novice and not to exceed 100)
Gap•Non-duplicated Gap Group—students only count once even if they are in multiple gap groups•Reading, Math, Science, Social Studies, Writing—all equally weighted•One point for each % of students proficient or higher
Growth•Student Growth Percentile—compares the growth of a student to the student’s academic peer group•Reading and Math Only—each equally weighted•One point for each % of students showing typical or high growth
Next-Generat
ion Learner
s
CurriculumReading, Math, and Writing—Common Core State
StandardsScience and Social Studies—Core Content 4.1
Test DesignBlended CRT/NRT
Multiple Choice, Short Answer, and Open Response Questions
Slibeck Francis, 2011
Middle School Accountability Middle School Accountability DesignDesign
Achievement•Reading, Math, Science, Social Studies, Writing—all equally weighted•One point for each % of students proficient or higher and ½ point for each % of students at apprentice. Bonus ½ point for %distinguished students (offset by novice and not to exceed 100)
Gap•Non-duplicated Gap Group—students only count once even if they are in multiple gap groups•Reading, Math, Science, Social Studies, Writing—all equally weighted•One point for each % of students proficient or higher
Growth•Student Growth Percentile—compares the growth of a student to the student’s academic peer group•Reading and Math Only—each equally weighted•One point for each % of students showing typical or high growth Curriculum
Reading, Math, and Writing—Common Core State Standards
Science and Social Studies—Core Content 4.1Test Design
Blended CRT/NRTMultiple Choice, Short Answer, and Open Response
QuestionsSlibeck Francis, 2011
Next-Generat
ion Learner
s
College/Career Readiness•One point for each % of students reaching benchmark scores on the EXPLORE in the areas of Reading (15), English (13) and Math (17)•Each content is averaged together to reach the Readiness Scores
High School Accountability High School Accountability DesignDesign
Achievement•End of Course exams in Reading, Math, Science, Social Studies, Writing—all equally weighted•One point for each % of students proficient or higher and ½ point for each % of students at apprentice• Bonus ½ point for % distinguished students (offset by novice and not to exceed 100)Gap•Non-duplicated Gap Group—students only count once even if they are in multiple gap groups•End of Course exams in Reading, Math, Science, Social Studies, Writing—all equally weighted•One point for each % of students proficient or higher
Growth•Student Growth Percentile—compares the growth of a student to the student’s academic peer group•PLAN to ACT—Reading and Math Only—each equally weighted•One point for each % of students showing typical or high growth
CurriculumReading, Math, and Writing—Common Core State
StandardsScience and Social Studies—Core Content 4.1
Test DesignEnd of Course Exams—English II, Algebra II, Biology,
US HistoryMultiple Choice and Constructed Response Questions(More End of Course Exams Will Be Added in Future
Years)Student Accountability
May Count for Up to 20% of Student’s Final Grade
Slibeck Francis, 2011
Next-Generat
ion Learner
s
College/Career Readiness•One point for each % of students reaching ALL CPE benchmark scores (Reading (20) , English (18) and Math (19)) on ACT, College Placement Tests (KYOTE or Compass) or Other Career Measures
Graduation Rate•AFGR—Average Freshman Graduation Rate for Classes of 2011, 2012, and 2013•Cohort Model—Class of 2014 and beyond•One Year Lag in Reporting
School and District School and District ClassificationsClassificationsThere are 100 points total for each
school. Each school will be classified according to the total number of points earned (cut scores yet to be determined . . .)
Classification CategoriesDistinguishedProficientNeeds ImprovementPersistently Low Achieving (bottom 5%)
Unbridled Learning Unbridled Learning Accountability ModelAccountability Model
AREAS OF MEASUREMENT
Achievement
Gap Closing
Individual Student Growth
College/Career Readiness
Graduation Rate
OVERALL SCORE
Scale of 0 - 100
RANKING BY PERCENTILE
CATEGORIES
Distinguished (90th percentile and above)
Proficient (70th percentile)
Needs Improvement (below 70th percentile)
•considered Progressing if meeting annual AMO beginning in year 2
Instructional inertia – the tendency to keep teaching the way we have always taught
What are the implications of this new system on classroom instruction and on district/school intiatives?
What must we do differently in order to ensure that gaps close and growth for every child occurs?
