PROGRESSPROGRESS &&
ACHIEVEMENT
ACHIEVEMENT
Pennsylvania Value-Added Assessment System (PVAAS)
The Power of
<X>School District<X>School District
PDE Data Tools
PSSA Data PSSA Data Interactive Interactive by eMetricby eMetric
Interactive tool for analyzing PSSA Interactive tool for analyzing PSSA data by district, school, grade, data by district, school, grade, subgroup and student – Reporting subgroup and student – Reporting category information is also availablecategory information is also available
PVAASPVAASValue-added tool that reports Value-added tool that reports growth growth of cohorts of studentsof cohorts of students and and projections projections for individual studentsfor individual students for performance for performance on future PSSAson future PSSAs
PA AYPPA AYPPublic site that reports all AYP related Public site that reports all AYP related data on district, school, grade and data on district, school, grade and subgroup - Reporting category subgroup - Reporting category information is also availableinformation is also available
4Sight 4Sight Member Member CenterCenter
Reporting site for benchmark data by Reporting site for benchmark data by grade, class, subgroup and student – grade, class, subgroup and student – intended to be formative and used intended to be formative and used often throughout the school yearoften throughout the school year
Stair-Step ExpectationsStair-Step Expectations
Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test
In a perfect world students would start school at about the same academic level. Proficiency levels could simply be achieved by students
gaining age-appropriate knowledge and skills each year.
Differentiated RealityDifferentiated Reality
Grade Three Grade Four Grade Five
Students begin school at different places; they often progress at different rates. Yet all
students are expected to achieve the same level of academic proficiency each year.
Grade 3 Test Grade 4 Test Grade 5 Test
The Need for Progress Measures
Grade Three Grade Four Grade Five
To measure school effectiveness in this “differentiated” world, we need to pay attention not only to proficiency levels, but also to how much progress students make in a given year.
Grade 3 Test Grade 4 Test Grade 5 Test
PVAASPVAAS ISIS::
Another tool for continuous school improvement
A statistical analysis of existing assessment data (PSSA).
A realistic measurement of a school’s progress/growth over time.
Not to be used in isolation
Progress data to add to achievement data as schools are analyzing their work with students and making improvement plans.
PVAAS is NOT:PVAAS is NOT:
Another Test
A measure for teacher accountability
The PVAAS MethodologiesThe PVAAS Methodologies
Today
Looking Back/Evaluation…
Value-added Growth Reports For Cohorts of Students
Looking Forward/Planning
…
PVAAS Projection Reports For Individual Students and Cohorts
of Students
PVAAS Fall 2008 ReportingPVAAS Fall 2008 Reporting(math, rdg; wrtg; science: coming late fall)(math, rdg; wrtg; science: coming late fall)
Value-added growth reports for grades 4 through 8, and 11 (Last’s years cohorts)
Projections to basic, proficient, advancedGrade last tested - 3 4 or 5 (future PSSAs)Grade last tested - 4 5 or 6 (future PSSAs)Grade last tested - 5 6 or 7 (future PSSAs)Grade last tested - 6 7 or 8 (future PSSAs)Grade last tested - 7 8 (future PSSAs)Grade last tested - 8 11 (future PSSAs)
Student ProjectionsStudent Projections
Wouldn’t it be great Wouldn’t it be great
to know the likelihood to know the likelihood
that a student will that a student will
be proficient on abe proficient on a
futurefuture PSSA? PSSA?
Student Projection MethodologyStudent Projection Methodology
A separate longitudinal modeling process A separate longitudinal modeling process focused on the performance scores of focused on the performance scores of individual students.individual students.
The model uses The model uses allall of the available data (both of the available data (both math and reading) to project future PSSA math and reading) to project future PSSA performances in Math and Reading.performances in Math and Reading.
Projections can be calculated for an Projections can be calculated for an individual student or groups of students.individual student or groups of students.
PVAAS Student Projection ReportPVAAS Student Projection Report
Identified Students Meeting Search Requirements
Data Packets: Student LevelData Packets: Student LevelTimingTiming SourceSource ReportReport
Initial Meeting – Initial Meeting –
Looking BackLooking BackPSSA: eMetric/GROWPSSA: eMetric/GROW Individual Student ReportIndividual Student Report
Initial Meeting – Initial Meeting –
TodayToday4Sight4Sight BaselineBaseline
Initial Meeting – Initial Meeting –
Looking ForwardLooking ForwardPVAASPVAAS PVAAS ProjectionPVAAS Projection
Periodic Meetings Periodic Meetings
(based on subject area and (based on subject area and grade level)grade level)
Examples:Examples:
DIBELSDIBELS
4Sight4Sight
GRADEGRADE
GMADEGMADE
Core PhonicsCore Phonics
Local FormativeLocal Formative
AssessmentAssessment
PVAAS Projections with Other Data:PVAAS Projections with Other Data:Elementary Scenario/ReadingElementary Scenario/Reading
Student NOW in 5Student NOW in 5thth Grade Grade
PSSAPSSAGrade 4 Proficiency Grade 4 Proficiency Range: 1255-1468Range: 1255-1468 12101210
4Sight 4Sight BaselineBaseline
Grade 5 Proficiency Grade 5 Proficiency Range: 1275-1496Range: 1275-1496 12001200
PVAAS PVAAS ProjectionsProjections
To Basic:To Basic:
To Proficient:To Proficient:
To Advanced:To Advanced:
93.6%93.6%
64.8%64.8%
5.5%5.5%
DIBELSDIBELS
ORFORF
Grade 5 – Low RiskGrade 5 – Low Risk
104+ at beginning of year104+ at beginning of year
124+ at end of year124+ at end of year
8989
Some Some RiskRisk
PVAAS Projections with Other Data:PVAAS Projections with Other Data:High School Scenario/MathHigh School Scenario/MathStudent NOW in 9Student NOW in 9thth Grade Grade
PSSAPSSAGrade 8 Proficiency Range Grade 8 Proficiency Range
1284-14451284-1445 14501450
4Sight 4Sight BaselineBaseline
Grade 9 Proficiency Range Grade 9 Proficiency Range 1304-15081304-1508 15071507
PVAAS PVAAS ProjectionsProjections
(11(11thth Grade) Grade)
To Basic:To Basic:
To Proficient:To Proficient:
To Advanced:To Advanced:
98.7%98.7%
89.9%89.9%
45.7%45.7%
Grade 8 Grade 8 Final Final GradesGrades
Final Exam: 91%Final Exam: 91%
Course Grade: 87%Course Grade: 87%
What Does This Mean to You?What Does This Mean to You?
The district could provide you each The district could provide you each with the probability of students in with the probability of students in your classrooms (current grades 4-9)your classrooms (current grades 4-9)
Grade level/course specific teams Grade level/course specific teams can collaborate to make sure can collaborate to make sure adequate supports are in place for all adequate supports are in place for all students to reach success.students to reach success.
How could we use this information?How could we use this information? For students with each level of probability For students with each level of probability
of proficiency, what do we need to keep of proficiency, what do we need to keep doing to keep at this level? What do we doing to keep at this level? What do we need to do to increase the level of need to do to increase the level of probability?probability?• 70-100% likelihood of proficiency 70-100% likelihood of proficiency
– – what is our action plan?what is our action plan?
• 40-70% likelihood of proficiency 40-70% likelihood of proficiency – – what is our action plan? Goal date for proficiency?what is our action plan? Goal date for proficiency?
• 0-40% likelihood of proficiency 0-40% likelihood of proficiency – – what is our action plan? Goal date for proficiency?what is our action plan? Goal date for proficiency?
Wouldn’t it be great Wouldn’t it be great to know if we made one to know if we made one year’s growth with last year’s growth with last year’s 4year’s 4thth, 5, 5thth, 6, 6thth, 7, 7thth, ,
and 8and 8thth graders graders?? 1111thth graders? graders?
Value-AddedValue-Added
Value-Added Measure of Value-Added Measure of Cohort GrowthCohort Growth
Comparison to a State Growth StandardComparison to a State Growth Standard
The Growth Standard specifies the The Growth Standard specifies the minimal designated academic gain from minimal designated academic gain from grade to grade for a cohort of students. grade to grade for a cohort of students.
The use of a Growth Standard creates the The use of a Growth Standard creates the possibility that possibility that ALLALL schools can schools can demonstrate appropriate growth.demonstrate appropriate growth.
Caution IndicatorCaution Indicator
Estimated gain below growth standard but Estimated gain below growth standard but by less than one standard error.by less than one standard error.Students in this cohort have grown less Students in this cohort have grown less than the standard.than the standard.
Stronger CautionStronger Caution
Estimated gain below growth standard by Estimated gain below growth standard by more than one but less than two standard more than one but less than two standard errors.errors.Students in this cohort have fallen behind Students in this cohort have fallen behind their peers.their peers.
Strongest Strongest WarningWarning
Estimated gain well below growth standard Estimated gain well below growth standard by more than two standard errors.by more than two standard errors.Students in this cohort have made little Students in this cohort have made little progress.progress.
PVAAS Value-added CohortPVAAS Value-added CohortGrowth DescriptorsGrowth Descriptors
Favorable Favorable IndicatorIndicator
Estimated gain at or above growth Estimated gain at or above growth standard. standard. Students in this cohort have made at least Students in this cohort have made at least one year growth.one year growth.All schools can achieve this rating.All schools can achieve this rating.
PVAAS School ReportPVAAS School ReportGrades 4 through 8Grades 4 through 8
Every grade in Every grade in the school has the school has met or met or exceeded a exceeded a year’s worth of year’s worth of growth?growth?
(“Getting Results!”™, p. 12)(“Getting Results!”™, p. 12)
1111thth Grade Growth Descriptors Grade Growth Descriptors
Below
Observed performance is significantly BELOW the expected performance.
NDD
Observed performance is NOT DETECTABLY DIFFERENT than the expected performance.
Above
Observed performance is significantly ABOVE the expected performance.
Grade 11 School Report
Every performance Every performance levels (below basic, levels (below basic, basic, proficient, basic, proficient, advanced) in each advanced) in each grade has met or grade has met or exceeded a year’s exceeded a year’s worth of growth?worth of growth?
(“Getting Results!”™, p. 12)(“Getting Results!”™, p. 12)
Performance Diagnostic ReportPerformance Diagnostic Report
What the whiskers tell us…What the whiskers tell us…
Growth Standard
(One Year’s Growth)
Met the Growth Standard;Made One Year’s Growth
Yellow
Below Growth Standard;Less than One Year’s Growth
Rose
Exceeded Growth Standard;More than One Year’s Growth
Green
Every Every performance level performance level (below basic, (below basic, basic, proficient, basic, proficient, advanced) in each advanced) in each grade has met or grade has met or exceeded a year’s exceeded a year’s worth of growth?worth of growth?
(“Getting Results!”™, p. 12)(“Getting Results!”™, p. 12)
Patterns: Performance Diagnostic ReportPatterns: Performance Diagnostic Report
Patterns of Growth
A
B
C
What Does This Mean to You?What Does This Mean to You?
The district could provide each grade The district could provide each grade level/course specific team with their level/course specific team with their growth reportsgrowth reports• Grades 4-8, and 11 Grades 4-8, and 11 (Last’s years cohorts)(Last’s years cohorts)• Subgroup reportingSubgroup reporting
How could we use this information?How could we use this information?
• At a grade level/course specific team, At a grade level/course specific team, discuss why and how you may have gotten discuss why and how you may have gotten the growth results on the report?the growth results on the report?
• Which students made a year’s worth of Which students made a year’s worth of growth?growth?
• Which did not?Which did not?• Why? Why not?Why? Why not?• What is our team’s plan of action as a What is our team’s plan of action as a
grade level/course specific team?grade level/course specific team?
Where do we go from here?Where do we go from here?
1.1. How do we plan to use our PVAAS data this next How do we plan to use our PVAAS data this next month? This school year?month? This school year?
• Value-added reportsValue-added reports• Diagnostic reportsDiagnostic reports• Student projectionsStudent projections
2.2. How do we plan to share our PVAAS data?How do we plan to share our PVAAS data?• With other Administrators?With other Administrators?• With Teachers?With Teachers?• With Parents?With Parents?• With Others?With Others?
District Goal: PVAAS District Goal: PVAAS will be used by teachers and will be used by teachers and administratorsadministrators for annual for annual school school evaluation and evaluation and planning, grade/subject level evaluation and planning, grade/subject level evaluation and planning, and for student level planning.planning, and for student level planning.
Data Packet: Annual School-Level AnalysisData Packet: Annual School-Level AnalysisQuestion Tool Report
1) School Met AYP? PAAYP Performance Chart
2) Every Grade Met AYP? eMetric 3-Year Portrait
3) Every Grade Met Growth Standard? PVAAS School Value-added Report
4) Every Grade 3 Year Trend Change in Proficiency?
eMetric 3-Year Portrait
5) Every Predicted Performance Level Met Growth Standard?
PVAAS Performance Diagnostic Report
6) Every Subgroup Met AYP? eMetric 3-Year Portrait
7) Achievement Gap Narrower? eMetric 3-Year Portrait
8) Every Grade Projected To Meet AYP Target?
PVAAS Grade Projection Summary
9) School and Subgroups Met Participation Targets?
PAAYP Data Table
10) Every Subgroup Met Growth Standard?
PVAAS Performance Diagnostic Report
11) Subgroups Performed Similarly in Reporting Categories?
eMetric 3-Year Portrait
Data Packets: Grade LevelData Packets: Grade LevelTimingTiming SourceSource ReportReport
Initial Meeting – Initial Meeting – Looking BackLooking Back
eMetriceMetric
eMetriceMetric
PVAASPVAAS
PVAASPVAAS
eMetric/eMetric/PVAASPVAAS
Performance LevelsPerformance Levels
Reporting CategoriesReporting Categories
Value Added ReportValue Added Report
Performance Diagnostic ReportPerformance Diagnostic Report
PSSA/ PSSA/ PVAAS ScatterplotsPVAAS Scatterplots
Initial Meeting– Initial Meeting– Looking ForwardLooking Forward
PVAASPVAAS
4Sight Baseline4Sight Baseline
4Sight Baseline4Sight Baseline
Projection Summary ReportProjection Summary Report
Performance LevelsPerformance Levels
Subscale ReportsSubscale Reports
Initial Meeting– Initial Meeting– Subgroup ComparisonsSubgroup Comparisons
Same as aboveSame as above Same as aboveSame as above
Periodic MeetingsPeriodic Meetings 4Sight4Sight
4Sight4Sight
Local FormativeLocal Formative
AssessmentAssessment
Performance LevelsPerformance Levels
Subscale ReportsSubscale Reports
Data Packets: Student LevelData Packets: Student LevelTimingTiming SourceSource ReportReport
Initial Meeting – Initial Meeting –
Looking BackLooking BackPSSA: eMetric/GROWPSSA: eMetric/GROW Individual Student ReportIndividual Student Report
Initial Meeting – Initial Meeting –
TodayToday4Sight4Sight BaselineBaseline
Initial Meeting – Initial Meeting –
Looking ForwardLooking ForwardPVAASPVAAS PVAAS ProjectionPVAAS Projection
Periodic Meetings Periodic Meetings
(based on subject area and (based on subject area and grade level)grade level)
Examples:Examples:
DIBELSDIBELS
4Sight4Sight
GRADEGRADE
GMADEGMADE
Core PhonicsCore Phonics
Local FormativeLocal Formative
AssessmentAssessment
Statewide PVAAS Professional Statewide PVAAS Professional Development, Support & Development, Support &
MaterialsMaterials
Contact InformationContact Information [email protected]@iu13.org
717-606-1911717-606-1911
PVAAS Websitehttps://pvaas.sas.com
Gerald L. Zahorchak, D.Ed. Secretary of Education
Commonwealth of Pennsylvania
www.pde.state.pa.us