Pennsylvania Department of Education
PROGRESSPROGRESS&&
ACHIEVEMENTACHIEVEMENT
Pennsylvania Value-Added Assessment System
Pennsylvania Department of Education
Session Overview
History of PVAAS
Pennsylvania’s Intent for PVAAS
PVAAS Methodology and Reports
Timelines for PVAAS Implementation
Data Needed for PVAAS Analyses
Costs
District Preparation for PVAAS
Supports
Pennsylvania Department of Education
A Focus on A Focus on StandardsStandards
Pennsylvania Department of Education
A Standards-based System
Goal of high achievement, college readiness
Standards aligned
Assessments aligned
Safety nets aligned
Curriculum aligned
Instruction aligned
Instructional materials aligned
Pennsylvania Department of Education
Data Driven Decision Making Tools Provided to Districts
Leading for Learning!Getting Ready for the PSSAGetting Results!Data ToolsData ReportsPA GROWContext ToolsQuality ReviewsTutoring EvaluationsPVAAS
Student Planning Process
Focus: Classroom of Students
Who: Teacher
Periodic Grade Level Planning Process
Focus: Groups of StudentsWho: Teacher Teams
How: Regular 1-2 hour meetings
Annual Building-wide Planning ProcessFocus: All Students
Who: School-wide TeamHow: Data Retreat, School Planning Process
District-Level Support(Budgetary Support, Professional Development, Resources and Time)
Student Learning Data
School Structures for Data-Informed Decision Making
School LevelPSSA & PVAAS
Standardized AssessmentsDistrict End-of-Year Tests
Final Benchmark Test
Classroom Level Initial: PSSA/PVAAS/final tests – student levelCyclical: •Benchmark Data – Student LevelContinuous•Individual Classroom Assessments•Progress Monitoring
Grade/Course LevelInitial: PSSA/PVAAS/final tests – class/subgroup levelsCyclical:
•Benchmark Data - grade level•District quarterly assessments•Common Classroom Data•Classroom Summaries
Demographic/Perceptual/Process Data
School Level •School Demographics•Discipline Data•Attendance Data•Mobility Rate
Grade/Course Level•Class Demographics •Class Engagement Data•Satisfaction Data•Attendance Data
Classroom LevelQualitative Data•Student Historical Information•Student Medical Information•Student Learning Information
© Lancaster-Lebanon IU13
Pennsylvania Department of Education
History of PVAAS
2002: Recommended by Pennsylvania
League of Urban Schools (PLUS) to
State Board of Education
2002: State Board of Education passed
resolution for “value-added approach
across the Commonwealth”
2002 (SY04-05): 3 Pilot Phases
100 Districts, including the School District of
Philadelphia and Pittsburgh City School
District
Pennsylvania Department of Education
History of PVAAS
Fall ‘04 PDE Statewide Workgroup
Led by PA Secretary of Education, Dr. Zahorchak
50+ Members
Districts
Governor’s Office
Higher Education
Statewide Associations including, but not limited to:
PSEA, PFT, PASA, PSBA, PAIU, PAESSP, PASCD
Legislators
Business and Industry
SY05-06 Statewide Implementation - 501 Districts
1st Reporting in Fall 06
Pennsylvania Department of Education
PA’s Intent for PVAAS
Another Tool for Continuous Improvement
Measure of Growth to Add to
Achievement/Status Measures
Not to be Used in Isolation
No Teacher Level Analyses
Pennsylvania Department of Education
PA’s Intent for PVAAS
Integration of PVAAS into existing PDE initiatives:School Improvement Planning
Distinguished Educator Training
Governor’s Institutes
EAP
4Sight
Progress Monitoring
Content Initiatives: Literacy and Mathematics
Response to Intervention
PIL
PIMS (PA Information Management System)
Pennsylvania Department of Education
What is PVAAS?What is PVAAS?
Pennsylvania Department of Education
Student Learning Measures
Status MeasuresPSSA
% proficient or not proficient
Other local assessment data
Improvement MeasuresSafe Harbor
Other local assessment data
NEW MEASURE: Progress/Growth MeasurePVAAS
Pennsylvania Department of Education
Value Added Analysis
Provides a measure of what “effect”
schooling has in a single year
Formally described: Mixed model multivariate
longitudinal analyses
– The statistical methodology has been reviewed by
nationally renowned statisticians since the 1980’s
– The reporting has been used by educators for more
than 10 years
Pennsylvania Department of Education
PVAAS IS NOT:
Another test
Being used for teacher
level accountability in
Pennsylvania
Pennsylvania Department of Education
PVAAS IS:
A statistical analysis of existing assessment
data
A precise measurement of student
progress/growth over time
Progress data to add to achievement data as
schools are analyzing their work with
students and making improvement plans
Pennsylvania Department of Education
How is PVAAS analysis done?
Year 1 Year 2 Year 3
PSSA PSSA PSSA (7th)
5th 6th Predicted Actual
1. We collect all of the individual student data that is available for a minimum of two years.
2. All of these data are used to develop individual student profiles.
Reading 1280 1300
Math 1330 1305
1290 ??
1320 ??
Student ATesting Scores
Pool of Testing Histories
Computer
Prediction Model 1
Prediction Model 2
Prediction Model 6
Prediction Model 3
Prediction Model 4
Prediction Model 5
The PVAAS Methodology – Part 1
The PVAAS Methodology – Part 2
Student ATesting Scores
Prediction Model 1
Prediction Model 2
Prediction Model 6
Prediction Model 3
Prediction Model 4
Prediction Model 5
Student D Predicted Score
Student E Predicted Score
Student A Predicted Score
Student B Predicted Score
Student F Predicted Score
Student C Predicted Score
Mean of Student Scores for a particular school
Mean of Predicted Scores for a particular school
School Effect Mean of Student Scores – Mean of Predicted Scores
PVAAS Ratings of School EffectRemember:
School Effect Mean of Student Scores – Mean of Predicted Scores
So, if the students perform as expected, School Effect 0.
The ratings are in terms of the expected value of 0.
-2 SE’s 0 +2 SE’s
Not Detectably
DifferentAboveBelow
Standard Error (SE) is a measure of variation in the prediction
Pennsylvania Department of Education
Achievement and Progress
Pennsylvania Department of Education
Types of PVAAS Reporting
District Level
By School
School Level
By Grade
Reading and Math
Student Reports
Longitudinal Reports
“Projections” on Future PSSAs
Diagnostic Reports
By Quintiles
By Performance Levels
Diagnostic Subgroup Reports
Custom Reporting/Querying
District Summary Report
School Report
Above Expected Progress
Below Average
Achievement
Pennsylvania Department of Education
Student Projections
Search for students
by subgroup or achievement
Identified students meeting search requirements
Pennsylvania Department of Education
How Can PVAAS How Can PVAAS Help Me?Help Me?
Pennsylvania Department of Education
Key Learning from PVAAS
Value-added information will tell you:
How much progress students have made in each subject area and grade level
How much progress students at different prior achievement levels have made.
How students’ progress in one curricular area or program compares to their progress in others
Pennsylvania Department of Education
PSSA and PVAAS – The Power of 2!
Without data all we have is an opinion!
PSSA tells us where a student is.
PVAAS tells us where that student is headed.
Location and direction are critical partners in the
quest for having all students achieve proficiency
The data from both achievement and progress
measures gives schools valuable information to
make data informed decisions.
Pennsylvania Department of Education
PA Proposal to USDOE
PA submitted proposal to USDOE to use
“projections” as means to recalculate
proficiency/AYP; did not include value-
added reporting as part of proposal
PA is waiting to hear from USDOE
Pennsylvania Department of Education
Timelines for Implementation
Timeline Released in 2002
SY01-02 and SY02-03: Pilot Phase I
SY03-04: Pilot Phase II
SY04-05: Pilot Phase III
SY05-06: Statewide Implementation
All 501 Districts Receive PVAAS Reporting in Fall ‘06
Pilot districts get comprehensive PVAAS reporting
401 Districts Receive Limited Reporting
Grades 4 & 6
SY06-07: Comprehensive Reporting for All Districts
Pennsylvania Department of Education
Data Needed for PVAAS
PSSA reading and math will be the only data
needed for statewide implementation
Only common assessment for all 501 districts
No data preparation is needed
No purchase of any data is needed
All PSSA data is sent direct from DRC to SAS,
Inc.
Pennsylvania Department of Education
Costs Covered by PDE
Costs for PSSA Data Submission
Costs for PVAAS Analyses
Costs for PVAAS Web-Based Reporting
Costs for Communication Materials
Costs for Professional Development Materials
Costs for Professional Development SessionsAvailable through SY06-07
Regional sessions
Videoconferences
Webinars
Pennsylvania Department of Education
How Can Districts Prepare?
Plan to Use PVAAS Data as part of Continuous
Improvement Planning
District Level
Building Level
Grade Level
Content level
Subgroup Level
Student Level
Pennsylvania Department of Education
How Can Districts Prepare?
Professional Development on PVAAS
District Administrative Team
Building Level Administrators
Teacher Leaders
Classroom Teachers
Other Key Constituents
Communications to Board and Community
Pennsylvania Department of Education
Supports
Communications KitsPosted on PDE Website
Professional Development Materials
Professional Development SupportsStaff Trained:
PVAAS Statewide Core Team/PATTAN
PDE Staff
IU Staff
Distinguished Educators
PVAAS Statewide Work Group Members
Pennsylvania Department of Education
For Information, Resources or Training
PVAAS Core Team:
717-560-4611
Pennsylvania Department of Education