Post on 22-Feb-2016
description
transcript
PVAAS F2F Overview:Evaluating Growth,
Projecting Performance
Fall 2013
F2F Overview 2013
Goal of SessionBy the end of the session, participants will be able to answer the
following questions:
• What is PVAAS?
• How does PVAAS measure growth?
• What does PVAAS tell us about growth in our district and school(s)?
• What does PVAAS tell us about how students are projected to perform on a future state assessment?
2
F2F Overview 2013
WHAT IS PVAAS?
3
F2F Overview 2013 4
What is PVAAS?
• Longitudinal analysis of existing state assessment data
• Reliable estimates of academic growth & projections of future performance
• Effectiveness of the educational system
F2F Overview 2013 5
What is PVAAS?
• Growth of a group of students over time• NOT a simple comparison of test scores
Grade 3
Grade 4
Grade 5 Grade 6
F2F Overview 2013
PVAAS Addresses Complex Issues!
Research-based Statistical Modeling
•Minimizes the effects of measurement error associated with any one test scoreMeasurement
Error
•Accounts for mobility of studentsMobility
•Compensates for missing dataMissing Data
6
F2F Overview 2013
Demographics and Growth
As long as you can sufficiently account for the prior achievement of the student,…
7
…there is NO significant relationship between demographics and student progress, or growth!
F2F Overview 2013
Prior AchievementHow can we adequately account for prior achievement?
• The SAS EVAAS team has found that the only way to adequately account for prior achievement is to use ALL available test data for each student to dampen the effects of measurement error.
• PVAAS does NOT need to adjust for the demographic status of students directly since it is accounting for students’ prior achievement.– In other words, demographics are inherently controlled for in
PVAAS by the use of all longitudinal data on all students.
8
ActivityWork with a partner to decide if each statement is true or false.
9F2F Overview 2013
F2F Overview 2013 10
PVAAS Analyses
PSSA Writing
Keystone Exams:Algebra I, Biology,
Literature
PSSA Science
PSSA Reading
PSSA Math
LOOKING AHEADPlanning for
Students’ Needs:Student
Projections to Future Tests
LOOKING BACKEvaluating School
Effectiveness:Value Added &
Diagnostic Reports
What is PVAAS?
F2F Overview 2013
PVAAS Value-Added ReportingHow is growth measured in PVAAS?
11
PVAASValue-Added Reporting
Testing Subjects – Grades Methodologies
Assessments in consecutive
years
Mathematics, 4-8Reading, 4-8 Growth Standard
Assessments not in consecutive
years
Writing – 5 & 8 Science – 4 & 8
Keystones – Algebra I, Biology, & Literature
Predictive
12F2F Overview 2013
F2F Overview 2013
Measuring Academic Growth
Students don’t all start the year at the same place academically
13
F2F Overview 2013
Measuring Academic Growth
• Standard for PA Academic Growth– Regardless of their entering achievement level, students
should not lose ground academically
• Is it not a reasonable goal to maintain the achievement level of a group of students?– Not necessarily increase their achievement level, but
maintain their achievement level.
14
F2F Overview 2013 15
PVAAS Value-AddedGrowth Indicators
F2F Overview 2013
Measuring Growth in PVAAS• Remember, PVAAS does NOT compare the
achievement of one group of students in a particular subject and grade with the previous year’s students in the same subject and grade.
• PVAAS does NOT use the percentages of students at various academic performance levels to measure growth.
• PVAAS uses students’ scale scores within those performance levels to better estimate growth.
16
F2F Overview 2013 17
Growth Measures Not Reportedfor Prior Years
• PA’s state assessment system is transitioning to include Keystone end-of-course exams and statewide assessments aligned to the PA Common Core over the next several years.
• Growth Measures in prior years will NOT be included in fall 2013 reporting (SY12-13).
• Anticipated that Growth Measures from up to 3 previous years with 3-year averages will be yielded and reported beginning with fall 2015 reporting (SY14-15).
F2F Overview 2013
PVAAS Value-AddedGrowth Measures
• For Science, Writing and Keystone content areas, Growth is measured in scale score points.
• The Growth Measure is NOT reported in years and cannot be referenced in terms of number of year’s growth.
• The Growth Measure is NOT reported in percentages and cannot be referenced in terms of an increase or decrease in percentage points.
18
F2F Overview 2013
Purpose of Value-Added & Performance Diagnostic Reports
Assessing curricular needs across grades, subjects, and Keystone content areas
Evaluating instructional needs of students at various achievement levels and in different subgroups
Ensuring instructional coherence/alignment• Curriculum Instruction Assessment• Supports, including scheduling and staffing needs
Monitoring implementation and effectiveness of grade-level or course-specific programs
19
F2F Overview 2013
What Are Projections?
Given a specific set of circumstances…
…what’s the most likely outcome?
20
F2F Overview 2013
Setting Expectations
How do teachers set expectations for their students?
• Informationo What do they know about this
student?
• Experienceo How have students like this one
performed in the past?
21
F2F Overview 2013 22
PVAAS ProjectionsWhat are they based on?
Expectations based on what we know:
About this student:
»Prior test scores
»Across subjects
F2F Overview 2013 23
PVAAS ProjectionsWhat are they based on?
About other students:
» Prior test scores across subjects
» Scores on the test to which we are projecting
F2F Overview 2013
Given this student’s testing history, across subjects…
…what is the student likely to score on an upcoming test, assuming the average schooling experience?
What Are PVAAS Projections?
24
F2F Overview 2013
Student Projection Methodology
What data are used in calculating projections?
25
Projection TO…. Data Used to Calculate ProjectionPSSA Math PSSA Math and ReadingPSSA Reading PSSA Math and ReadingPSSA Science PSSA Math, Reading, and Science (in grades available)PSSA Writing PSSA Math, Reading, and Writing (in grades available)Keystone content areas (Algebra I, Literature, Biology)
PSSA Math, Reading, and Science (in grades available)
F2F Overview 2013
Student Projection Methodology
Which students have projections? In what subjects?
– Projections are available for all students who have a minimum of 3 data points. The exception to this are projections to grade 4 where only 2 data points from grade 3 are needed.
– The next two slides detail the grade levels, subjects, and Keystone content areas in which projections are reported.
26
PVAAS Projection Reporting: Projections to Keystones (Basic, Proficient, Advanced)
27
• Anticipating projections to future Keystone exams will be available to students who last tested in grade 5
• Remember, projections to a Keystone exam are NOT grade specific
• The projection probability indicates the probability of a student reaching the indicated performance level on a future Keystone exam – not necessarily at a specific grade level.
F2F Overview 2013
F2F Overview 2013
PVAAS Student Projection Report
28
F2F Overview 2013
PVAAS Student Projection Report
29
F2F Overview 2013
PVAAS Student Projection Report
30
F2F Overview 2013 31
LOOKING AHEAD
LOOKING BACK
F2F Overview 2013
PA’s Intent for PVAAS
• Measure of Growth to Add to Existing Achievement Measures
• Indicator of Likelihood of Future Student Performance to Add to Current Achievement Data
• Another Tool for Continuous Improvement
• Not to Be Used in Isolation
32
F2F Overview 2013 33http://www.education.state.pa.us/pvaas
Additional PVAAS Supports
www.pde.state.pa.us
PENNSYLVANIAValue Added Assessment System
______________________________________________________________________
PVAAS
333 Market Street Harrisburg, PA 17126