December 10, 2010Elaine Wood, Science Assessment Specialist

Upcoming Science Assessments:

the 2011 HSPE and the 2012 Biology End-of-Course Exam

Science Assessment OfficeElaine Wood—Secondary Specialist

elaine.wood@k12.wa.us

Kara Monroe—Elementary Specialistkara.monroe@k12.wa.us

Troi Williams—Administrative Assistanttroi.williams@k12.wa.us

TopicsGrade 5 & 8 Measurements of

Student ProgressNew StandardsStandard Setting

High School Update2011 High School Proficiency Exam 2012 Biology End-of-Course ExamStudent Performance

Resources available

Measurements of Student Progress

Assessing new standards in 2011

From WASL to Measurements of Student Progress

Embedded pilot items bring the item total to 40.The K-12 Science Learning Standards will be assessed for the first time in 2011.

Grade 8 2009WASL

2010MSP

2011MSP

Multiple Choice 28 30 26-29

Completion NA pilot only 1-4

Short Answer 7 5 5

Extended Response 2 0 0

% Points from MC/CP 56% 75% 75%

Operational ScoringOperational ScoringOperational RangefindingOperational

Rangefinding

Operational TestingOperational Testing Data ReviewData Review Pilot ScoringPilot Scoring

Pilot RangefindingPilot RangefindingScenario & Item

Pilot TestingScenario & Item

Pilot Testing

Science Assessment Development Cycle

Science Assessment Development Cycle

Scenario & Items Reviewed for

Content & Bias/Sensitivity by

Committees

Scenario & Items Reviewed for

Content & Bias/Sensitivity by

Committees

Scenario & Items Reviewed for

Content by OSPI

Scenario & Items Reviewed for

Content by OSPI

Scenario & Item Writing

Scenario & Item Writing

Develop Item and Test

Specifications

Develop Item and Test

Specifications

Standard Setting

for Science MSP in 2011

Science Standard Setting Plan

Date EventJune 2009 New Science standards approvedMay 2010 Spring 2010 MSP; New items pilotedAug 2010 Test Map meeting

Sept 2010 Performance Level Descriptors (PLDs) developedOct 2010 Spring 2011 tests built

Science Standard Setting Plan

Date EventJune 2009 New Science standards approvedMay 2010 Spring 2010 MSP; New items pilotedAug 2010 Test Map meeting

Sept 2010 Performance Level Descriptors (PLDs) developedOct 2010 Spring 2011 tests built

Feb/Mar 2011 Teachers from across state trained on how to use PLDs to participate in Contrasting Groups Study (CGS)

Mar/Apr 2011 NTAC and SBE formally approve standard setting planApril 2011 Teachers participate in CGS by providing ratings of studentsMay 2011 Spring 2011 MSP administered

Science Standard Setting Plan

Date EventJune 2009 New Science standards approvedMay 2010 Spring 2010 MSP; New items pilotedAug 2010 Test Map meeting

Sept 2010 Performance Level Descriptors (PLDs) developedOct 2010 Spring 2011 tests built

Feb/Mar 2011 Teachers from across state trained on how to use PLDs to participate in Contrasting Groups Study (CGS)

Mar/Apr 2011 NTAC and SBE formally approve standard setting planApril 2011 Teachers participate in CGS by providing ratings of studentsMay 2011 Spring 2011 MSP administeredAug 2011 Standard Setting events:

--Grade level panels (use “Bookmark process”, start with teacher ratings)

--“Policy panel” recommendations--NTAC certifies the planned process was followed

Aug 2011 SBE Sets Achievement Standards (Reviews all recommendations)

“Contrasting Groups” method affords broader involvement in standard setting

Everyone in the state who teaches science in grades 5 and 8 is invited to help set the performance standards.

•Performance Level Descriptors (PLDs) are developed for “Basic” and “Proficient”

•Web-based training on PLDs is provided to participating teachers

•In April, participating teachers judge each of their students as being “Basic or below” or “At or above Proficient”

•Two distributions of scores are produced: Scores obtained by students rated as “Basic”, and scores for “Proficient”

•Where the two distributions merge represents the separation point between “Basic” and “Proficient.”

Students rated as “Basic or below” using criteria in PLD for Basic

Students judged to be at or below "Basic"

Points on MSP (simulated)0 2 10 6 84 12 14 16 18 20 22 23 24 26 30 3228 34 40 36 38

Raw score points

Number of

students

Students rated as “Proficient or above” using criteria in PLD for Proficient

Students judged to be "Proficient" or above

Points on MSP (simulated)0 2 10 6 84 12 14 16 18 20 22 23 24 26 30 3228 34 40 36 38

Raw score points

Number of

students

Students judged to be at or below "Basic"

Students judged to be "Proficient" or above

Points on MSP (simulated)0 2 10 6 84 12 14 16 18 20 22 23 24 26 30 3228 34 40 36 38

Intersection indicates a region for where “Basic” separates from “Proficient”

Raw score points

Number of

students

Upcoming Events – Grade 5 & 8

Scenario and Item Writing – Feb. 14-18, 2011Content Review – April 4-8, 2011

PLD Training – Available via Moodle, Feb. 1, 2011Learn to use evidence to evaluate 5th or 8th grade

students’ proficiencyTakes 3 hours; 3 free clock hours are availableTo sign up, go to link and create an account

http://moodle.ospi.k12.wa.us/course/view.php?id=19

Contrasting Groups Study – Available via EDS, April 11-29After completing the PLD Training, 5th or 8th grade

teachers can predict their students’ performance on the MSP

5th and 8th grade science teachers can gain access to the EDS application by contacting a District Assessment Coordinator

Standard Setting – August 1-4, 2011

High School Proficiency Exam

A graduation requirement for sophomores in 2011

From WASL to High School Proficiency Exam to End of Course

High School 2009WASL

2010HSPE

2011HSPE

2012 EOC

Multiple Choice/Completion 28 35 35 ?

Short Answer 11 5 5 ?

Extended Response 3 0 0 ?

% Points from MC 45% 78% 78% ?

Embedded pilot items bring the item total to 45.Completion items will be piloted in 2011 and will be operational in 2012.The K-12 Science Learning Standards will be assessed beginning in 2012.

Implementation TimelineHSPE for 2011 will assess the 2005 GLEs

April 12, 2011

Biology EOC in 2012 will be administered to any student enrolled in a course credited as high school level biologyK-12 Science Learning StandardsOnline participation will be voluntary

Goal: 50% Online in 2012

First retake opportunity for biology EOC is likely to be January/February 2013

Possible additional science EOCs? TBD by LegislatureSuperintendent to make recommendation

Science Graduation Requirement: Classes of 2013 and BeyondUnder current rules, the students in the classes of

2013 and beyond can meet the science graduation requirement by:

Passing the science HSPE in spring 2011 and/or summer 20119th graders are not eligible to take science HSPE

Passing the biology EOCBegins in spring 2012; scores available August 2012

Meeting comparison scores on the SAT 2 or ACTScores set during the 2012-13 school year

Passing a science Collection of EvidenceBecomes available in the 2011-12 school year

Operational ScoringOperational ScoringOperational RangefindingOperational

Rangefinding

Operational TestingOperational Testing Data ReviewData Review Pilot ScoringPilot Scoring

Pilot RangefindingPilot RangefindingScenario & Item

Pilot TestingScenario & Item

Pilot Testing

Science Assessment Development Cycle

Science Assessment Development Cycle

Scenario & Items Reviewed for

Content & Bias/Sensitivity by

Committees

Scenario & Items Reviewed for

Content & Bias/Sensitivity by

Committees

Scenario & Items Reviewed for

Content by OSPI

Scenario & Items Reviewed for

Content by OSPI

Scenario & Item Writing

Scenario & Item Writing

Develop Item and Test

Specifications

Develop Item and Test

Specifications

Upcoming Events – High SchoolScenario and Item Writing

Feb. 21-25, 2011

June 20-24, 2011

Content Review March 28 – April 1, 2011

Aug. 15-19, 2011

Test Map – Spring 2011PLD Development – September 2011PLD Training – Winter 2011/12Standard Setting – August 2012

Patterns in Student Performance

High School WASL and HSPE

Strengths Areas to Improve Writing conclusions

52% to 58% earn full points

Identifying variables in investigations72% to 88% correct

Writing constructed response items related to investigations or solving human problems.

Explaining energy transfers or transformations in a system

Addressing persistent misconceptions

Resources Available

From the OSPI website www.k12.wa.us

OSPI Science Assessment Webpage

www.k12.wa.us

www.k12.wa.us/science

www.k12.wa.us/Science/Assessments.aspxSelect Science

Assessment from

Assessment Drop-down

Select Assessm

ents

Science Assessment ResourcesTest and Item Specifications—more

information about:Grade 5 & 8:

Types of scenariosTest mapItem specificationsVocabulary listsCognitive complexity

High School:Item specifications for 2005 GLEsVocabulary lists

NOTE: Test and Item Specifications for EOC coming late fall 2011

www.k12.wa.us/Science/Assessments.aspx

Grade 5 & 8 Test and Item Specs

www.k12.wa.us/Science/TestItemSpec.aspx

Select Test and

Item Specificati

ons

Select a

Grade

www.k12.wa.us/Science/Assessments.aspx

2011 HSPE Test and Item Specs

www.k12.wa.us/Science/TestItemSpec.aspx

Select Test and

Item Specificati

ons

Select GLEs with MSP and HSPE Evidences of Learning Science Item Specifications

Science Assessment ResourcesUpdates for 2011—each grade level

contains:General information on the 2011 testSample itemsScoring information, including rubrics

Updates for 2011

www.k12.wa.us/Science/Assessments.aspx

http://www.k12.wa.us/Science/EducatorResources.aspx

Select a

Grade

Select Educato

r Resourc

es

Science Assessment ResourcesTeacher Resource tool:

Types of scenariosGeneric questionsItem specifications (2005 GLEs)Cognitive complexity

Note that all tests posted here assess the 2005 GLEs.

Teacher Resource Tool

www.k12.wa.us/Science/Assessments.aspx

Select a Grade and a Year

Select Science

Select Teacher

Resource Tool

www.k12.wa.us/TeacherResourceTool2009-10/default.aspx

Teacher Resource Tool

Type of scenario

Generic question,

rolls over to item

specification

Percent correct or

mean score

Additional ResourcesPowerful Classroom Assessments (PCAs)—

Teaching materials developed from WASL released itemsNew PCAs will be developed in future for K-12 Science Learning

StandardsEach PCA includes:

Sample itemsScoring information, including rubrics

Item Analysis from the Assessment OverviewWebsite

https://eds.ospi.k12.wa.us/WASLTestItems/

Summary of performance on released WASL items by school, district and state

Thank you.