Next Generation Science MCAS: Updates from ESE Massachusetts Science Education Leadership Association 2016 Conference, Marlborough, MA Katie Bowler Brendan Harmon, Ph.D. Jass Stewart October 20, 2016
Project Update: Next-generation MCAS and learning standards review
Jass Stewart, Special Assistant to the Chief of Staff Office of the Commissioner
Today’s update includes: Performance of MA students 1993 education reform State and local K-12 spending Upgrading standards and assessments The need for a next-generation MCAS Project timelines and advisory workgroups Process for finalizing workgroup
recommendations How to learn more
Massachusetts Department of Elementary and Secondary Education
3
MA students are #1 in the country
Massachusetts Department of Elementary and Secondary Education
4
2005 2007 2009 2011 2013 2015
Grade 4 Reading 1st 1st 1st 1st 1st* 1st
Grade 8 Reading 1st 1st* 1st* 1st* 1st* 1st*
Grade 4 Math 1st 1st 1st* 1st* 1st* 1st*
Grade 8 Math 1st 1st 1st 1st 1st 1st* Source: The National Assessment of Educational Progress (NAEP), also known as the Nation’s Report Card * Statistically tied with other states
MA students are globally competitive
Massachusetts Department of Elementary and Secondary Education
5
2011 TIMSS (8th-grade students)
2nd* Science
5th* Math
2012 PISA (15-year-old students)
4th Reading
7th Science literacy
10th Math
Source: Trends in International Mathematics and Science Study (TIMSS); Program for International Student Assessment (PISA) * Statistically tied with another country
The 1993 “Grand Bargain”
Massachusetts Department of Elementary and Secondary Education
6
More equitable funding
High learning standards
High-quality student
assessments
School choice (vo-tech schools,
charters)
Accountability for results
State Budget: K-12 education spending is $5.2 billion
Massachusetts Department of Elementary and Secondary Education
7 88.2%
5.4%
5.3%
0.6%
0.5%
Chapter 70 Aid
Grant Program/Direct Services to S&D SPED Circuit Breaker Program State Supported Services General Administration
Local School Aid
Grants
SPED Reimbursement
Core Services (Assessment, Licensure, Accountability, etc.)
Administration
ESE and educators are upgrading our standards and assessments
Upgrading Academic Standards
English/Language Arts Mathematics
Science/Technology/Engineering History and Social Science
Upgrading Assessments (next-generation MCAS)
Aligned to upgraded college- and career-ready MA academic standards
Built on existing development Augmented with new development
Massachusetts Department of Elementary and Secondary Education
8
Why MCAS is an important tool for schools, parents, and students
Massachusetts Department of Elementary and Secondary Education
9
Curriculum improvements (local level)
School and district accountability levels
(state level)
Parents’ and teachers’ monitoring of student
progress
In addition: •In high school, MCAS is a graduation requirement.
•Student progress on statewide assessments over time is one of many factors that inform educator evaluations.
The need to align MCAS with our college- & career-ready standards
Current MCAS designed to measure readiness to graduate from high school, not readiness for college and career
More than 1/3 of MA students who enroll in MA public colleges are placed in remedial English and math courses
Massachusetts Department of Elementary and Secondary Education
10
State Board of Education vote to develop next-generation MCAS
Mass. “test drove” PARCC, an assessment created by the multi-state PARCC consortium. Some MA districts used PARCC instead of MCAS, and some used the computer-based format, introducing MA to a computer-based assessment aligned to college- and career-ready standards.
MA Board voted in November 2015 to move forward with a MA-specific assessment that will include new material and elements of both MCAS and PARCC while maintaining MA control. The new assessment is the next-generation MCAS.
Massachusetts Department of Elementary and Secondary Education
11
Spring 2016 assessments bridged old and new Exposure to next-generation test items and questions
72% of MA students in grades 3–8 took PARCC for English language arts and math
MCAS English language arts and math tests for grades 3–8 included PARCC items, which were scored for diagnostic purposes only
Administration of PARCC computer- vs. paper-based tests 44% of PARCC districts administered PARCC on a computer
39% of PARCC districts used the paper version
17% of PARCC districts administered a mix of paper and computer-based tests
Massachusetts Department of Elementary and Secondary Education
12
Timeline to update MCAS and the state’s learning standards
Massachusetts Department of Elementary and Secondary Education
13
Nov Board of Ed vote
Jan Advisory workgroups convene
Feb ELA and Math Standards Review Panel begins
Spring RFR issued for Next-Gen MCAS vendor Spring Test administration window (Some districts chose to administer PARCC this year, some chose MCAS)
Summer/Fall Test development
Spring Grades 3–8 take next-gen MCAS in ELA and mathematics Grades 4 and 8 take computer-based version of next-gen MCAS as part of plan to phase in computer-based testing Districts can offer computer-based version of next-gen MCAS to other grades (3, 5–7) if desired or use paper versions for those grades
2015 2016 2017
Summer Assessment contract awarded
Next-generation MCAS workgroups and committees
Massachusetts Department of Elementary and Secondary Education
14
Board Assessment Committee
Project Management Workgroup
Project Communications Workgroup
Procurement Management Workgroup
Test Administration Workgroup
High School Testing Workgroup
ELA and Mathematics Standards Review Panel
MCAS Accessibility Workgroup
Digital Learning Advisory Council
Test Content Review Panels MCAS Technical Advisory
Committee Standard Setting
Committee School and District
Accountability and Assistance Advisory Council
History and Social Science Workgroup
PARCC Consortium Committee
Jan-May 2016
Workgroup recommend-
ations
Apr-Summer 2016
Analysis of recommend-ations and
refinements
Fall 2016
Public comment
Fall/Winter 2016
Final recommend-ations to the
state Board of Education
Process for finalizing first phase of workgroup recommendations
Massachusetts Department of Elementary and Secondary Education
15
Q&A and more information Write [email protected] Visit www.doe.mass.edu/mcas/nextgen/
Science and Tech/Eng MCAS: Update Katie Bowler, Administrator for Science Test Development Brendan Harmon, Science Test Development Specialist
Updates
2016 Student Performance
STE MCAS Timeline
Proposed Test Designs
Grades 5 and 8 Tryouts
Questions
Massachusetts Department of Elementary and Secondary Education
18
2008-2016 Statewide Science MCAS Results
Massachusetts Department of Elementary and Secondary Education
19 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2008 2009 2010 2011 2012 2013 2014 2015 2016
Per
cent
age
of S
tude
nts
scor
ing
Pro
fici
ent
or h
ighe
r
Year
High School Grade 8 Grade 5
High School MCAS Participation by Grade
Biology (June)
Chem Intro Physics
Tech/Eng Total
Gr. 9 23,079 322 15,903 1,614 50%
Gr. 10 36,752 1,940 1,077 637 50%
2008
2016 Biology (June)
Chem Intro Physics
Tech/Eng Total
Gr. 9 34,148 120 14,475 2,390 72%
Gr. 10 17,755 759 903 417 28%
Which High School STE MCAS Are Students Taking in 2016?
Biology 67%
Chemistry 0%
Physics 28%
Tech./Eng. 5%
Grade 9
Massachusetts Department of Elementary and Secondary Education
21
Biology 90%
Chemistry 4%
Physics 4%
Tech./Eng. 2%
Grade 10
Biology 73%
Chemistry 1%
Physics 22%
Tech./Eng. 4%
Combined
Overview of Item Development Takes 2-3 years to develop an item Work with contractor to develop item requests DESE and Teachers review new items If accepted, will be field tested the following
year DESE and Teachers review field-test data If item is accepted, can be common (count)
the year following field testing
22
MCAS STE Transition Timeline
STE MCAS Transition Timeline can be found at http://www.doe.mass.edu/news/news.aspx?id=23891
Takes 2-3 years to develop new items Grades 5 & 8 tests span grades: (3, 4, & 5) and
(6, 7, & 8) No “easy” transition For MCAS purposes, 3 years to fully transition to
new standards for Grades 5 & 8
23
Transition Timeline for Gr. 5 & 8 2017
based on 2001/06 STE standards will release list of standards to be tested for Gr. 5 & 8 in
November 2016 paper-based only
2018 based on overlapping standards from old and new
standards (will release list of “overlapping” standards) computer-based (tentative) based on current tests
2019 based on 2016 standards computer-based with new test design performance standards are set
24
Transition Timeline for High School MCAS STE Tests Considerations: Board of Elementary and Secondary Education
must approve high school plans Graduation Requirement Scholarship Requirement Number of students taking each test
Need time to transition grades below high school first
25
Proposed Transition Timeline for High School MCAS STE Tests 2017 & 2018
based on 2001/06 STE standards paper-based only
2019 based on overlapping standards from old and new
standards computer-based (tentative) based on current tests
2020 based on 2016 standards computer-based with new test design performance standards are set
26
Proposed Test Designs for “Next Gen” STE MCAS Similar timing and number of points to current
tests Grades 5 and 8:
2 sessions 38 items 54 points
High School: 2 sessions 42 items 60 points
27
Proposed Gr. 5 & 8 Reporting Categories
28
Content Area % Points
Earth and Space 25 13-14
Life 25 13-14
Physical 25 13-14
Tech/Eng 25 13-14
Proposed Practices Reporting Categories for All STE MCAS
29
Code Practice % Range
Gr. 5 & 8 Points
High School Points
A Scientific Questioning and Investigations
15-22 8-12 9-13
B Analyzing Scientific Data and Using Mathematics
15-22 8-12
9-13
C Scientific Modeling and Explanations
15-22 8-12
9-13
Note: Items will not be coded to the Argumentation or Obtaining and Communicating Information practices.
STE MCAS Item Types Current Item Types: 1 pt. multiple-choice (4
options) and 4 pt. open-response New Test Item Types:
Machine-scored 1 or 2 point items Multiple-choice, multiple-select, technology-
enhanced items (e.g., drag & drop, hot spots, select from list, etc.)
30
STE MCAS Item Types Hand-scored (constructed-response)
2 or 3 point items Fill-in/short answer (2 pt.) Open-response (3 pt.)
Modules: groups of items based on scenario/ phenomenon/model , includes: 1-pt. machine-scored items 3-pt. open-response item 2-pt. fill-in/short answer (high school only)
31
Proposed Gr. 5 and 8 STE MCAS Blueprint
Item Type Number of Items Number of Points
Module 2 (3 MS-1 pt, 1 OR)
12
Machine-scored, 1 pt. 20 20 Machine-scored, 2 pt. 6 12 Hand-scored, 2 pt. 2 4 Hand-scored (OR), 3 pt. 2 6 Total 38 54
32
Proposed High School STE MCAS Blueprint
Item Type Number of Items Number of Points
Module 3 (3 MS-1 pt, 1 MS-2 pt,
1 OR)
24
Machine-scored, 1 pt. 19 19 Machine-scored, 2 pt. 5 10 Hand-scored, 2 pt. 2 4 Hand-scored (OR), 3 pt. 1 3 Total 42 60 33
2017 Gr. 5 and 8 Tryouts Voluntary, does not count toward student
scores Computer-based only with new item types See how items perform
Review statistics with teachers
Release items to field for instruction Sign up coming soon—check DESE web site/
email/MSELA messages 34
Questions, Comments, and Contacts
Katie Bowler, Administrator for STE Test Development [email protected]
Brendan Harmon, STE Test Development Specialist [email protected]
Jass Stewart, Special Assistant to the Chief of Staff [email protected]
**Physics Test Developer Position is Open**
Massachusetts Department of Elementary and Secondary Education
35
