Smarter Balanced: On Track and Moving Forward

Jacqueline E. King, Ph.D.

National Association of State Boards of EducationMarch 11, 2014

Agenda

• A Quick Refresher on Smarter Balanced

• Progress in Developing the Assessment System

Field Test

Setting Performance Standards

• Addressing Key Issues:TechnologyStudent PrivacyCostSustainability

Smarter Balanced Assessment Consortium

• 24 states & territories (22 governing, 1 advisory, 1 affiliate)

• K-12 & Higher Education Leads in each state

About Smarter Balanced

A Balanced Assessment System

Common Core State Standards specify

K-12 expectatio

ns for college and

career readiness

All students

leave high

school college

and career ready

Teachers and schools have information and tools

they need to improve

teaching and learning Interim

assessments Flexible, open,

used for actionable feedback

Summative assessments

Benchmarked to college and

career readiness

Teacher resources for

formative assessment

practicesto improve instruction

Describe Explain

Interpret

Level One(Recall)

Level Three(Strategic Thinking)

(Extended Thinking)

Level Two(Skill/

Concept)

Design

Synthesize

Connect

Apply Concepts

Critique

Analyze

Create

Prove

Arrange

Calculate

Draw

Repeat Tabulate

Recognize

Memorize

Identify

Who, What, When, Where, Why

List

Name

Use

Illustrate

Measure

Define

RecallMatch

Graph

Classify

Cause/Effect

Estimate

Compare

Relate

Infer

Categorize

Organize

Interpret

Predict

Modify

Summarize

ShowConstruct

Develop a Logical ArgumentAssessRevise

Apprise

Hypothesize

InvestigateCritique

Compare

Formulate Draw ConclusionsExplain

Differentiate

Use Concepts to SolveNon-Routine Problems

Level Four

Source: Webb, Norman L. and others, “Web Alignment Tool” 24 July 2005. Wisconsin Center of Educational Research, University of Wisconsin-Madison, 2 Feb 2006 6

Assessing the Common Core

Smarter Balanced assessments move beyond basic skills and recall to assess higher-order skills such as critical thinking and problem solving

Assessing College Readiness

Typical Readiness Testing Today

• Each college or system sets its own standards and selects its own measures.

• K-12 typically has no information about the standards.

• Students don’t know about tests and don’t prepare for them.

• Predictive validity of tests is often unknown.

• Students who “played by the rules” end up in remediation.

Smarter Balanced Vision

• Assessments designed around known, agreed-upon standards (Common Core).

• Performance standards set through an open process with substantial higher education involvement.

• Everyone (students, teachers, parents, etc.) knows the expectations.

• Students address deficiencies in high school.

Assessing Career Readiness

• Career Readiness Task Force developed recommendations and resource materials for score interpretation that are now available for public review at www.smarterbalanced.org

• Use of materials is voluntary and state customization is encouraged

• Academic readiness for careers = readiness for further career education and training

• Students who are not yet ready for entry-level general education courses can be ready for some postsecondary vocational/technical programs, so career-ready and college-ready are not equivalent.

Educator Involvement

Item Development & Review

• 500+ educators from member states

Digital Library of Formative Assessment Tools & Practices

• State Leadership Teams: 7 to 12 member steering committees• State Network of Educators: 70 to 150 members per state identifying and reviewing materials• 1,500 educators involved across consortium

Teacher Ambassadors

• 125 teachers drawn from leadership of NEA and AFT• Trained on Smarter Balanced so that they can keep their fellow teachers informed• Contributing to Digital Library

Accessibility & Accommodations

Note: Not all features are available for all grades or portions of the assessment. For detailed information, see the Smarter Balanced Usability, Accessibility, and Accommodations Guidelines.

Test Development Update

Assessment Development

Cognitive Labs and Small-scale Trials (2012-

13)

Field Test Item Development Complete

(December 2013)

Preliminary Standard-setting (September

2014)

Pilot Test: Approximately 650,000 students at more than 5,000 schoolsField Test: Expect more than 3 million students at more than 20,000 schools

Field Testing: A Practice Run of New

Assessments• Ensures fair and accurate assessments for

all students in spring 2015• Gives teachers and schools chance to

practice test administration procedures and gauge tech readiness

• Gives students opportunity to experience the new assessments

• No individual or school scores/results will be reported or shared

• Dates: March 18 to June 6

• All 22 Governing States and USVI

• At least 3 million students (research sample = 1.9M)

– Several states (CA, CT, ID, MT, SD) testing most or all students

– More than 20,000 schools in more than 4,000 districts

• 21,000 questions/performance tasks in English and math, grades 3-8 and 11

Field Test Facts

Resources to Help Schools Prepare

• Practice Tests Available since May 2013 Both subject areas, grades 3 through 8 and 11 Approx. 23 items & 1 performance task in each subject Uses same software as operational test Many accessibility and accommodation resources

• Training Test Now Available Quick Introduction to Smarter Balanced interface, item types, and

resources 14 items combined in English and math (all item types

represented) Grade bands (3 to 5, 6 to 8, and high school) Most accessibility and accommodation resources

• Also On-line: All test design documents: blueprints, content specifications,

item specifications Technology specifications and Bandwidth Checker Test Administration Manual and Usability, Accessibility, and

Accommodations Guidelines Training modules on an array of topics

Common Performance Standards: A Reminder of How We Got Here

Grade 4 Reading State Proficiency Standards Compared to NAEP, 2009

Setting Performance Standards

• Unprecedented in the United States for K-12 accountability assessments to share common performance standards

• Will require consensus among member states• In order to provide information to states for 2014-

15 interim & summative assessments, a very tight timeframe:– April: States vote on standard setting design– Summer: Score and calibrate field test data,

prepare for standard-setting– September:

• Distributed Standard Setting (up to 250K participants)

• Standard Setting Workshop (~500 participants)

• State Vote (Chiefs—w/ Higher Education for Gr 11)

– As Required, State Board Approval

Addressing Key Issues

Technology Requirements Responsive to School Needs

• Standards released last year for new and existing hardware & software, updated to reflect new products

• Schools do NOT need one-to-one computers Technology standards maximize access to online testing

(support for older machines and operating systems, small file size to reduce bandwidth requirements)

School with 600 students could test online with a single 30-computer lab

Smarter Balanced Tech Readiness Calculator allows schools to estimate number of testing days and associated bandwidth

• Pencil-and-paper option available for three-year transition period

• Computer-adaptive format allows long and flexible test administration window without sacrificing security

Safeguarding Student Privacy

• States endorsed principle that each state will retain control of its student data.

– Smarter Balanced will share no student-level information with the federal government.

– Smarter Balanced will not sell student data.

– PARCC and Smarter chiefs recently sent letter to Secretary Duncan affirming that consortia will not change state reporting practices.

• Each state will enter data privacy and security agreements to safeguard student data.

Student Privacy (cont.)

• Smarter Balanced will collect only the following information: An identification number (recommended that this number

be different from the state’s official unique student identifier, so only the state can tie back to the student’s official education record)

Race/ethnicity, gender, grade level, school attended Eligibility for English language development services or

special education services; Eligibility for Title I compensatory programs; and Smarter Balanced test scores, achievement levels, and

responses to test items.

• Smarter Balanced will NOT require states to report student name or date of birth.

• Smarter Balanced will never sell student information to anyone.

Anticipating and Managing Costs

• States now spend an average of $35 per student (0.3% of per pupil expenditures).

• Two components of cost: Smarter Balanced Membership Fee for member-

governed maintenance and continuous improvement of the assessment system

An estimate of costs for member-managed activities, including test administration, scoring, and training

• Pricing is per student participant in the summative assessment, with the total number of students capped at 1 million per state.

• Cost estimates below current expenditures for two-thirds of Smarter Balanced states.

Price Structure

Smarter Balanced Membership Fee

Member-Managed Activities Estimate Total

Complete System (Interim and Formative Tools Included for Grades K through 12)

Summative Assessment Options

Grades 3 through 8 and 11 $9.55 $17.75 $27.30

Grades 3 through 8 only $10.10 $17.75 $27.85

Basic System (Summative Only -- Interim and Formative Tools Excluded)

Summative Assessment Options

Grades 3 through 8 and 11 $6.20 $16.30 $22.50

Grades 3 through 8 only $6.20 $16.30 $22.50

Optional Summative Assessments for Grades 9, 10, or 12

Grades 9, 10, or 12 $6.20 $16.30 $22.50

Sustaining Smarter Balanced

• Working with UCLA Graduate School of Education and Information Studies to serve as host and partner for a sustainable Smarter Balanced.

• Key principles: Retain state led governance of the

Consortium (only minor changes to governance structure envisioned).

Shared state ownership of the item pool, digital library, and other IP.

Smarter Balanced will perform services necessary to maintain quality and comparability of the assessment system; states and their vendors will manage test administration.

Learn More and Stay Engaged

• Visit SmarterBalanced.org for the latest news and developments

• Sign up for the e-newsletter

• Follow on Twitter: @SmarterBalanced