REGIONAL SMARTER BALANCED ASSESSMENT WORKSHOPSJanuary/February 2014

Introductions• Jennifer Judkins

• OSPI Mathematics Assessment, EOC• Jennifer.Judkins@k12.wa.us

• Anton Jackson • OSPI Mathematics Assessment, Grades 6 – 8• Anton.Jackson@k12.wa.us

• Julie Wagner • OSPI Mathematics Assessment, Grades 3 – 5• Julie.Wagner@k12.wa.us

• Kim Andersen• OSPI Collection of Evidence• Kim.Andersen@k12.wa.us

• Scott Killough• OSPI Collection of Evidence• Scott.Killough@k12.wa.us

Housekeeping• Be sure to sign-in.• Be on time when returning from breaks.• Use cell phones and personal electronic devices at

breaks.• Keep side conversations to a minimum.• Use the Parking Lot for non-item-writing questions.• Collect evaluations and provide clock-hour forms at end of

day.

Workshop Program

• Smarter Balanced Assessment System Overview• Shifts in the CCSS• Smarter Balanced Summative Assessment• Claims and the CCSS• Progressions• Smarter Balanced Resources• Claim 1 Item Writing• Claim 2-4 Item Review

Purposes and Goals of Workshop• Increase understanding of the Smarter Balanced

assessment system• Increase understanding of Smarter Balanced claims, item

changes, and how the shifts in the CCSS manifest in items

• Observe how the progression of learning is assessed through the years

• Bring awareness to the Smarter Balanced resources that are available to educators

• Provide participants with knowledge of the item-writing process

5

SMARTER-BALANCED ASSESSMENT SYSTEMFormative Resources

Interim Assessments

Summative Assessments

A Balanced Assessment SystemPage 7

Common Core State

Standards specify

K-12 expectations for college

and career

readiness

All students

leave high

school college

and career ready

Teachers and schools

have information and tools

they need to improve teaching

and learning

Summative: College and career

readiness assessments for

accountability

Interim: Flexible and open

assessments, used for actionable

feedback

Formative resources:

Digital Library with instructional and

professional learning resources for

educators to improve instruction

FORMATIVE RESOURCESDigital Library with formative assessment instructional and professional learning resources for educators

Digital Library Leadership Structures

National Advisory Panel (NAP)

• 11-20 experts• Began December

2012• Provided policies and

criteria for resources

State Leadership Team (SLT)• 5- 8 members• Recruiting began

November 2012• Provides training for

State Network of Educators

State Network of Educators (SNE)

• 70 to 100 members• Representation from

LEAs, AEAs, content leaders, ELL, IHE

• Began recruiting in January 2013

• Provides reviews and resources

9

Resources in the Digital Library

10

• Commissioned Professional Learning Modules• Resources for students and families• Frame the formative assessment process within a Balanced

Assessment System• Articulate the formative assessment process• Highlight formative assessment practices and strategies

Assessment Literacy Modules

• Commissioned Professional Learning Modules• Instructional materials for educators • Instructional materials for students• Demonstrate/support effective implementation of the formative

assessment process• Focus on key content and practice from the Common Core State

Standards for Mathematics and English Language Arts

Exemplar Instructional Modules

• High quality vetted resources and tools for students and families• Reflect and support the formative assessment process• Reflect and support the Common Core State Standards for English

Language Arts/Literacy and Mathematics• Create Professional Learning Communities

Education Resources

* Resources include the following file types: Video, HTML5, Audio, PPT, Excel, Word, and PDF.

Digital Library Functionality

11

Enable trained educators (SNEs and SLTs) to submit, review, and publish resources Allow educators to view, download, and rate resourcesUse state-of-the-art tagging and search to quickly find resources by CCSS and other topicsEnable educators from across the Consortium to collaborate and share their knowledge

Resource Submission Process

SNE1

Submit Resource

withCover Profile

SNE1

SNE

Choose Resource to Review in SNE’s Queue

SNE

Initial Review of Resource with Gate-Keeping Criteria

SNE

Review of Resource with Quality Criteria

RubricReviewed by 3 SNEs

Resource Passes Gate-Keeping

Resource Fails Gate-Keeping—Return to

SNE to Revise

SNE SNE

SNE 1:The SNE who reviews the resource with the Gate-Keeping Criteria will also be one of the

3 SNE reviewers.

INTERIM ASSESSMENTSICAs and IABs

Interim AssessmentTwo Types:

Interim Comprehensive Assessments (ICA)• Fixed form in fall 2014 for all grades• Adaptive (as pool allows)• Will be taken online in same form as summative• Some level of hand-scoring

Interim Assessment Blocks (IAB)• Fixed form for most blocks in all grades• Adaptive for certain blocks (as pool allows)• Adaptive for all blocks – later• Will be taken online in same form as summative• Some level of hand-scoring

IABs for math: Current Thinking on CoverageProposed Interim Assessment Blocks

(IAB)Grades 3-5

1 Operations and Algebraic Thinking

2Numbers and Operations in Base Ten

3 Fractions

4 Geometry

5 MeasurementProposed Interim Assessment Blocks

(IAB)Grades 6-7

1Ratios and Proportional Relationships

2 Number System

3 Expressions and Equations

4 Geometry

5 Statistics and Probability

Proposed Interim Assessment Blocks (IAB)

Grade 8

1 Expressions and Equations - I

2 Expressions and Equations - II

3 Functions

4 Geometry

Page 15

IABs for math: Current Thinking on CoverageProposed Interim Assessment Blocks (IAB)

High SchoolAlgebra and Functions

1 Linear Functions2 Quadratics3 Exponentials4 Polynomials5 Radicals6 Rationals7 Trigonometric

Geometry8 Transformations9 Right Triangle Ratios

10 Three-Dimensional Geometry11 Proofs12 Circles13 Applications

Other14 Statistics and Probability

Page 16

SUMMATIVE ASSESSMENT

Assessment Claims

Test Map

Computer Adaptive Testing (CAT)

Item Types

Performance Tasks

Smarter Balanced Claims

• Claims are the broad statements of the assessment system‘s learning outcomes, each of which requires evidence that articulates the types of data/observations that will support interpretations of competence towards achievement of the claims.

Overall Assessment Claims

• Claim for Grades 3-8:

• “Students can demonstrate progress toward college and career readiness in mathematics.”

• Claim for Grade 11:

• “Students can demonstrate college and career readiness in mathematics.”

Assessment Claims 1- 4• Claim 1 – Concepts & Procedures• Claim 2 – Problem Solving• Claim 3 – Communicating Reasoning• Claim 4 – Modeling and Data Analysis

The claims are supported by the Standards for Mathematical Practice.

Major and Supporting/Additional Clusters

• http://www.k12.wa.us/CoreStandards/Mathematics/default.aspx

Test Map Example – Grades 6-8• http://

www.smarterbalanced.org/wordpress/wp-content/uploads/2011/12/Smarter-Balanced-Preliminary-Test-Blueprints.pdf

Estimated Testing Times for 2014 Field Test and 2015 Operational Assessment

Test TypeGrade

s CAT

Perf. Task Only Total

Class Activity Total

English Language

Arts

3-5 1:30 2:00 3:30 :30 4:00

6-8 1:30 2:00 3:30 :30 4:00

11 2:00 2:00 4:00 :30 4:30

Mathematics

3-5 1:30 1:00 2:30 :30 3:00

6-8 2:00 1:00 3:00 :30 3:30

11 2:00 1:30 3:30 :30 4:00

COMBINED

3-5 3:00 3:00 6:00 1:00 7:00

6-8 3:30 3:00 6:30 1:00 7:30

11 4:00 3:30 7:30 1:00 8:30

Times are estimates of test length for most students. Smarter Balanced assessments are designed as untimed tests; some students may need and should be afforded more time than shown in this table.

Computer Adaptive Testing

• “Based on student responses, the computer program adjusts the difficulty of questions throughout the assessment. … By adapting to the student as the assessment is taking place, these assessments present an individually tailored set of questions to each student and can quickly identify which skills students have mastered.”

Smarter Balanced

Summative Assessment Parts• Computer Adaptive Testing (CAT)

• Item Types• Single response multiple choice, Multiple response multiple

choice, Equation/Numeric• Matching Tables, Drag and Drop, Hot Spot, Fill-in Table

• Scoring• Machine Scored

• Performance Tasks• Item Types

• Same as CAT• Text Response

• Scoring• Machine Scored• Hand Scored

KEY CCSS SHIFTSFocus

Coherence

Rigor

Shift One: Focus strongly where the Standards focus

• Significantly narrow the scope of content and deepen how time and energy is spent in the math classroom

• Focus deeply only on what is emphasized in the standards, so that students gain strong foundations

Mathematics topics intended at each grade by at least two-thirds of A+ countries

Mathematics topics intended at each grade by at least two-thirds of 21 U.S. states

The Shape of Math in A+ Countries

1 Schmidt, Houang, & Cogan, “A Coherent Curriculum: The Case of Mathematics.” (2002).

Focusing attention within ‘Number and Operations’

Operations and Algebraic Thinking

Expressions and Equations

Algebra

Number and Operations—Base Ten

The Number System

Number and Operations—Fractions

K 1 2 3 4 5 6 7 8 High School

Shift Two: Coherence Think across grades, and link to major topics within grades

• Carefully connect the learning within and across grades so that students can build new understanding onto foundations built in previous years.

• Begin to count on solid conceptual understanding of core content and build on it. Each standard is not a new event, but an extension of previous learning.

Informing Grades 1-6 Mathematics Standards Development: What Can Be Learned from High-Performing Hong Kong, Singapore, and Korea? American Institutes for Research (2009, p. 13)

4.NF.4. Apply and extend previous understandings of multiplication to multiply a fraction by a whole number.

5.NF.4. Apply and extend previous understandings of multiplication to multiply a fraction or whole number by a fraction.

5.NF.7. Apply and extend previous understandings of division to divide unit fractions by whole numbers and whole numbers by unit fractions.

6.NS. Apply and extend previous understandings of multiplication and division to divide fractions by fractions.

6.NS.1. Interpret and compute quotients of fractions, and solve word problems involving division of fractions by fractions, e.g., by using visual fraction models and equations to represent the problem.

Grade 4

Grade 5

Grade 6

CCSS

Coherence Example: Grade 3

The standards make explicit connections at a single grade

Properties of Operations

Area

Multiplication and Division 3.OA.5

3.MD.7c3.MD.7a

Shift Three: RigorEqual intensity in conceptual understanding, procedural skill/fluency, and application

• The CCSSM require a balance of:• Solid conceptual understanding• Procedural skill and fluency• Application of skills in problem solving situations

• This requires equal intensity in time, activities, and resources in pursuit of all three

(a) Conceptual Understanding...or is it?

Place Value Concepts

• 206 = ____ hundreds ____ tens ____ones• 206 = ____ tens ____ ones• 206 = ____ ones• 347 = ____ tens ____ ones• 452 = 3 hundreds ____ tens ____ones• 521 = ____ hundreds 22 tens ____ ones

Is the comparison true or false?• 12 tens + 11 ones > 1 hundred + 2 tens

(b) Fluency

• The standards require speed and accuracy in calculation.

• Teachers structure class time and/or homework time for students to practice core functions such as solving equations so that they are more able to understand and manipulate more complex concepts.

• Select all of the equations that have integer solutions:

• A) = x - 4• B) x - = • C) - 4 = x• D) = 25• E) 2x =

Fluency Beyond the Facts

(c) Application

• Students can use appropriate concepts and procedures for application even when not prompted to do so

• Provide opportunities at all grade levels for students to apply math concepts in “real world” situations, recognizing this means different things in K-5, 6-8, and HS

• Teachers in content areas outside of math, particularly science, ensure that students are using grade-level-appropriate math to make meaning of and access science content

Application

Operationalizing the CCSS Shifts

Do:

Read the standards including critical areas of focus. Know the structure of the standards.

Know the major, supporting and additional clusters for your grade.

Study the progression documents with colleagues.

Think about unit design first, then lesson, then task. It is not about what you are going to ‘do’ but what the students will learn.

Activity

• Turn to your elbow partner:• Explain in your own words what the three shifts are in the CCSS.

• Explain to your partner how these shifts will change your instructional practice.

THE SMARTER BALANCED SUMMATIVE ASSESSMENTConsiderations

Claims

Item Types

Performance Tasks

CONSIDERATIONS

Six Key Concepts of Evidence-Centered Design

6. Develop Items or Performance Tasks

1. Define the domain Common Core Standards Math

2. Define claims to be made 4 Math ClaimsContent Specifications

3. Define assessment targets Knowledge, Skills, & Abilities

4. Define evidence required Evidence to be Elicited from Student

5. Develop Task Models Methods for Eliciting Evidence

Universally Designed Items and Tasks – Some Guidelines

• Simplify vocabulary and sentence structure

• Place problems in familiar contexts with familiar items

and names

• Use graphics only when necessary

• Simplify graphics and tables

• Avoid the use of color and maximize contrast

• Avoid content that spans multi-pages or that contains

multiple-elements

• Use highly interactive manipulations only when necessary

• Identify terms and visuals that should not be altered

Bias Concerns

• Bias occurs when content contained in an item or task creates an unfair disadvantage for a sub-group of students• Unfamiliar contexts or examples

• Unusual names of people or places

• References to local events or issues

Jimmy is the star on his football team. This season he scored ten touchdowns, kicked twenty extra points, and had eight field goals.

What is the total number of points Jimmy has scored this season?

What is this question

asking me?

What is a field goal?104

Sensitivity

• Sensitivity focuses on content that creates unease, provokes negative feelings, or challenges beliefs or values

• Religions, religious practices, and religious figures

• Political topics

• Issues of gender, race, and ethnicity

THE CLAIMS

Assessment Claims• Claim 1 – Concepts & Procedures

• “Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency.”

• Claim 2 – Problem Solving• “Students can solve a range of complex well-posed problems in pure

and applied mathematics, making productive use of knowledge and problem solving strategies.”

• Claim 3 – Communicating Reasoning• “Students can clearly and precisely construct viable arguments to

support their own reasoning and to critique the reasoning of others.”

• Claim 4 – Modeling and Data Analysis• “Students can analyze complex, real-world scenarios and can

construct and use mathematical models to interpret and solve problems.”

Math Practices in the Claims

• Claim 1 – MP 4, 5, 6, 7, 8

• Claim 2 – MP 1, 5, 7, 8

• Claim 3 – MP 3, 6

• Claim 4 – MP 2, 4, 5

Comparing the Claims - Activity

• Individually review the items on the ‘Comparing the Claims’ Worksheet.

• Review with a partner the items by claim number.

What are the differences in the way the problems are written by claim?

How does the focus on four claims change the way you instruct and assess students in the classroom?

ITEM TYPES

CAT Item Types

• Multiple Choice• Single correct answer• Multiple correct answers

• Equation/Numeric• Matching Table• Fill-in Table• Grid Items

• Drag and Drop• Graphing• Hot Spot

Multiple Choice – single correct response

Example Stem 1 (VE): Which model shows 18 of the whole figure shaded?

A.

B.

C.

D.

Grade 3

C)

D)

Multiple Choice – multiple correct responses

A)

B)

Select all the graphs that show a proportional relationship between x and y.

Grade 8

C)

D)

Equation/Numeric

A triangular-shaped garden is shown.

Enter the area of the garden, in square meters.

Grade 5

Equation/Numeric

Matching TableGrade 11

Fill- in Table

Drag and Drop

Kayla asked 10 students in her class whether they owned a dog or a cat or both.Drag one number into each box to complete the table, given this information: 40% of the students own a dog.30% of the students own a cat.10% of the students own both a dog and a cat.

Grade 8

Drag and Drop

Graphing

Grade 8

Hot SpotGrade 11

Item Type Implications

Review the ‘Item Type’ worksheet and talk to a partner about these questions. Be prepared to share your discussion:

• What implications do these item types have, if any, on your instruction and assessment of students?

• What resources do you have that might allow you to create similar items and imbed them in instruction and assessment?• N-rich• Virtual manipulatives

PERFORMANCE TASKS

Performance Tasks• 6 questions

• Distributed among Claim 2, 3, and 4• Item types on CAT may be used in a PT

• Also included short, constructed response:• “Show your work.”• “Explain your reasoning.”

• Classroom activity to precede work on PTs• 2 items automatically scored, rest hand-

scored

Classroom Activity

Sample TaskGrade 3

Culminating Question

Performance Task Implications

Talk to a partner about these questions and be prepared to share your discussion:

• How would you prepare students for sustained, connected activities like the performance tasks?

• What resources do you have for authentic, real-world applications of the mathematics students are learning?• Illustrative Mathematics• Engage NY

PROGRESSION OF LEARNING

Progression Documents • http://ime.math.arizona.edu/progressions/

Progression of Learning and Items through the Grades

Individually review the items in the progressions document.

Find an example where what is learned in a prior year is fundamental to what is learned later.

With a partner, determine the new learning that is added from year to year.

Be prepared to share.

SMARTER BALANCED RESOURCES

Smarter Balanced Website

For Educators• Field Test• Practice and Pilot Tests• Sample Items and Performance Tasks• Item Writing and Review

• Content Specifications• Item Specifications• Considerations, Style Guide, etc.

• Achievement Level Descriptors and College Content Readiness

CLAIM 1 ITEM WRITINGDepth of Knowledge

Item Specification Documents

DEPTH OF KNOWLEDGE

Cognitive Rigor and Depth of Knowledge

• Level of complexity of cognitive demand • Level 1: Recall and Reproduction

• Requires eliciting information such as a fact, definition, term, or a simple procedure, as well as performing a simple algorithm or applying a formula.

• Level 2: Basic Skills and Concepts• Requires the engagement of some mental processing beyond

a recall of information.

• Level 3: Strategic Thinking and Reasoning• Requires reasoning, planning, using evidence, and explanations

of thinking.

• Level 4: Extended Thinking• Requires complex reasoning, planning, developing, and

thinking most likely over an extended period of time.

Level 1 – Recall and ReproductionSimplify each expression.

74

7–3

1

3

2 1

3

9

(–5)6

(–5)10

87 8–12 ·

·

Level 2 – Application of Skills/ Concepts

A cylindrical tank has a height of 10 feet and a radius of 4 feet. Jane fills this tank with water at a rate of 8 cubic feet per minute. How many minutes will it take Jane to completely fill the tank without overflowing at this rate? Round your answer to the nearest minute.

Level 3 – Strategic Thinking andReasoningThe total cost for an order of shirts from a company consists of the cost for each shirt plus a one-time design fee. The cost for each shirt is the same no matter how many shirts are ordered.

The company provides the following examples to customers to help them estimate the total cost for an order of shirts.

• 50 shirts cost $349.50

• 500 shirts cost $2370 Part A: Using the examples provided, what is the cost for each shirt, not including the one-time design fee? Explain how you found your answer. Part B: What is the cost of the one-time design fee? Explain how you found your answer.

Level 4 – Extended Thinking

During the task, the student assumes the role of an

architect who is responsible for designing the best plan for

a park with area and financial restraints.

The student completes tasks in which he/she compares the

costs of different bids, determines what facilities should be

given priority in the park, and then develops a scale

drawing of the best design for the park and an explanation

of the choices made.

This investigation is done in class using a calculator, an

applet to construct the scale drawing, and a spreadsheet.

Depth of Knowledge

• Look at the ‘Comparing the Claims’ and determine the Depth of Knowledge level of each.

• Be prepared to share.

ITEM SPECIFICATIONSHow to read these documents

Terminology for CCSS

Cluster LabelingA

B

C

D

E

4.OA.A

4.OA.B

4.OA.C

4.NBT.A

4.NBT.B

Claim 1 Assessment Targets

Targets by Grade Level

Grade Level Assessment Targets Documents

3 A - K 114 A - L 125 A - K 116 A - J 107 A - I 98 A - J 1011 A - P 16

Item Specification Overview

• As each section of an item specification document is reviewed, find the same section in an item specification document at your grade-band level.• Grade 3-5 – Target G, Grade 4

• Grade 6-8 – Target A, Grade 6

• Grade 11 – Target M, Grade 11

Description of the Target

Targeted Standards

Relevant above and

below grade

standards

Achievement Level

Descriptors

Depth of Knowledge

Statements of Required

Evidence (Derived

from Standards)

Allowable Response

TypesAllowable Tools

Allowable Stimulus

Materials and Mathematics Vocabulary

Task Models Flow from Evidence Statements

94

Details about stimuli

for task model

Required evidence is identified

Task Models Flow from Evidence Statements

95

Stem asks for the required evidence

Stimulus gives you, the item

writer, information

Context should be changed from item

to item

Task Models Flow from Evidence Statements

Use the same stem language

from item to item

Numbers will change from item

to item

Procedures that Support Validity

• Item Specifications documents• Start with a clear explanation of what students are expected to know and be able to do

• Control irrelevant sources of variability by:• Standardizing items using Task Models.• Standardizing items using a Style Guide.

97

Claim 1 Activity

• Choose a task model from your item specification document and write an item to meet that task model.

• Share and review an item with a partner.

• How can you use this process to write items for your classroom or the district?

CLAIM 2 - 4 ITEM REVIEW

Claim 2 - 4 Activity

• Go through the “Claims 2 – 4 Sample Items” document. Identify characteristics of these sample items:• What are distinguishing features of items in each claim?• How are the questions asked differently in each claim?• How are claim 2 – 4 items different than Claim 1 items?

• Share your thoughts with a partner.

• Be prepared to share what you have learned about Claim 2 – 4 items.

Targets for Claim 2• Target A: Apply mathematics to solve well-posed

problems in pure mathematics and arising in everyday life, society, and the workplace.

• Target B: Select and use appropriate tools strategically.

• Target C: Interpret results in the context of a situation.

• Target D: Identify important quantities in a practical situation and map their relationships (e.g., using diagrams, two-way tables, graphs, flowcharts, or formulas).

Targets for Claim 3

• Target A: Test propositions or conjectures with specific examples.

• Target B: Construct, autonomously, chains of reasoning that will justify or refute propositions or conjectures.

• Target C: State logical assumptions being used.

• Target D: Use the technique of breaking an argument into cases.

• Target E: Distinguish correct logic or reasoning from that which is flawed and – if there is a flaw in the argument – explain what it is.

• Target F: Base arguments on concrete referents such as objects, drawings, diagrams, and actions.

Targets for Claim 4• Target A: Apply mathematics to solve problems arising in

everyday life, society, and the workplace.

• Target B: Construct, autonomously, chains of reasoning to justify mathematical models used, interpretations made, and solutions proposed for a complex problem.

• Target C: State logical assumptions being used.

• Target D: Interpret results in the context of a situation.

Targets for Claim 4• Target E: Analyze the adequacy of and make

improvements to an existing model or develop a mathematical model of a real phenomenon.

• Target F: Identify important quantities in a practical situation and map their relationships.

• Target G: Identify, analyze, and synthesize relevant external resources to pose or solve problems.

Day 1 Wrap-Up

• What are the next steps you would do to use the information you learned today in your classroom, building and/or district?

• What role will you play in these next steps?

• Evaluations• Clock hours

THANK YOU

DAY 2 - ITEM WRITINGRegional Smarter Balanced Assessment Workshops

Day 2 Item Writing Agenda• Welcome Back• Forms and Paperwork• Group Norms• Item-Writing for the Common Core Assessment

• Purposes and Goals• Importance of Validity• Documents Used to Support Validity

• Claim 1 Item Writing Considerations• Procedure for Writing and Reviewing Items• Claim 1 Item Writing• Claim 4 Item Writing Considerations• Claim 4 Item Writing

108

Forms and Paperwork

• Sign - in• Non-Disclosure Agreement

• The items are secure, but the process is not.

• Substitute Reimbursement• Evaluation and Clock Hours

109

Public Disclosure• Please be aware that your participation on this committee is considered public record under Washington State Public Record Act RCW 42.56.

• Please be assured that your personal and residential information are not released.

110

Group Norms• Be on time when returning from breaks.• Use cell phones and personal electronic devices

at breaks.• Keep side conversations to a minimum.• Respect each other’s work styles (need for quiet).• Ask for and give feedback.• Use the Parking Lot for non-item-writing

questions.

111

Purposes and Goals of Workshop• Write assessment items that are aligned to the

Common Core State Standards for Mathematics• Increase understanding of Smarter Balanced item

specifications and other item-writing documents• Involve Washington educators in the assessment

process• Provide participants with knowledge to share item-

writing process with others and increase student achievement

112

CLAIM 1 ITEM WRITING CONSIDERATIONS

113

Mathematical PracticesMathematical Practices that support Claim 1:

• MP 4: Model with mathematics• MP 5: Use appropriate tools strategically.• MP 6: Attend to precision.• MP 7: Look for and make use of structure.• MP 8: Look for and express regularity in repeated reasoning.

Item Writing Considerations

Reviewed yesterday:• Alignment to CCSS-M Standards• Item Specifications and Task Models• Plain Language and Vocabulary• Bias and Sensitivity• Universal Design• Cognitive Complexity

Moodle PowerPoint:• Style Guide

115

Metadata Table – Claim 1

Achievement Quintiles

Five divisions are:• 1 - Very easy• 2 - Easy• 3 - Medium• 4 - Hard• 5 - Very hard

Maximum, Minimum Grade

• Claim 1 items will often be written to a specific grade level so both maximum and minimum grade will be the same. Some items where there is a direct building of skills from grade to grade may make the maximum or minimum grade one above or below.

Score PointsMaximum Points

• Score Points:• 0, 1 for a one-point item• 0, 1, 2, for a two-point item.

• Maximum Points:• 1 for a one-point item• 2 for a two-point item.

Depth of Knowledge• DOK 1—Recall; perform a rote procedure• DOK 2—Application of Skill/Concept• DOK 3—Strategic Thinking• DOK 4—Synthesizing information and

creating new example

• Most of Claim 1 items will be DOK 1 or 2 but watch what is written in the item specification documents.

• If multiple DOKs are listed, you must pick one.

Presentation Format

• T – Text • TG – Text with Graphics

Allowable Response Types

DAS

Multiple Choice, single correct response (MC)

Multiple Choice, multiple correct response (MS)

Matching Tables T/F and Y/N variations (MA)

ITS

Equation/Numeric (EQ)

Fill-in Table (TI)

Drag and Drop, Hot Spot, and Graphing (GI)

Lessons Learned from Certification Set

Successes• Identifying the Task Model• Following the Task Model

• Task models without context must be followed explicitly• Task models with context have a bit more flexibility

• Alternatives for MC and MS items• Rubric and Scoring information

Foci of Attention• Including only what you want the “student” to read in the item• Rationales for MC and MS alternatives

Rationales• MC and MS items need rationales for each

alternative• Use “Correct Answer” for correct answer(s)• Item spec documents usually give examples of reasonable

distractors (incorrect alternatives)• You can include other types of distractors

• Describe, in positive terms, or show work as to why students would choose a distractor:• The student added 3 + 4 before multiplying by 6• The student multiplied the exponents 4 and 7 instead of

adding the exponents• Student adds the leg lengths of the triangle and does not

square the lengths first

Questions?

Steps to Write Item• Work with a partner.• Choose a standard from the folders and take an item-specific sheet.

• Check item specification for item considerations for the given target.

• Check task model rules for item type, cognitive complexity, required evidence, tools

• Write item.• Follow Peer Review Process.

126

Peer Review Process

• Ask a pair of colleagues to work your item and make suggestions for edits.

• Submit for approval.• Specialist will review item and provide feedback to writers and peer reviewers, if appropriate.

• Specialist may ask for revisions/edits. Writers make edits and resubmit item to same specialist.

127

Let the writing begin!

Write 1 - 2 items in Claim 1

128

CLAIM 4 ITEM WRITING

Claim 4

• Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.

Modeling and Data Analysis

• Modeling is the process of choosing and using appropriate mathematics and statistics to analyze empirical situations, to understand them better, and to improve decision-making.

Smarter Balanced

Activity• Review the Claim 4 Document with your group.

• Front Matter• Assessment Targets• Sufficient Evidence for Claim 4• Task Models and Example Items

• Be prepared to speak to the differences between writing Claim 1 items and writing Claim 4 items.

Modeling Contexts/Situations Do we

• Brainstorm contexts/situations that are appropriate for your grade band

• Consider specific contexts from the designated standards for this claim

• Contexts/situations can include content from multiple domains.

Claim 4 Mathematical Practices

• Mathematical Practices that support Claim 4:

• MP 2 – Reason abstractly and quantitatively.• MP 4 – Model with mathematics.• MP 5 – Use appropriate tools strategically.

Secondary Claim

• Items written primarily to assess Claim 4 will necessarily involve some Claim 1 content targets.• The content should draw upon the knowledge and skills articulated in the progression of standards leading up to and including the targeted grade

• Related Claim 1 targets should be listed below the Claim 4 targets in the metadata form.

Primary and Secondary Standard/Cluster

While tasks developed to assess Claim 4 will have a primary content focus, components of these tasks will likely produce enough evidence for other content domains that a separate listing of these content domains needs to be included where appropriate.

Targets for Claim 4• Target A (TM1): Apply mathematics to solve problems

arising in everyday life, society, and the workplace.

• Target B (TM 2): Construct, autonomously, chains of reasoning to justify mathematical models used, interpretations made, and solutions proposed for a complex problem.

• Target C (TM3): State logical assumptions being used.

• Target D (TM 4): Interpret results in the context of a situation.

Targets for Claim 4• Target E (TM 5): Analyze the adequacy of and make

improvements to an existing model or develop a mathematical model of a real phenomenon.

• Target F (TM 6): Identify important quantities in a practical situation and map their relationships.

• Target G*: Identify, analyze, and synthesize relevant external resources to pose or solve problems.

* Measured in Performance Tasks only

Assessment Targets• Any given task should provide evidence for several of the

preceding assessment targets; each of the targets should not lead to a separate task. Multiple targets should be listed in order of prominence as related to the task.

Distinguishing Claim 4• Distinguishing between Claim 4 and Claims 1 and 2:• In early grades when equations are still new to students,

an important distinction between Claim 2 and Claim 4 is requiring a model that would lead to a problem's solution.

• In Claim 2, problems are well posed, while in Claim 4, they may have extraneous or missing information.

• In Claims 1 and 2, measurements of objects or figures can be accurately determined. In Claim 4, modeling is used to make approximations.

• In Claim 1, data analysis is straightforward and procedural. In Claim 4, the analysis should be tied to some useful purpose in the real world.

Claim 4 CCSS Standards

A Review of Metadata Fields• Mathematical Practices• Achievement Quintiles• Maximum, Minimum Grade• Score Points• Maximum Points• Depth of Knowledge• Presentation Formats• Response Type

Scoring

• Scoring rules and answer choices will focus on a student’s ability to use the appropriate reasoning. For some problems, multiple correct responses and/or strategies are possible.

• Items may be worth 1 or 2 points; Justification for more than 1 point must be clear in the scoring rules.

Lessons Learned from Certification Set

Successes• Identifying the Task Model• Following the Task Model

• Task models without context must be followed explicitly• Task models with context have a bit more flexibility

• Alternatives for MC and MS items• Rubric and Scoring information

Foci of Attention• Completing the metadata table• Including only “student” information in item• Rationales for MC and MS alternatives

Preparing to Write Claims 4 Items• Claim 4 document – Standards assessed• Claim 1 document – Development Notes• Context and Standard(s) assessing• Claim 4 document – Targets, Task Models, and Sample

Items• Item Type, Metadata, Scoring, Rubric

Claim 4 Items

• Start with these item types:• MC, MS, and EQ

• Do not copy examples or try to clone them.

• Items must be original and contain several standards that are in alignment with the chart for that grade level.

Item Writing Process• Review Claim 4 document carefully.• Work with a partner.• Decide to which target you will write.• Use the Claim 1 documents as a reference for how the problem is posed to assess Claim 1.

• Identify how the Claim 4 item you are writing differs from Claim 1 items.

Questions?

Debriefing Claim 4

• How was this writing different from Claim 1?

• The process for Claims 2 and 3 is similar. How might you prepare to write these types of items?

• What are your next steps in your classroom, building, or district?

Day 2 Item Writing Wrap-up• Parking Lot Questions/Comments• Feedback

• Suggestions to improve process.• Other questions?

• Evaluations• Clock hours• Substitute reimbursement form

150

THANK YOU