Iowa Assessment Update

School Administrators of IowaNovember 2013Catherine Welch

Iowa Testing Programs

Reporting Empirically validated

Standardization Research based

Past -- Emphasis

Guided by mandate to be linked to previous forms for accountability reporting (A, B, E, F)

Guided by desire to inform instruction Emphasis on level of information in reporting structure Emphasis on reliable reporting

Guided by the quality of the information provided to users Emphasis on measuring growth Emphasis on college and career readiness

Present – Forms E and F

Forms E and F – Test Similarities

Balance of Concepts, Applications and ProceduresRange of Cognitive Demands

Aligned to Common Core Domains Connects Practice to Content

Forms E and F – Technical Similarities

Vertical Scale to Measure GrowthAllows Direct Comparisons to Previous FormsParallel Forms allow the Tracking of Students

Addresses NCLB RequirementsValid, Reliable and Technically Sound

Form F -- Enhancements

Alignment• Domain coverage

at the same level• Expanded

standards covered

Format• Paired Passages• Additional

informational texts

Rigor• Using evidence

from texts in social studies and science

• Assessing research and inquiry across all areas

Current Results

• State Performance• Growth• Use of Information

Reading Results – 2012-2013

3 4 5 6 7 8 9 10 11160

180

200

220

240

260

280

300

Grade

Stan

dard

Sco

re

Mathematics Results – 2012-2013

3 4 5 6 7 8 9 10 11160

180

200

220

240

260

280

300

Grade

Stan

dard

Sco

re

Compared to Nation

Growth in Reading

Year 12011-2012

Year 22012-2013 Growth Expected

Growth3rd to 4th 188 204 16 15

4th to 5th 203 219 16 14

5th to 6th 218 227 9 13

6th to 7th 226 242 16 12

7th to 8th 241 253 12 11

8th to 9th 253 273 20 10

9th to 10th 273 284 11 8

10th to 11th 284 290 6 7

Growth in Mathematics

Year 12011-2012

Year 22012-2013 Growth Expected

Growth3rd to 4th 190 204 14 15

4th to 5th 203 218 15 14

5th to 6th 217 228 11 13

6th to 7th 227 243 16 12

7th to 8th 243 256 13 11

8th to 9th 256 275 19 10

9th to 10th 275 285 10 8

10th to 11th 284 291 7 7

G3 G4 G5 G6 G7 G840

45

50

55

60

65

Math scores by level of STEM Career Interest

Very InterestedSomewhat InterestedNot Interested

Grade Level

NPR

G3 G4 G5 G6 G7 G840

45

50

55

60

65

70

Science scores by level of STEM Career In-terest

Very InterestedSomewhat InterestedNot Interested

Grade Level

NPR

Relationship between STEM Interest and Achievement

Future

Next Generation Iowa AssessmentTest Component Description

Design and Development

Aligned to Iowa CoreVariety of item types (essay, constructed-response, technology-enhanced, multiple-choice)Developed by Iowa educatorsField tested on Iowa students

Delivery Online and Paper/pencilComparable scores between modes

Scoring and ReportingOnline reporting systemAutomated scoring engine where applicableHuman scoring where applicable

Additional ComponentsCustomizable to state of Iowa needsGrowth measuresCollege and career readiness indicatorsComparable scores over time

Timeline for Next Generation Iowa Assessment

SchoolYear

Design and Development Delivery Scoring and

ReportingRelated

Research

2012-13

Item designItem writingField testing

Comparability research

Online reporting development

Growth research

2013-14Technology-enhanced platformOnline pilot

Online growth reporting

Career and college-ready research

2014-15Online option available for Iowa Assessments

Online reporting available

Validity studiesLinkingScaling

2015-16Online option available for Iowa Assessments

Online reporting available

Psychometric studies

2016-17 New assessment implementation

Information ProvidedELA Math Other

Iowa Assessment

• Claim scores• Domain scores• Proficiency levels• National

comparisons• Growth indicators• College readiness

• Claim scores• Domain scores• Proficiency levels• National

comparisons• Growth indicators• College readiness

• Social Studies

• Science Skills

Smarter Balanced Assessment Consortium

• Claim scores• Proficiency levels• Comparison to

SBAC states

• Claim scores• Proficiency levels• Comparison to

SBAC states

• NA

Considerations for Moving Forward

Considerations for Moving Forward

• Evaluate Purpose of Assessments• Evaluate Products to Match Purpose• Make Appropriate Comparisons• Define Criteria for Evaluation from House File 215– aligned with the Iowa common core standards in both

content and rigor; – accurately describes student achievement and growth for

purposes of the school, the school district, and state accountability systems; and

– provides valid, reliable, and fair measures of student progress toward college or career readiness

Evaluate Purpose

• Accountability– Administrator– Teacher– Student

• Summative• Formative• Interim

Evaluate Products and Reports to Match Purpose

• Review the items and tests• Review the score reports• If adaptive, review the algorithms for adaptive testing– For example, understand how high-level 4rd graders will be

tested and the implications for alignment– Understand how low-level 8th graders will be tested and

the implications for comparisons across students• If fixed form, review the implications for the balance

and selection of content• Compare level of reporting to requirements

Make Appropriate Comparisons

Define Criteria for Evaluation

• Aligned• Reliable• Valid• Growth • Readiness• Technology• Costs

• How much is enough?• Who are the experts to

evaluate these criteria?• What are the

compromises to be made?

• How are these compromises tied to purpose?

165 170 175 180 185 190 195 200 205 210 215 220 225 230 235 240 245 250

National Average 201

State Average 204

State Proficiency 189

College Readiness 218

Range of District Averages (342 Districts)

Range of Individual Students(35,000 Students)

4th Grade Mathematics Performance

5th Grade Mathematics Performance

165 170 175 180 185 190 195 200 205 210 215 220 225 230 235 240 245 250

National Average 214

State Average 218

State Proficiency 200

College Readiness 236

Range of District Averages (342 Districts)

Range of Individual Students(35,000 Students)

Alignment

Alignment?

• “Aligned” by 15 trained experts• “Aligned” to 11 different standards• “Aligned” to 4 different grade levels• “Aligned” to 3 different domains

SBAC Alignment?

Alignment?

• “Aligned” by 7 trained experts• “Aligned” to 6 different standards• “Aligned” to 3 different grade levels• “Aligned” to 3 different domains

Define Criteria for Use

• What are the metrics to be reported?• Are they understood by parents?• Will teachers have enough information from

the results to shape their instruction?• Will teachers receive the responses from the

open-ended, constructed-response items?• Will teachers receive professional

development on the usefulness of the results?

Define Criteria for Technology

• Identify the vendor that will deliver the assessment

• Pilot test the types of items to be used• Pilot test in a variety of districts in the spring of

the year during the same 12-week window; during the busiest times of the day

• Pilot test using the vendor-developed online tools• Pilot test using the graphics, videos, that will be a

part of the assessment

Define Criteria for Administration

• Demand same time-of-year testing• Demand reports/information prior to the end

of the year• Demand reports that show growth, readiness,

achievement indicators and instructional-relevance