NCEOUniversity of Minnesota

Martha ThurlowDeb Albus

Laurene ChristensenYi-Chen Wu

Sheryl Lazarus

Search for state CBTs

• Searched state department of education Web sites late October/November, 2009

• Verified in February/March, 2010

4

Formative/Diagnostic

Regular (with Exit)

End of Course

Alternate based

on Modified Achievement

Standards

English Language

Prof. Test

# States

7 16 13 3 2

# Tests

8 24 14 3 2

Table 1. Number of States with CBTs Found by Type of Assessment

Table 2. Approaches of States Offering Paper Versions of Regular CBT

Any Accomm.Requires

Paper

Certain Accomm. Requires

Paper

Paper Version

is an Accomm.

Other Situations Needing

Paper

Anyone May Use

Paper Prohibited

TX CTFLORVA

KSOK

CTMDMN

UT

WV

INMD

None

# States

1 4 2 5 2 0

Table 3. Approaches of States Offering Paper Versions of Computer-based End of Course Tests

Any Accomm. Requires

Paper

Certain Accomm. Requires

Paper

Paper Versionis an

Accomm.

Other Situations Needing

Paper

Anyone May Use

Paper Prohibited

None OK LA MS

SDGAIN

MDMOSCTXVA

None

# States

0 1 1 2 7 0

Current Accommodations Policies

• Collecting policy information since 1992• Synthesis report every two years• Current data is from 2009• Does not specifically address computer-based

assessment• Goal is to describe what is, not to determine

whether or not states comply with NCLB/ESEA

Analysis Procedures• Obtain publicly available information from

state websites (all states plus DC)• Read the participation and accommodation

policies and enter data• Verify the data with states• Write the report• Data is made available through a searchable

online tool called the Data Viewer (http://data.nceo.info)

Relevant AccommodationsAccommodation State Policy

Computer or Machine Allowed=33Allowed with some qualification=13Prohibited=1

Communication Device Allowed= 21Allowed with some qualification=8Prohibited=0

Spell Checker/Assistance Allowed=8Allowed with some qualification=13Prohibited=16

Speech/Text Device Allowed=12Allowed with some qualification=14Prohibited=1

Assistive Technology Allowed=16Allowed with some qualification=13

Future Accommodations on Computer-based Platforms

• Read-aloud

• Sign interpretation

• Highlighting

• Masking/templates

• Refreshable Braille

• Calculator

• Manipulatives

• Magnification/large print

• Sound amplification

• Others?

Test Read Aloud Directions-Computer

The following information applies only to students assigned the Tests Read Aloud accommodation. Before beginning a test, ensure the following: ● MP3 files are not blocked or restricted on computers used for testing students with this accommodation. Consult the technology coordinator or network administrator, if necessary.● Headphones are available to students. Students may use personal headphones or ear buds. ● The mute setting is turned off.● The volume is set to a reasonable level so that other students cannot hear the audio.Although directions for playing the audio and controlling the volume appear on the instructions page at the beginning of each test session, it is important to make sure students understand these tools before testing. Use the information below to prepare students for testing.

Test Read Aloud Directions-Computer (continued)

Using the Audio Tool:● Play All, Stop, and Play/Pause buttons are available using the audio tool at the top of the screen (figure 4.2). To hear everything showing on the screen including directions, the test item(s), and answer options, click Play All. The Play/Pause button will alternate between Play and Pause when clicked. To pause, click Pause. To resume audio, click Play. To stop, click Stop. ● The Volume control bar is also located at the top of the screen. To raise the volume, slide the Volume bar to the right. To lower the volume, slide the Volume bar to the left. EOC Test Administration Manual, April/May 2011 21● Audio icons appear to the left of each part of a test item (figure 4.2). Click on the icon next to each part of the item to hear the audio file for that part.

Considerations for Technology-Based Testing Accommodations

1. Do technology-based assessments mean that students will no longer need accommodations?

2. What is the difference between accommodations, embedded features, and good testing practices?

3. Who makes decisions about what is used during technology-based assessments, and when are those decisions made?

4. What training is needed for teachers and students?

5. What should be tracked and documented within the technology-based platform?

Do technology-based assessments mean that students will no longer need

accommodations?

What is the difference between accommodations, embedded features, and

good testing practices?

DefinitionsAccommodations are changes in materials and

procedures designed to give access to students who need them, and to produce test scores with greater validity for those students.

Embedded Features are interactive tools that are part of the test platform and used to customize the assessment forindividual test takers.

Good Testing Practices are testing practices and procedures that should be available to all.

Who makes decisions about what is used during technology-based assessments, and

when are those decisions made?

Technology-based accommodations require thinking through logistics!

What training is needed for teachers and students?

What should be tracked and documented within the technology-based platform?

23

Background information• Oregon Assessment of Knowledge and Skills

(OAKS)• Students can choose they want to take paper or

online version• Every student in the state will have up to three

opportunities to take the test• One of the highest score will be marked as the

“Best score”• Document accommodation “Yes vs. No”

24

Research Questions• How many students with disabilities take a paper version of

the OAKS?• Did students with disabilities have less opportunity to take

online test compared to students without disabilities?• In which month did students with disabilities generally take

OAKS online and how does this compared to students without disabilities?

• How many students with disabilities used accommodations for OAKS online?

• What are the performance patterns of students with disabilities for OAKS online?

25

How many students with disabilities take a paper version of the OAKS?

• Very few students with disabilities took the paper version

• None of the students took the paper version more than once

26

Did students with disabilities have less opportunity to take online test?

• Reading

• No. SPED students have more opportunities to take the test than Non-SPED student

• Math

27

In which month did students generally take OAKS online?

One opportunity

• Results are similar across content area • SPED: more spread out from Nov. to April• Non-SPED : Feb-April

28

In which month did students generally take OAKS online?

• SPED

• 1st : Jan & Feb• 2nd : April & May

• Non-SPED

• 1st : November & Jan • 2nd : April & May

29

Two Opportunities

In which month did students generally take OAKS online?

• SPED

• 1st : Nov., Dec, & Jan.• 2nd : March & April• 3rd: May

• Non-SPED

• 1st : Oct. &Nov. • 2nd : March & April• 3rd: May

30

Three Opportunities

How many students with disabilities used accommodations for OAKS online?

• Reading

• Less than 20% of SPED received accommodations across different opportunities in Reading.

• Math

• Less than 19% of SPED received accommodations across different opportunities in Math.

31

What are the performance patterns of students with disabilities for OAKS online?

• Reading

• Most of students took the test once. • For students took the test twice, more than 65% of them got the best score on the

second attempt [Non-SPED: 26/(14+26); SPED:25/38].

32

What are the performance patterns of students with disabilities for OAKS online?

• Math

• Most of students took the test once. • For students took the test twice, more than 69% of them got the best score on the

second attempt [Non-SPED: 30/41; SPED:27/39].

33

Conclusions

• SWD have more opportunity to take online test

• SWD (one opportunity): were more spread out across months.

• Less than 20% of students with disabilities used accommodations for OAKS online.

• The performance patterns of students with disabilities for OAKS online is similar to students without disabilities.

Questions? Comments?

Presenter Contact Information:Martha Thurlow thurl001@umn.eduDeb Albus albus001@umn.eduLaurene Christensen chri1010@umn.eduYi-Chen Wu

wuxx0207@umn.eduSheryl Lazarus laza0019@umn.edu

.