NAEP Alliance

NAEP Technology Based AssessmentsNational Conference on Student Assessment

June 22, 2015 2:00-3:30

2

NAEP Alliance

Introduction to eNAEP

3

NAEP Alliance

NAEP Model (2014 & beyond)

• Administration on NAEP-provided touch-enabled devices

• 2 sessions of 25 students per school• eNAEP delivery system

Web application Compatible across various devices, operating systems, and web

browsers (i.e., future-proofed) Can run on virtually any resolution (flexible layout)

4

NAEP Alliance

eNAEP Development

• eNAEP is built to support all NAEP subjects.• Provides a consistent user experience across all

subjects. • Planning for a progressive rollout of functionality

over the next few years:– 2014: Initial version (subset of item types & tools) – 2015: Enhanced version (more tools, features &

item types)– 2016: Additional item types – 2017: Additional interactive item types

5

NAEP Alliance

Summary of Planned eNAEP System Features

Features 2014 Proof of Concept 2015 TBA Startups & Pilots

Volume Yes Yes

Tutorials No Yes

In-System Help Yes Yes

Text-to-Speech No Yes

Zoom No Yes

Highlighter N/A* Yes

Elimination Tool N/A* Yes

Change Color Theme No TBD**

*There are no discrete or questionnaire items in 2014 POC, so these tools are not applicable.

**Some accommodations and potential universal design features require discussion by the AICC.

6

NAEP Alliance

Additional Features2015 & Beyond

Additional Features 2015 & beyond

Accommodations, e.g., bilingual, magnification

Universal Design features, e.g., text-to-speech, zoom

508 compliance

Observable data capture

Discrete item types and survey questions

Disconnected mode

Updates in the field supported

Subject-specific tools, e.g. calculator, equation editor

Adaptive testing TBD

Dynamic distribution of forms TBD

Automated scoring TBD

7

NAEP Alliance

NAEP 2015 TBA Start Up January 26 – March 6, 2015 132,000 students Administered on tablets Mathematics, reading, and science Grades 4, 8, and 12 Interactive computer tasks (ICTs) and hands-on

tasks (HOTs) in science

8

NAEP Alliance

Equipment Surface Pro 3 tablets Laptop server Routers and Ethernet cables MiFi device Extension cords, power strips, and tape Earbuds and headphones Styluses Flash drives Antibacterial wipes

9

NAEP Alliance

Surface Pro 3 Tablets

– 27 Tablets 25 Student tablets 2 Monitor tablets

10

NAEP Alliance

eNAEP Network Equipment Laptop server Router Blue Ethernet

cable

11

NAEP Alliance

MiFi Device Wireless device to send student response files from

the laptop server to the central server

12

NAEP Alliance

NAEP Earbuds NAEP students keep earbuds Headphones available when requested

13

NAEP Alliance

Styluses 30 Styluses Re-use for all assessments

14

NAEP Alliance

Flash Drives 2 Flash drives Used for disconnected mode assessments

15

NAEP Alliance

2015 Feedback

Choice Number of schools Percentage of schools

1 = Touchscreen (specify) 117 3.5%

2 = Touchpad (specify) 22 0.7%

3 = Stylus (specify) 363 10.8%

4 = Keyboard (specify) 52 1.5%

5 = None of the above 2811 83.5%

Did students report (or did you observe) any problems with using any of the following?

Source: Field Staff Session Debriefing Form

16

NAEP Alliance

2015 Feedback

In your observations, did more students use the touchscreen, touchpad, or stylus?

Source: Field Staff Session Debriefing Form

Choice Number of schools Percentage of schools

1 = Yes 2322 67.1%

2 = No – How many did you have to plug in?

1136 32.9%

17

NAEP Alliance

2015 Feedback

Did all the tablet batteries last the entire group?

ChoiceNumber of

schoolsPercentage of

schools

1 = Touchscreen 130 3.7%

2 = Touchpad 201 5.8%

3 = Stylus 1839 52.9%

4 – Used equally 1307 37.6%

Source: Field Staff Session Debriefing Form

18

NAEP Alliance

2015 Feedback

Overall, how would you rate the students' engagement in the TBA Assessment?

Source: Field Staff Session Debriefing Form

ResponseNumber of

schoolsPercentage of

schools

1 = Very engaged 3152 89.8%

2 = Somewhat engaged 349 9.9%

3 = Not at all engaged 10 0.3%

19

NAEP Alliance

eNAEP System Demonstration

20

NAEP Alliance

Demonstration Features

• Supervisor interface• Student login• Tutorial Demonstration• Monitoring Demonstration• Tools Demonstration

NAEP Alliance

21

NAEP Alliance

Questions?