Spring 2015 California Alternate Assessments (CAA) Field Test for English–Language Arts (ELA) and

Mathematics

March 11, 2015Sheila Self, M.Ed., California Department of Education

Kellie Taylor-White, MA, Lead Test Developer, ETSDan Stanley, Program Manager, ETS

California Assessment of Student Performance and Progress (CAASPP)

CAA Field Test Introduction and Instructions for Administration 2

Purpose and Target Audience

• This Webcast will help LEA CAASPP Coordinators to:– Learn all you need to know to prepare for administering the

CAA Field Test in your LEA– Learn how to prepare your Site Coordinators for CAA Field

Testing– Site Coordinators may also wish to view this Webcast

• This Webcast also serves as an introduction and explanation of the CAA Field Test for Site Coordinators and CAA Examiners– CAA Test Examiners are also required to view a Test

Administrator Tutorial that will be available starting March 20

CAA Field Test Introduction and Instructions for Administration 3

Agenda• Overview• Student Eligibility Requirements• Purpose of the CAA• Features of the CAA• Roles and Responsibilities• Test Security• Registering Students for CAA• Universal Tools, Designated Supports, and

Accommodations• Preparing Technology• Administering the Tests• After Testing

CAA Field Test Introduction and Instructions for Administration 4

Overview—Test Name

• A new alternate assessment for California as part of the CAASPP program

• The California Alternate Assessments (CAA)– English–language arts (ELA) – Mathematics

• Replaces the California Alternate Performance Assessment (CAPA) for ELA and mathematics

• Commonalities with other “next generation” alternate assessments

CAA Field Test Introduction and Instructions for Administration 5

• Testing window for all LEAs is April 15 through June 10, 2015.– All scheduling within this window is according to

local preference– Do not need to inform ETS or CDE of testing

schedule– All testing must take place within window,

including make-ups

Overview—Testing Window

CAA Field Test Introduction and Instructions for Administration 6

• All eligible students in grades 3–8 and grade 11 must participate– Students in ungraded programs will need to be

assigned a grade level for testing– Students in grades 5, 8, and 10 still take the CAPA

for Science (in addition to CAA at grades 5 and 8)

Overview—Test Takers

CAA Field Test Introduction and Instructions for Administration 7

• Testing is online only, using a compatible Web-browser with no software installation necessary.

• One-on-one test sessions with a Test Examiner, who is familiar with the student.

• All items are grade level, aligned with the Common Core State Standards (CCSS) and based on the Core Content Connectors developed by NCSC.

• Three different levels of complexity, and all three levels will be administered to all students.

• Items are being tested, not students.

Overview—Test Delivery

CAA Field Test Introduction and Instructions for Administration 8

Key resources for CAA Field Test:• CAA Field Test Administration Manual (TAM)• Sample Directions for Administration (DFA)

– Will be paired for use with a Training Test when available

• Grade-level DFAs• Individual Student Assessment Accessibility Profile

(ISAAP) Tool• Test Administration Tutorial

Overview—Resources

http://caaspp.org/about/caa/

CAA Field Test Introduction and Instructions for Administration 9

1. The student has a significant cognitive disability.2. The student is learning content derived from the

Common Core State Standards (CCSS). 3. The student requires extensive direct individualized

instruction and substantial supports to achieve measurable gains in the curriculum.

Additional detail on these requirements can be found in the TAM on page 5.

Student Eligibility Requirements

CAA Field Test Introduction and Instructions for Administration 10

Purpose of the CAA

• Give students with significant cognitive disabilities the opportunity to demonstrate their achievement of the Common Core State Standards (CCSS) by taking a test commensurate with their abilities. – Assessed through Core Content Connectors

(CCCs) that are rewritten main achievement standards from the CCSS.

– Provide more access to students with significant cognitive disabilities.

CAA Field Test Introduction and Instructions for Administration 11

• CAA is a grade-level assessment only• Student grade levels are taken from CALPADS.

– For students in ungraded programs, testing grades are calculated by the Test Examiner using the “subtract 5” rule

• For the Field Test, students in the same grade will see items reflecting all levels of difficulty, from least complex to more complex.

• Approximately 15 items per content area that take 30–60 minutes to administer.

Features of the CAA

CAA Field Test Introduction and Instructions for Administration 12

• Tests are administered with grade-specific and version-specific DFAs.

• Versions are assigned at the school level.– Version assignments for each school in California

will be posted to caaspp.org in late March, shortly before DFAs are available for download in TOMS.

Features of the CAA (cont.)

http://caaspp.org/about/caa/

CAA Field Test Introduction and Instructions for Administration 13

Roles and Responsibilities—LEA CAASPP Coordinator

For the CAA Field Test, LEA CAASPP Coordinators are

responsible for all of the following activities:

• Registering students in TOMS

• Training CAASPP Test Site Coordinators (SCs)

• Configuring student test settings in TOMS (with SCs)

• Confirming with SCs that each CAA Test Examiner has

viewed the Test Administration Tutorial and verifying training

completion

• Ensuring that all Test Examiners have read and signed a test

security affidavit

• Reporting all test security irregularities and/or breaches

CAA Field Test Introduction and Instructions for Administration 14

For the CAA Field Test, CAASPP Test Site Coordinators are responsible for the following activities:• Identifying certificated CAA Test Examiners, ensuring

that they view the CAA Test Administration Tutorial, and collecting the resulting certificate of completion. – CAA does not use the train-the-trainer model of

providing instruction to Test Examiners. Instead, Test Examiners must participate in the Test Administration Tutorial.

– The Test Administration Tutorial is required for all Test Examiners.

Roles and Responsibilities—CAASPP Test Site Coordinator

CAA Field Test Introduction and Instructions for Administration 15

• Collecting signed CAASPP Test Security Affidavits from CAA Text Examiners

• Adding each new CAA Test Examiner into TOMS and tracking completion of the Test Administration Tutorial

• Entering and/or verifying test settings for students in TOMS (universal tools, designated supports, and accommodations [UDAs] that were uploaded into TOMS with the ISAAP tool)

• Creating or approving testing schedules and procedures for the school

Roles and Responsibilities—CAASPP Test Site Coordinator (cont.)

CAA Field Test Introduction and Instructions for Administration 16

• Working with technology staff to ensure that testing devices are functioning properly

• Coordinating with CAA Test Examiners so they administer all assessments

• Mitigating and reporting all test security incidents in a manner consistent with state and LEA policies

• Collecting secure DFAs and any print-on-demand items from Test Examiners following testing and securely destroying them

Roles and Responsibilities—CAASPP Test Site Coordinator (cont.)

CAA Field Test Introduction and Instructions for Administration 17

For the CAA Field Test, CAA Test Examiners are responsible for the following activities:• Reading and submitting a signed CAASPP Test

Security Affidavit to your SC• Viewing the Test Administration Tutorial, printing the

resulting certificate, and submit to your SC• Reviewing designated supports and accommodations

in TOMS for each student you are testing• Downloading from TOMS and reading the DFAs

before testing

Roles and Responsibilities—CAA Test Examiner

CAA Field Test Introduction and Instructions for Administration 18

• Conducting a student response check for each student (required)

• Completing the learner characteristics inventory for each student (optional)

• Administering the tests in accordance with the CAA Field Test Administration Manual

• After testing, give any paper copies of the DFAs to your SC for secure destruction and work with your SC to permanently delete any PDF copies

Roles and Responsibilities—CAA Test Examiner (cont.)

CAA Field Test Introduction and Instructions for Administration 19

• Follow the same security guidelines as for other CAASPP assessments.

• Information about Field Test items, including DFA content, should not be shared with anyone not directly responsible for test administration.

Test Security

CAA Field Test Introduction and Instructions for Administration 20

• Students are registered for CAA in TOMS• Any students previously registered for “alternate” using the

CAPA flag in the CAASPP registration upload template (available since November) should automatically be registered for CAA by TOMS. – Registrations were accepted for all CAA/CAPA grades (3–8, 10, 11).

• Because tests are grade-level only, students must be assigned a grade for testing.

• For students who are in ungraded programs and whose IEP teams designate that they take the CAA Field Test, their grade for testing is calculated by subtracting five from their chronological age on September 1, 2014. – A student who was 10 years old on September 1 would be given the

grade five test (10 – 5 = 5).

• Confirm the grade assignment is correct for each student.

Registering Students for CAA

CAA Field Test Introduction and Instructions for Administration 21

• Starting March 9, one-by-one student registration available in TOMS– Registration for CAA/CAPA automatically un-registers the student for

any other CAASPP assessments– When CAPA or CAA is selected, student will be registered for both.– To unregister a student from CAPA/CAA, a CST or CMA must be

selected and CAA must have a selection of “No”. – Starting on March 20, CAA registrations will be viewable in the

TOMS student roster report

• Bulk registration is still available, for CAA only, using the same paper-pencil registration template– In TOMS, select “Paper test assignments” from the “Students” menu– Registering for CAA/CAPA will not automatically change any CAPA

materials orders. If needed, contact CalTAC to order additional CAPA materials through the supplemental order process.

Registering Students for CAA (cont.)

CAA Field Test Introduction and Instructions for Administration 22

• There is no specific deadline for registering students for CAA, however:– Registration should be completed as soon as possible

so that student rosters will accurately reflect the correct test assignments.

– Students should be registered for the correct assessments before test settings (designated supports and accommodations) are configured for those students in TOMS.

Registering Students for CAA (cont.)

CAA Field Test Introduction and Instructions for Administration 23

Universal Tools, Designated Supports, and Accommodations

The CAA field test makes many of the same universal tools, designated supports, and accommodations (UDAs) available as for other CAASPP assessments. • Universal tools are available to all students by default, although

they can be disabled if they are distracting for a student. • Designated supports can be provided at the discretion of a

teacher familiar with the student’s needs. • Accommodations must be specified in a student’s IEP.

– Note that, for the field test, students may use any accommodation routinely used in the classroom, even if not currently in IEP.

– During the next cycle of IEP review, you may update with state testing accommodations.

CAA Field Test Introduction and Instructions for Administration 24

Many of the same UDAs available for other CAASPP assessments are

available in the same way for the CAA field test, with the following

exceptions listed below:

• Text-to-speech, which is not available as an embedded support

but may be provided as a non-embedded support and/or through

an assistive device.

• Braille, which is not available for the CAA field test.

• American Sign Language, which is not available as an embedded

support.

• Translations/Glossaries, which are not available as embedded

supports.

• Closed Captioning, which is not available for the CAA field test.  

Universal Tools, Designated Supports, and Accommodations (cont.)

CAA Field Test Introduction and Instructions for Administration 25

The Individual Student Assessment Accessibility Profile (ISAAP) tool should be used to identify the right UDAs for each student taking the CAA.• Same tool for Smarter Balanced is now enhanced to work

for CAA. Select the “CAA” checkbox to configure the tool according to UDA availability for the CAA Field Test.

• Enter the specific needs of students and tool will automatically generate a file that can be uploaded into TOMS.

• ISAAP tool is available from caaspp.org, from http://caaspp.org/about/caa/.

Universal Tools, Designated Supports, and Accommodations (cont.)

CAA Field Test Introduction and Instructions for Administration 26

• The CAA are designed for one-on-one administration between a student and a certificated Test Examiner.

• Online testing using a supported Web browser (secure browser not used) – A list of supported systems, devices and browsers can be

found at http://ca.browsers.airast.org/supported-browsers/  – Refer to “TA Sites” column in the table

• A single device is needed to launch a test session and administer the test. (TA Interface not used)

• Other technology and device specifications, including headphones and keyboard requirements, are generally the same as for Smarter Balanced. – Assistive devices may be used, according to the

communication and response method of each student.

Preparing Technology

CAA Field Test Introduction and Instructions for Administration 27

Other Technology Resources

Refer to the following CAASPP resources at http://caaspp.org/administration/instructions/ for other instructions and guidelines. (Note that parts of these manuals will not apply to CAA, such as use of the secure browser, TA Interface, and the test settings not available for the CAA field test):• Technical Specifications Manual—for instructions on

preparing technology for testing• Online Test Administration Manual—for general

CAASPP system instructions such as security• 2015 Online Student Test Settings User Guide—for

information about assigning accessibility tools and supports

CAA Field Test Introduction and Instructions for Administration 28

Administering the Tests• A separate, grade- and version-specific Directions for

Administration (DFA) document guides Test Examiners through the CAA with item-specific administration instructions, including scoring rubrics for writing items. – Version assignments for each school in California will be

posted at http://caaspp.org/about/caa/ and DFAs will be available for download from TOMS before the testing window opens.

– DFAs are in PDF format and may be printed or used electronically according to local preference.

– Because these DFAs contain live testing material, they must be kept secure at all times and must be securely destroyed after testing. LEA or site coordinators must ensure that electronic

copies are securely destroyed on each device they were loaded to.

CAA Field Test Introduction and Instructions for Administration 29

Administration steps on test day:1. On the device that will be used for testing, Test Examiner

opens a supported Web browser and navigates to the CAA Online Testing Interface, with DFA in-hand or on a separate device.

2. Test Examiner logs on using TOMS username and password.3. Test Examiner logs onto student test session using student

first name and SSID. 4. Test Examiner is presented with three testing options:

a. Learner Characteristics Inventory (LCI)b. ELAc. Mathematics

5. Test Examiner completes the LCI (optional)

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 30

Administration steps on test day:

6. Test Examiner launches ELA or mathematics test and engages student for the start of testing

7. Test Examiner is prompted to administer a Student Response Check (SRC) to observe and record the student response mode and how the student will indicate their responses during testing. Answering these SRC questions is required for all students. Examples of student responses include:

Mouse/keyboard Verbal response Touch screen, gestures, or point Augmentative and/or alternate communication device Eye gaze

8. ELA and mathematics tests are untimed, and the Test Examiner may administer as many sessions over as many days as necessary according to individual student needs.

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 31

Learner Characteristics Inventory• An optional Learner Characteristics Inventory (LCI) is

available after logging in to a student’s tests. If completed, the LCI questions should be answered by the Test Examiner before the start of ELA and mathematics testing for each student, based on his/her familiarity with that student.

• Please note that LCI responses will be aggregated; neither Test Examiners nor student(s) will be identified in or by the results.

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 32

Student Response Check • CAA Test Examiners must administer the Student

Response Check (SRC) to each student at the beginning of testing. The purpose of the SRC is to document that the student has a consistent and observable way to communicate his/her response to the test items, using the intended response mode(s). 

• Hand-over-hand or other physical prompting by the test examiner is not permitted in the CAA field test.

• The SRC does not need to be administered twice. (Can be skipped at the start of the second content area).

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 33

Student Response Check (Cont.)There are three possible outcomes from the SRC: 1. The student demonstrates an observable, consistent

response 2. The student demonstrates an observable, but

inconsistent, response3. The student does not demonstrate any observable

responses

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 34

Student Response Check (Cont.)• If the outcome is “1,” Test Examiners administer the entire

assessment.  • If the outcome is “2,” Test Examiners administer the first four ELA

or mathematics items. If a consistent, observable response is elicited for any of these four items, administer the entire assessment. – The response does not need to be correct but rather the

student orients to the test and attempts to respond. – If no observable response for the first four items, the Test

Examiner should stop testing and report this to the Site Coordinator.

• If the outcome is “3,” Test Examiners do not administer the assessment.

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 35

Pause RulesThe following pause rules apply regardless of whether the student or the Test Examiner pauses the test or there was a technical issue that resulted in the student being logged off. The CAA may be paused and resumed as many times as necessary to elicit the student’s best testing performance. • When a test is paused, you must log back on to resume

testing. Upon resumption, students will automatically be directed to the first page that has an unanswered item.

• In the event of a technical issue (e.g., power outage or network failure), student’s test will be paused and the students will be logged off.

• Students will return to their last unanswered test question when logged on again.

Administering the Tests (cont.)

CAA Field Test Introduction and Instructions for Administration 36

Administering the Tests (Cont’d)

Test Timeout (Due to Inactivity)

• As a security measure, test sessions are automatically logged off

after 30 minutes of test inactivity. This timeout also results in the

test being paused automatically.

• Activity is defined as selecting an answer or navigation option in

the test (e.g., selecting [Next] or [Back] or using the Questions

drop-down list to navigate to another item). Selecting an empty

space on the screen is not considered activity.

• Before the system logs off of the test, a warning message will be

displayed on the screen. If you do not select [OK] within 30

seconds after this message appears, he or she will be logged off.

CAA Field Test Introduction and Instructions for Administration 37

CAA Training Test

• For practice and training purposes, Test Examiners will have access to an online training test starting in early April.

• Used in conjunction with the sample DFA, this training test can help both Test Examiners and students become familiar with the new testing format and functionalities of the CAA assessments.

• More information about the Training Test will be provided when available.

CAA Field Test Introduction and Instructions for Administration 38

After Testing

• Site Coordinators ensure all DFAs are collected and securely destroyed– Shred or otherwise destroy printed copies– Permanently delete electronic PDF copies from all

devices• As necessary, enter any special testing condition

codes in TOMS. • After testing, you may be asked to complete a survey

about your field-testing experience. This information will help to improve the test.

CAA Field Test Introduction and Instructions for Administration 39

Comparing CAPA to CAAAspect CAA FT CAPA

TrainingTraining methodology for Test Examiners Training is provided via an on-demand test

administration tutorial and is required in order to administer the CAA.

Train-the-trainer model: Training was provided by SCs who were trained by LEA CAASPP Coordinators.

AdministrationTesting mode Online only (A print-on-demand accommodation

is available through the online system for test items by request) 

Paper only

Testing format Test is administered one on one. Test administered one on one.

Levels Grade level only; calculation is used to assign grades to students in un-graded program for testing.

Levels and grades

Physical prompting of students No physical prompting of any kind is permitted. Hand-over-hand prompting was permitted.

Entering responses The student may enter their responses using a mouse or keyboard or the CAA Test Examiner enters responses into the online testing interface reflecting the student’s indicated response.

CAPA Examiner filled in the answer document.

Scoring CAA Test Examiner provides scores for writing items as directed in the DFA. Scoring of other items is done by the online system. No student scores will be available for the Field Test.

CAPA Examiner provided scores.

Second Rating for Test Examiner-scored items. There is no second rater. A second rater scored 10% of exams.

ToolsPhysical stimulus, manipulatives None Test used physical stimuli and manipulatives.

AccommodationsAvailability Similar universal tools, designated supports, and

accommodations to other CAASPP assessments, with some exceptions.

None; the CAPA was administered and adapted based on student need.

CAA Field Test Introduction and Instructions for Administration 40

Support for the CAA Field Test

The CAASPP portal at http://caaspp.org/ is the home for all CAASPP assessment administration information, including the CAA field test. All CAA materials and resources will be

available on the CAASPP CAA Field Test Web page at http://caaspp.org/about/caa/.

The California Technical Assistance Center (CalTAC) is here to support all LEA CAASPP coordinators!

Monday–Friday from 7 a.m.–5 p.m. PTE-mail: caltac@ets.orgPhone: 800-955-2954

Web site: http://caaspp.org

CAA Field Test Introduction and Instructions for Administration 41

Questions?