Computer Adaptive Testing
FAIRFAX COUNTY PUBLIC SCHOOLS
A JOINT PRESENTATION BY:
THE OFFICE OF INSTRUCTIONAL SERVICES
(PRE K – 12 CURRICULUM & INSTRUCTION)
AND
THE OFFICE OF STUDENT TESTING
1
VDOE introduces
Computer Adaptive Testing for Grade 6 Math 2
This year, in Virginia, the 6th grade Math SOL will
be administered using a Computer Adaptive
Testing (CAT) model.
The CAT model was piloted during the 2013-2014
school year for both Mathematics and Reading
SOLs.
What is CAT?
3
Briefly, CAT provides a customized assessment for
every student. Answers are scored as each
question or problem is answered, and the
computer selects subsequent items based on the
student’s performance on the items already
completed.
What is CAT?
4
Computer Adapted Testing is a proven technology;
it has been around for over 30 years and has been
shown to increase student engagement and test
security.
For example, since 2002 Virginia has used the
Algebra Readiness Diagnostic Test (ARDT) which
is administered in the CAT format.
The GRE exam is also a CAT.
A Computer Adaptive Test Versus a Traditional Test
• A Computer Adaptive Test is an assessment that
is customized for every student based on how the
student responds to the questions.
• A traditional test is an assessment in which all
students who take a particular version of the test
respond to the same test questions.
5
CAT Implementation Schedule
VDOE’s Implementation Schedule:
It is very likely that the next SOL test or tests that become computer
adaptive will be in mathematics. However, the timeline for
implementation in mathematics, as well as other content areas, is
uncertain due to budget constraints.
6
Similarities between a CAT and a Traditional Online Test
Both test types:
• Assess the Standards of Learning (SOL) for the course (e.g., Grade 6 Mathematics).
− This means that the CAT does NOT assess standards below or above grade level.
− The CAT adjusts the difficulty level of the questions from within the item pool for the test.
− ARDT and GRE are CATs that use a different model and assess across a wider range of difficulty
7
Similarities between a CAT
and a Traditional Online Test
Both test types:
• Contain sample items*, multiple-choice questions,
and approximately 15% technology-enhanced items
(TEIs).
• Are untimed, but should generally be completed
within one school day.
*The responses to the sample items do not impact the difficulty
level of the questions selected for the student in a CAT.
8
Similarities between a CAT and a Traditional Online Test
Both test types:
• Require students to log in to the test using a Student Authorization Ticket.
• Allow students access to the same test manipulatives (e.g., calculator, scratch paper).
• Allow students access to the same online tools on the TestNav tool bar (e.g., highlighter, ruler).
• Contain the same section breaks (if applicable).
• Allow for accommodations.
9
Differences between a CAT and a Traditional Online Test
The number of questions (or test items) on a CAT will
be different from the number of test items on a
traditional test. However, all students taking a CAT
will be administered the same number of items.
See the Grade 6 Mathematics Test Blueprint on the
VDOE web site at, as referenced in Superintendent’s
Memo 251-14.
10
Differences between a CAT and a Traditional Online Test
The Grade 6 Mathematics CAT has 45 operational items and 8 field test items for 53 total items.
The Grade 6 Mathematics traditional test has 50 operational items and 10 field test items for 60 total items.
Grade 6 Mathematics Test Blueprint Summary Table
11
Differences between a CAT and a Traditional Online Test
Difference in item type:
• The CAT test will have some multiple-choice and
technology-enhanced items with only 3 answer
options.
• The directions in the Examiner’s Manual mention
this. However, prior to the day of testing, students
should be made aware so they will not think a
question with three answer options is an item with
an error.
12
12
Differences Continued… Navigating Through the Test
On a CAT:
• Each question must be answered completely
before the student can move to the next question.
• Questions cannot be skipped.
• Students cannot go back and change responses to
questions that have been previously answered;
there is no “Previous” button.
13
On a CAT:
• Since students must answer each question before
moving forward and students are not allowed to go
back to previous questions, it is not necessary to
flag questions for review. Therefore, the Flag for
Review button has been removed.
• Since all questions must be answered to reach the
end of the section or test, there is no Section
Review Screen.
14
Differences Continued… Navigating Through the Test
On a CAT:
• The test does not have a Final Submit button. It is
automatically submitted for the student after the
last question has been answered.
15
Differences Continued… Navigating Through the Test
On a CAT:
• The student cannot submit a test if it has not been
completed.
• All questions must be answered.
16
Differences Continued… Navigating Through the Test
On a CAT:
• Students may take individual breaks, but no group
break is allowed since each student will have a
unique set of test questions.
• There is a transition screen between the
non-calculator section and the calculator section
of the test. Students must be given calculators
before continuing.
17
Differences Continued… Administering the CAT
The Test Directions
instruct students to
check this box once a
calculator is received,
but students will be
able to proceed past
this screen without
entering a seal code
and without checking the box.
18
Differences Continued… Administering the CAT
Administration of a CAT: Student Accommodations
All* student accommodations that are available on
a traditional test are available on a CAT. However,
the manner in which accommodations are
administered may be different.
*Plain English Audio CAT will be available but the
implementation may be delayed until the 2015-
2016 school year.
19
Because each student will have a unique set of test
questions, the Read Aloud or
Interpreting/transliterating accommodation must be:
• Provided in a one-on-one testing environment.
• Administered without an Examiner Authorization ticket. The
Examiner will read from the student’s computer, but the student
must maintain control of the computer. The Examiner is not
able to review the test ahead of time.
With either accommodation:
• The session must be recorded or a proctor must be present
throughout the administration of the test.
Administration of a CAT: Read Aloud or Interpreting/transliterating Accommodation
20
21
Remember that a student receiving the Read Aloud accommodation on classroom assessments may be administered the Audio format of the test on the day of SOL’s.
Please refer to the FCPS SOL Accommodations Reference Chart for further details:
http://fcpsnet.fcps.edu/ssse/SpecialEducation/assessment/sol/SOL-Accommodations.pdf
Administration of a CAT: Read Aloud or Interpreting/transliterating Accommodation
New Special Testing Accommodations Sections
created in test manuals for Computer Adaptive
Tests:
• Read-Aloud Guidelines for Computer Adaptive Tests (CAT)
• Interpreting/Transliterating Computer Adaptive Test (CAT) Items
Located in:
• Appendix B - Examiner’s Manuals
• Appendix C - Test Implementation Manuals (TIM)
22
Administration of a CAT: Read Aloud or Interpreting/transliterating Accommodation
Administration of a CAT: Reporting a Problem During Testing
If a student is completing a CAT and raises a
question or concern regarding a test item, please
contact a Test Administration Specialist with the
following information: school name, the reporting
student’s STI, and the number of the item (e.g., #5,
#16) the student is questioning. This is necessary
to identify the specific test item the student is
seeing.
The Office of Student Testing will communicate with
VDOE regarding the student’s concern.
23
23
Scores and Score Reports
CAT Scaled Scores:
• Scaled scores range from 0 to 600, with 400
indicating pass/proficient and 500 indicating
pass/advanced.
• The scaled score is determined by the number of
questions answered correctly AND the difficulty
level of the questions answered correctly.
• There is no longer a cut score for passing, and the
raw scores will no longer be reported.
24
Grade 6 Mathematics Scaled Scores:
• Scaled scores have the same meaning across all
grade 6 mathematics tests that assess the 2009
Mathematics Standards of Learning.
• This means, for example, that a scaled score of
400 on this year’s CAT test has the same meaning
as a score of 400 on last year’s traditional grade 6
mathematics online test or the grade 6
mathematics paper test.
Scores and Score Reports
25
CAT individual score report availability:
• Individual score reports will be available within 24
hours after testing is completed.
Scores and Score Reports
26
For a CAT, score reports will include:
• The student’s score, performance level, and a
description of the student’s performance level.
• Item level performance information for the student
through the Student Details by Question (SDBQ)
report.
Scores and Score Reports
27
Superintendent’s Memo 255-14 announced these
resources on the VDOE web site :
• A video entitled A Guide to Computer Adaptive
Testing;
• A video entitled Computer Adaptive Testing in
Virginia;
• A document titled Frequently Asked Questions
About Computer Adaptive Testing.
CAT Resources
28
Additional resources on the VDOE website include:
• A “Traditional Test versus CAT” comparison
document.
• A CAT Training Test.
CAT Resources
29
• The CAT Training Test provides students with a
simulated experience for navigation through a CAT form.
• Students can practice answering a small number of
questions (grade 6 mathematics released items) to
familiarize themselves with CAT features that are similar
to a traditional test and those that are new or different.
• The CAT Training Test will be shorter than an actual CAT
(due to a limited number of grade 6 mathematics
released test items) and will not have the same level of
customization for students. It serves as a brief
simulation for CAT training purposes only.
The CAT Training Test
30
CAT Questions?
31
Questions may be sent to either a Math Specialist or
a Test Administration Specialist
Rose Moore Coordinator, Mathematics (K-12 Instruction) 571-423-4725
David Van Vleet Education Specialist- MS Mathematics 571-423-4723
Spencer Jamison Education Specialist- ES Mathematics 571-423-4728
Sheri Masich Test Administration Specialist 571-423-1403
Erica Meadows Test Administration Specialist 571-423-1419
Stuart Pierce Test Administration Specialist 571-423-1413