35
Standing Firm on Test Security Principles National College Testing Association 2015 Conference – Rising Tides & Shifting Sands September 10, 2015
Standing Firm on Test
Security Principles
National College Testing Association 2015 Conference – Rising Tides & Shifting Sands
September 10, 2015
Introductions
2
Faisel Alam
LSAC
Ray Nicosia
ETS
Rachel Schoenig
ACT
Agenda
3
Rules and Principles
Basic Test Security Principles
Applying Principles to Protect Assets and
Ensure Valid Results
What Happens Next?
Testing Rules
More Cheating Tools = More Rules
Rules v. Principles
7 Principles
Test Security Principles
1. Protect the integrity of testing
assets throughout the entire testing life
cycle.
2. Promote conduct that enhances test
security. Deter & detect conduct that
materially & negatively affects Testing
Assets.
Test Security Principles
3. Ensure that a reported test score is
an accurate & valid indicator of the test
taker’s own academic capabilities.
4. Foster effective communication that
enables prompt reporting &
resolution of test security concerns.
Test Security Principles
5. Ensure everyone in the testing
process is aware of and supported in
their roles.
6. Avoid placing individuals in situations
that may pose a conflict of interest or a
safety concern.
Test Security Principles
7. Build a sense of community,
collaboration, and trust that engages
and empowers people to act upon these
principles.
Applying Principles
Applying Principles: Check-in
Supervisor Manual requires you to check
photo IDs (unless examinees have a valid
religious exemption).
Examinees without proper IDs cannot enter
the examination room
Applying Principles: Check-in
Main principles behind identification rules:
3. Ensure that a reported test score and associated
information are accurate and valid indicators of the
test taker’s own independent achievement….
2. Promote conduct that enhances test security.
Deter and detect conduct that will materially and
negatively affect…test scores and the
trustworthiness of the testing brand.
1. Ensure that business processes…are “secure by
design.”
Applying Principles: Check-in
Scenario 1
As examinees are entering the testing room, you
see your boss fail to check IDs for certain
examinees. After double checking, you see they
didn’t have any religious exemption.
Applying Principles: Check-in
Scenario 1: Your boss fails to check IDs.
Poll: What would you do?
A. Assume she knows what’s she’s doing and let it go.
B. Go to the testing room and check the ID’s yourself. If they match, apply the “no harm / no foul” rule.
C. IDs are easily forged, so why bother checking them anyway?
D. Ask the supervisor her reason for not checking identification. If possible, ensure ID’s are checked and then report (either through irregularity report or a hotline).
Applying Principles: Check-in
Examinees are prohibited from having cell phones in the exam room. This is set forth in the T&Cs at registration, stated on the examinees admission ticket, and posted on signs before examinees enter.
Question: What are the primary principles behind these rules? (Choose all that apply)
Applying Principles: Check-in
Scenario 2
During check in, what should you do if you see
someone with a visible outline of what appears to be
a cell phone in their pocket? There’s nothing in the
supervisor’s manual that directly addresses this
situation, but it does provide that cell phones are
prohibited.
Applying Principles: Check-in
Scenario 2 - You see a visible outline of a cell phone.
POLL: What would you do?
A. Nothing. If it’s in the test taker’s pocket then they don’t plan on using it.
B. Fill out an irregularity report and let the examinee test.
C. Let the examinee test, but put her in the front of the class under the proctor’s nose.
D. Do my best impersonation of a TSA agent and pat them down. If she won’t submit to a pat-down, I don’t let her test.
E. Use my stern teacher voice and ask the examinee to take out what is in her pockets. If she won’t empty her pockets, I won’t let her test.
Applying Principles: During the Test
The terms and conditions state that no
electronic devices are allowed in the testing
room and that no external testing aids are
permitted. This is also on signs at the test
center, in your test supervisors manual, and
in the verbal instructions you read aloud to
the examinees before the test started.
Applying Principles: During the Test
Scenario 3
You are watching examinees and notice an
examinee moving the leg of their shorts up and
down. Upon closer observation, you notice they
have what appear to be crib notes written on their
thigh.
Applying Principles: During the Test
Scenario 3: You see crib notes on his thigh
Poll: What would you do?
A. Fill out an irregularity report and let them test if
they will let you take a photo of the “notes.”
B. Tell them that unless they stop moving the leg of
their shorts, they will have to stop testing.
C. Let them test, but keep and eye on them and
fill out a report.
D. Stop them from continuing to test and request a
photo of the notes to submit with the report.
Applying Principles: During the Test
No electronic devices are allowed in the testing
room and that no external testing aids are permitted.
Applying Principles: During the Test
Scenario 4
During testing, you notice that an examinee wearing
a religious headwrap is tilting his head and fiddling
with his wrap and ear. As you walk to the back of
the room, you suspect you see a wire running from
the back of his headwrap into the back of his shirt.
Applying Principles: During the Test
Scenario 4: You suspect a concealed device
POLL: What would you do?
A. There’s nothing I can do. I don’t want to violate
someone’s religious freedoms.
B. Write up an irregularity report about the situation.
C. Ask them to step outside the room into a private
area and ask to inspect the head wrap.
D. Tell them to remove their headwrap and that you’ll
need to keep it for the duration of the exam.
Applying Principles: After the Test
The test supervisors manual requires you to
store test materials in a secure storage
location and to account for all materials
before and after testing. It also requires
that only authorized individuals have
access to tests and that you maintain a
chain a custody form.
Applying Principles: After the Test
Scenario 5
You are picking up testing materials and notice one
test booklet has had pages cut out. You believe
the examinee who was assigned that booklet is
hiding the pages in his pants.
Applying Principles: After the Test
Scenario 5: Concealed testing pages.
POLL: What would you do?
A. This is way too awkward. I’m not addressing it.
B. Fill out an irregularity report.
C. Ask the examinee to submit to a pat down in the
presence of campus security.
D. Ask the examinee to produce the test materials or
you will call the police.
E. Keep everyone in the exam room until the materials
are produced.
Applying Principles: After the Test
The test supervisors manual requires you to
store test materials in a secure storage
location and to account for all materials
before and after testing. It also requires
that only authorized individuals have
access to tests and that you maintain a
chain a custody form.
Applying Principles: After the Test
Scenario 6
You stop by the test center supervisor’s area to say
good-bye and notice he is working on some
paperwork that is spread across his desk. As you
look more closely, it appears he has erased and
changed responses on two or three answer sheets.
Applying Principles: After the Test
Scenario 6: Possible response tampering
POLL: What would you do?
A. Nothing. Why would I question my boss?
B. Ask him what he’s doing and if I get a satisfactory
answer, then let it go.
C. Report it through the hotline and hope there is a
reward.
D. Assume this must just be part of the job and offer
to help.
What happens next …?
After you send in reports,…
we apply Principles, too!
3. Ensure that a reported test score is an accurate & valid
indicator of the test taker’s own academic capabilities.
4. Foster effective communication that enables prompt
reporting & resolution of test security concerns.
7. Build a sense of community, collaboration, and trust that
engages and empowers people to act upon these
principles.
Irregularity Report
Reporting Process
Potential Follow-up
Irregularity Report Process
*Within days decide if score(s) goes on hold
*Assign to one of three level staff members
1. Group/High Level Investigation
2. Immediate Action
3. Case Processing
*Feedback to test taker and testing staff
Rules v. Principles
7 Test Security
Principles
Thank You!
Faisel Alam, LSAC
Ray Nicosia, ETS
Rachel Schoenig, ACT
