Chloe Mendonça
UK Marketing Manager, Questionmark
Enhance Exam Integrity with Online Proctoring
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Agenda
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
What is online proctoring and how does it work?
What are the benefits of online proctoring?
Managing test security threats
Considerations & Best Practices
A bit about us…
• Founded in 1988
• Offices in US, UK, Canada; Distribution partners in 27 countries
Background
• Multilingual Assessment Management System
• Questionmark OnDemand (Cloud/SaaS)
• Questionmark Perception (Installable software)
• Assessment technologies and portal solutions that enable organisations to measure knowledge, skills, and attitudes securely for certification, regulatory compliance and learning
Products & Services
• Enterprise customers worldwide within the corporate, government; higher education; awarding body; test publisher sectors
Customers
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Online Proctoring / Online Invigilation
• We used to use the term invigilation
• With the proliferation of US Companies providing Online Invigilation the industry appears to be accepting the term Online Proctoring
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Online Proctoring
X
A webcam is used to authenticate the candidate’s identity
The proctor observes the candidate as they complete their exam
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Candidates can take exams online from their home or office using their own laptop
Options for Proctoring Tests and Exams
Record & Review
Dedicated Kiosk with multiple cameras
Test Centers
Events (Classrooms or Conventions)
Remote real-time webcam
Un-Monitored
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
• Convenient, personable, consistent
• Proctor vulnerable to unfair influence
Events (Classrooms or Conventions)
• Personable, consistent
• Proctor vulnerable to unfair influenceTest Centers
• Proctor is less vulnerable
• New and not yet widespread
Dedicated Kiosk with multiple cameras
• Proctor is less vulnerable
• Easy to deploy for use at home
Remote real-time webcam
• More cost effective than remote real-time
• Nothing for content theftRecord & Review
• Works for employees
• Nothing for content theftUn-Monitored
Comparing the options
ID F
rau
d
Pro
tect
C
on
ten
t
Min
imiz
es
Ch
eat
ing
√ √ √
√ √ √
√ X
√ X X
√ √ √
√
√ √ √
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Traditional Proctoring•Proctor observes test taker physically in
person / in the same roomOnline Proctoring• Proctor observing test taker live via webcam
• Internet connection must be continuous
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Typical Candidate Experience
1Candidate login
and system-check
2Verification of
candidate ID and work space
3Candidate
begins exams
Initiates proctored
session
Exam released to candidate
4Candidate completesexam and proctored
session ends
Candidate minimizes
proctor window
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Who is using it?
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Higher Education and Certification Institutions
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
What is driving this need? – Explosion of MOOCs
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
“Students definitely prefer online proctoring. It’s more
convenient and affordable than going to a testing centre.”
- Chris Heather, Product Manager, Coursera
What is driving this need? – Reducing time-to-market
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Release Cycles Happen More Frequently
Which means that….
Qualifications for credentials/certification need to keep pace
Exam content needs to be updated more frequently
(sometimes several times a year)
Which means that….
“Release cycles are getting quicker and quicker. For certification, this means that consultants and others have to show that they are always up to date and are certified on the latest release.
But of course if you are taking certification exams more often, certification costs have to be much lower. In some regions, people have to travel large distances to get to a test centre. Our aim is to allow certification anytime and anywhere using the cloud.” - Ralf Kirchgaessner, Manager of Global Certification, SAP
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
What is driving this need? – Enhancing Exam Integrity
Director Varinder Bajarh, Director of Studentway Education told the undercover test taker:
"Someone else will sit the exam for you. But you will have to have your photo taken there to prove you were present.“
The researcher was told a "guaranteed pass" would cost £500 - about three times the proper fee for the exam
Student visa system fraud exposed in BBC investigation . At least 29,000 exam results were invalid!
Benefits of using Online Proctoring
Professionally trained Technology Supported
Lower Costs
Independent Proctors
Fairness Candidate Satisfaction
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
How does online proctoring deal with exam security threats?
Impersonation. Cheating. Content Theft.
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Key threats to consider
Candidate Impersonation Collusion Content Theft
Access to restricted aids Reliability and robustness Privacy & data protection
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Dealing with impersonation / proxy test takers
Enabling proctor controls, e.g. pause stop etc.
Recording / storing session and time stamping
any incidents
REC SAVE
Authenticating candidates
Options for…
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Dealing with cheating aids / collusion
Camera can see large portion of the room
Camera & microphone pick up sounds / talking
Secure browser to restrict access to online aids
Checking of area with reflective surface
Panning of room with webcam
Proctors can be prohibited from seeing
test taker screens
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Mitigating risk of content theft
Secure browser to restrict right click, copy paste and other functions
Live monitoring throughout the exam
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Considerations for Online Proctoring
Must have high speed internet connection
Reasonably comfortable with tech
Proctor language
Breaks are restricted Computers not standardised
Test program customisation
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Online Proctoring Best PracticesSix Tips
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
#1 Develop & Define Policy
• Policies for technical problems
• Policies for accessibility accommodations
• Policies for following up on infractions
Anticipate problems and have a way to manage them
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
#2 Set Candidate Expectations
Prepare and educate candidates with training resources
Blog Articles
Online Information
Videos
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
#3 Use a comprehensive lock-down program
Should allow prevention of:
PrintingScreen capture
Copy/ PasteLaunch of applications and
communication toolsTask switching
Browser control / navigationControl/system keys
Right-click / context menu optionsView source
Accidentally exiting exam
Should be compatible across multiple devices and operating systems:
Windows Mac iPad
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
#4 Allow communication between proctor and test-taker throughout session
Participants should be able to communicate via:
• Audio
• Video
• Chat
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
#5 Allow proctor to control test session
Proctor controls should include the ability to pause, suspend or cancel the test.
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
#6 Record and store exam sessions (including chat log)
• Record entire session including all communications
• Time stamp incidents to make them easy to review
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Online Proctoring in the future
It’s only going to get better…
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
More options for candidates
Online Proctoring is broadening the horizon for candidates and test sponsors
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Keep in touch!
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Follow our blogblog.questionmark.com
Connect with us on social mediaYouTube, Twitter, Instagram, LinkedIn, Google+, Facebook
Thank you for attending!Watch out for our next session: An Introduction to Questionmark
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Chloe MendoncaUK Marketing Manager