C O N F E R E N C E 2 0 1 9
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Essential Elements of Test Security
John Kleeman, Executive Director and Founder
David Hunt, Information Security Officer
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
In many industries, assessments increasingly important
Ensure workers knowledgeable, safe, competent in practices
Ensures skills and competence certified
Threat of an unscrupulous person who chooses to cheat putting co-workers / organization / public at harm
Why Test Security Matters
Slide 2
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
General security
• The concept of risk and how it can drive security
• General security good approaches
Nine common risks to test security
• What they are
• Real life examples
• Mitigations
Practical demonstration of
some security risks
Using Questionmark
securely
What we are going to cover today
Slide 3
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Data Security
System Security
Slide 4
Trustable Results
Planning Assessment
Authoring Items
Assembling assessment
Pilot and Review
Delivery
Analyze Results
Trustable Results Rely on a Trustable & Secure Process
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Data Security
System Security
Slide 5
Trustable Results
Planning Assessment
Authoring Items
Assembling assessment
Pilot and Review
Delivery
Analyze Results
Content theft
Content theft
Test center disruption
Identity fraud
Unauthorized disclosure
Tampering
Some Risks to Trustable Results
Unauthorized aids / help
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Cheating on Exams a Problem Worldwide
Slide 6
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Meanwhile Data Breaches Rise …
Slide 7
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
A. Content theft / leakage
B. Unauthorized aids for cheating during exams
C. Identity fraud
D. Results tampering
E. Personal information breached
F. Something else
Question: What keeps you up at night?
Slide 8
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Managing risk is the key to being secure
Slide 9
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Confidentiality
IntegrityAvailability
Confidentiality
• Keep questions secure -avoid content theft
• Results only available to those who should know
Integrity
• Right person takes assessment
• Assessment process fair and robust
• No cheating
Availability
• Assessments can be taken when needed
• Results are stored safely
First Start with Your Security Objectives
Slide 10
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Formal process can identify all threats, less likely to miss key threats
Quantifying risk allows you to prioritize actions that will reduce risk
Recommended by all respected security authorities: ISO 27001, NIST & many more
Example▪ 2017 Verizon data breach report reported 81% of hacking-related
breaches involved weak or stolen passwords.▪ For most organizations, a risk assessment is going to highlight this
and suggest some mitigations
Why Assess Risks?
Slide 11
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
No News Driven Security
1. Assets (What's important)
2. Asset data▪ Value
▪ Impact
▪ Threats/Vulnerabilities
▪ Likelihood and Frequency
3. Repeatable risk assessment process
4. Report and action findings
5. Monitor and Measure
Risk Based Security Decisions
Slide 12
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
High probabilityLow impact
High probabilityHigh impact
Low probability.Low impact
Low probability.High impact
LOW HIGH
LOW
HIG
H
Impact
Pro
bab
ility
Then look at Risk Probability and Impact
Prioritize security mitigations based on risk impact and probability
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Security
vs
UsabilityVS
Security vs
Usability
The Great Balancing Act
Slide 14
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
After delivery
Tampering with results Unauthorized disclosure of results
During test delivery
Identity fraudContent theft /
harvestingCopy answers from
another Getting help from
othersUnauthorized test aids
Before delivery
Content theft from item bank Disruption at test center
9 Common Risks to Consider
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk:
Content theft from item bank
Authoring Items
Assembling assessment
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Content theft from item bank
Slide 17
Author leaks content deliberately to help their students
Author leaks content by mistake or by using poor security
Technical vulnerability in item bank
Potentially very high: requires rewriting all items
Could invalidate test for all test takers
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
More real world examples
Slide 18
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Use a secure cloud service (Look for compliance with known security standards)▪ Avoids local files▪ Avoids email of questions▪ ISO 27001 and good technical security
Good HR practices with authors▪ Training and education▪ Confidentiality agreements▪ Remove access when people leave project▪ Good passwords▪ Extend to translators too if test is translated
Use permissions and roles▪ Authors should only have access to the minimum they need for a project▪ Restrict access to questions/assessments they do not need▪ Single sign-on
Some mitigations to prevent item bank leakage
Slide 19
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Just enough capability to get the job done
In assessment management systems:▪ Use roles and permissions to give
people the minimum capabilities they need
▪ Restrict access to questions to those who need
▪ Have a separate user account for privileged (high capability) actions
One person
requests
Another person
approves
Activity authorized
Two important security principles help
Slide 20
Least privilege Segregation of duties
By requiring collusion it becomes more difficult than one rogue person!
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Disruption at test center
Slide 21
Pilot and Review Delivery
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Disruption at a test center
Slide 22
Poor technical security at test center allows questions to be seen
Test center proctors corrupt
Test center proctors / teachers have incentive to help their students
Can result in a batch of test takers at the test center all being helped
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
More real world examples
Slide 23
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Don’t allow download of questions to test center in advance
Consider not using test centers in pilot process
Data analysis / forensics
Incident response plan in place and tested
Using good technical measures▪ High quality TLS encryption
▪ Proctors unable to see questions
Use online proctoring▪ Remote proctors who do not know test takers and cannot receive a bribe less likely
to collude with test takers
Possible mitigations
Slide 24
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Traditional Proctoring•Proctor observes test taker physically in
person / in the same room Live Online Proctoring• Proctor observing test taker live via webcam
Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
A practical demonstration
How secure are our normal devices?
Slide 26
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Identity fraud
Slide 27
Delivery
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Identity fraud
Slide 28
Candidate gets a friend to take an exam for him/her
Candidate pays someone to take an exam for him/her
At least one result is meaningless
Can threaten acceptance of programme
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Proctoring▪ Proctor checks government issued photo-ID▪ If concerned about proctors colluding with candidates, use online
proctoring
Frequent testing▪ Series of tests harder to get someone to take for you than single large test▪ With online assessment and online proctoring, much more realistic to have
more frequent, smaller tests
Use single sign-on (SAML)▪ People less keen to share credentials if they allow access to all their other
data
Reduce the motivation to cheat
Some ways to mitigate candidate impersonation
Slide 29
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Fraud needs:▪ Motivation▪ Opportunity▪ Rationalization
For a high stakes test, there may be Motivation
Anti-cheating measures seek to reduce Opportunity
How can we reduce Rationalization?
Fraud triangle
Slide 30
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Mitigating Rationalization
Slide 31
Rationalization
“Everyone else cheats so why shouldn't I?”
“I have no alternative but to cheat and I have to pass because …?”
“I can get away with this”
“I didn’t know I was doing anything wrong”
• Explain the facts:• Most people do not cheat• Consequences of cheating
Mitigation
• Provide environments for the candidate to be able to pass the exam honestly
• Explain the security measures in place and the consequences if they’re caught
• Explain and gain positive agreement to theo Honesty Codeo Code of ethicso Non-disclosure
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Content theft / harvesting
Slide 32
Pilot and Review Delivery
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Content theft / harvesting
Slide 33
Screenshot or other technical copy of test questions
Group harvesting, everyone remembers a few questions and shares online
Someone pays to take test just to see and remember questions
Slow degradation of integrity of test
Expensive as need to write new items
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Use proctoring but don’t allow proctors to see content
Secure browsers like Questionmark Secure
Large and changing item banks with randomized test content
Reduce motivation to cheat / honour code
Use video/audio as part of stimulus to make harder to copy
Prevent harvesters from taking tests (pre-requisites, limit retakes)
Keep time limit as short as sensible
Possible mitigations
Slide 34
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
A secure browser reduces content theft and cheating
While using a secure browser, it is difficult for participants to▪ Make screenshots▪ Copy content▪ Run other programs▪ Search the Internet or use chat
Questionmark has a Questionmark Secure App and other vendors have other approaches
Secure browsers
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Copy answers from another
Slide 36
Delivery
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copy answers from another
Slide 37
Candidate looks at the screen of the person next door to him/her and copies their answers
In a take-home exam, two people do it together
Organized answer sharing
Invalid test results for a candidate
Devalues credibility for others
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Appropriate separation between workstations
Randomization of question order / question selections / choice order
Use “Privacy Screen Filters” to reduce visibility angles
How to mitigate copying answers
Slide 38
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Getting help from others
Slide 39
Delivery
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Getting help from others
Slide 40
Use a cellphone or instant messenging to get live help from a friend or coach
Someone sits by test taker to give coaching
Invalidates one result
If prevalent, degrades program
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
More real world examples
Slide 41
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Use a secure browser (e.g. Questionmark Secure) for online tests
Proctoring (especially online)
Reduce motivation to cheat / honour code
Frequent testing
Forensics / analysis
Consider other approaches than essays
Do not tie test-taker performance to Instructor compensation
Some mitigations
Slide 42
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Unauthorized test aids
Slide 43
Delivery
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Unauthorized test aids
Slide 44
Cheat sheets / access to notes
Test taker Googles the answers
Calculators / other tools
Makes results less valid
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
More real world examples
Slide 45
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Secure browsers
Proctoring
Make exam open book▪ In the real world, people have access to reference material, is it fair and valid to deny
it during an exam?
Ask higher level questions (e.g. Situational judgement)
Reduce motivation to cheat / honour code
Limit or prohibit personal items in test area
Mitigations
Slide 46
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Tampering with results
Slide 47
DeliveryAnalyze Results
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Tampering with results
Slide 48
An insider changes someone’s score out of favouritism
Participant bribes someone to change score
Technical vulnerability exploited to adjust scores
Ranges from just one score awry to whole exam invalidated
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Robust delivery and results platform
ISO 27001 or similar security management
Store results in an online cloud system
Permissions / roles / segregation of duty
Robust audit trail
Care with results when leave the assessment system
How to mitigate tampering with results
Slide 49
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Risk
Unauthorized results disclosure
Slide 50
Analyze Results
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Unauthorized results disclosure
Slide 51
Mistake by administrators discloses data
Malware or other technical vulnerability discloses data
Serious embarrassment and negative publicity
Government fines
Invasion of privacy
Threats Risk impact Real world example
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Ensure that your supplier(s) have strong security▪ ISO 27001 is good
▪ Check they remain committed to security
Train your personnel well including security awareness
Remove access from personnel who leave organization/project
Ensure your systems are well secured if results go there
Strong passwords and single sign on
Permissions / roles / least privilege
Mitigations
Slide 52
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
After delivery
Tampering with results Unauthorized disclosure of results
During test delivery
Identity fraudContent theft /
harvestingCopy answers from
another Getting help from
othersUnauthorized test aids
Before delivery
Content theft from item bank Disruption at test center
The 9 risks we have looked at
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
It’s important to have Defense in Depth
Slide 54
Layers of protection rather than a single point of failure!
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Many of the mitigations for one risk also apply to other risks
A single mitigation or control is vulnerable to bypassing
A defense in depth strategy involves having several controls, for example:▪ No cellphone policy
▪ Proctoring
▪ Lock down browser
▪ ID checks
▪ Random selection of questions
All complement each other
Why defend in depth?
Slide 55
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Identify goals of your assessment program
Look at risks that apply to you
Implement mitigation measures based on importance of those risks
Improved security gives better data for better decisions
Improved security also increases trust from stakeholders
Summary
Slide 56
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Some additional Questionmark specific advice
Slide 57
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
If you were to lose your data▪ You’d have to re-create all your questions and assessments▪ You would have lost all your past participant results – no evidence, no
item analysis, no records▪ Your assessment program would be down for a considerable period
If you use Questionmark OnDemand, we do it for you
If you use Perception / On Premise▪ Set up a reliable backup
▪ Conduct regular restore tests to check the backups can be restored
Backup
Slide 58
Ris
ksH
ow
to
dea
l w
ith
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
If you give someone a wide set of privileges, this can cause damage▪ If their credentials are stolen▪ If they make a mistake (e.g. delete something)▪ If they become untrustworthy
Limit “super user” permissions If you need “super users”, create two accounts
▪ One for normal access▪ One for privileged access
Reduces risk of super user account causing damage
Least privilege
Slide 59
Ris
ksH
ow
to
dea
l w
ith
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Define roles that match needs
Allocate users to roles
Consider▪ Least privilege
▪ Segregation of duties
Use role-based security in Questionmark OnDemand:
Use roles and permissions
Slide 60
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Passwords can be compromised in many ways including:▪ Weak passwords being guessed or cracked▪ People writing down passwords insecurely▪ Phishing attacks▪ Malware (e.g. keylogging)
Upgrade to the latest version (OD3 or Questionmark OnPremise) so you can set a good password policy (passphrases!)
Train your users on passwords and general IT security Limit rights to the least privilege that people need Consider SAML which allows two-factor authentication
Password weaknesses
Slide 61
Ris
ksH
ow
to
dea
l w
ith
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
▪ Common for organizations to fail to remove permissions from leaving or transferring employees/contractors
▪ Suppose someone finds a way to create themselves an administrator user or give themselves more permissions without you knowing?
▪ At regular intervals▪ Review all Questionmark administrators
Are they who you expect?
Are they still employed and/or allocated to your project?
Do they still need all the access they have?
▪ SAML can be another approach
Review usage rights
Slide 62
Ris
ksH
ow
to
dea
l w
ith
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 63
Results Forensics: Assessment Completion
Time report
• Spot anomalous completion times
• Identify potential cheating
• Flag potential content theft
• Determine if allotted time is sufficient for completing the assessment
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 64
Results Forensics: Assessment Results
Over Time
• Spot anomalous results for specific participants
• Mean score
• Minimum score
• Maximum score
• 95% confidence interval
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Most secure as updated monthly, often fixing security issues
If you’re not already there, please move to OD3
Authoring Manager Is less secure
If you can’t move to OnDemand, you should move to Questionmark OnPremise▪ Removes Authoring Manager
▪ Allows stronger passwords and SAML
▪ Many, many security improvements
Upgrading to the latest version doesn’t just give you more features, it also makes you more secure
Slide 65
Questionmark OnDemand On Premise
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Data Security
System Security
Slide 66
Trustable Results
Planning Assessment
Authoring Items
Assembling assessment
Pilot and Review
Delivery
Analyze Results
Trustable Results Rely on a Trustable & Secure Process
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Copyright © 1995-2019 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Your questions
Slide 67