Post on 22-Jan-2020
transcript
Cheater Cheater
Pumpkin Eater
How to Beat the Cheat
PROTECT
ENFORCE
INVESTIGATE
Victoria Quinn-Stephens
Career Certifications
Program Manager
Cisco
Peggy Crowley
Anti-Piracy Program Manager
Microsoft
Liz Burns
EMC Proven Professional
Program Manager
EMC
John Fremer
President
Caveon Test Security
Session Moderator
PROTECTVictoria Quinn-Stephens
Career Certifications
Program Manager
Cisco
Cisco Certifications
• Three levels of certification:
– Associate (ex: Cisco Certified Network Associate - CCNA)
– Professional (ex: Cisco Certified Security Professional – CCSP)
– Expert: (ex: Cisco Certified Design Expert –CCDE)
• Target markets:
– Customers, Partners, Cisco Employees
– Educators and Students
• Certified one millionth candidate in February ’08
Cisco Certifications
Exam security policies driven by:
– Diverse global constraints and objectives
– Aspiration to maintain integrity of program
• Credibility with customers
• Value to employers
• Platform for career development
Ensuring that those who achieve a Cisco
certification possess the requisite skills and
knowledge
Cisco: Methods of Security Planning
Detection
Reporting
Communications
Prevention
Enforcement
Security
Threats
Brain
dumps
Proxy
TestingCheating
Cisco: Protect
• CHIP Security Pilot [China, Hong Kong, India & Pakistan]
– Test new channel security features in high risk countries
– Communicate stricter security standards in CHIP
– Measure results in a more controlled environment
– Survey candidate response to increased security
• Single vendor migration to VUE 8.1.07
– Partner for continuous security improvements
– Innovate new security solutions
– Scale security protections to market expansion
Cisco: Protect
A comprehensive
strategy that
integrates proactive
measures and best
practices.
• Harden testing channel
• Improve agreements
• Build better exams
• Institute continuous
improvement plan
• Support industry security
initiatives
Cisco: Protect
• Harden Testing Channel– Continual TCA training and re-certification
requirements
– Candidate authentication process [“CAP”]
– Real time data forensic enabled results hold feature
– Photo on score report
• Delayed [after forensic & psychometric analysis]
• Photo ID included on score report
– Certification credentials & photo verifications on web
– Increase volume of integrity shops
– Security country zones for test centers
– Systemic retake policy
– Test center security signage & tip line promotion
Cisco: Protect
• Improve Agreements– Candidate NDA
– Candidate testing rules
– Candidate conduct policy
– Candidate code of ethics
– Exam policies
– Retake policies
– Penalties and appeals process
– Test center policy & procedure guidelines
– Testing vendor contracts
Cisco: Protect
• Build better exams– Build more performance-based testing
– Add more complex simulations
– Randomize answer option [multiple choice]
– Clone items
• Increases amount of content that can be
developed in a shorter period of time
• Watermark cloned items – unique identification
– Increase rate of content development
• More content increases difficulty of memorization
– Localization of exam content
Cisco: Protect
• Institute continuous improvement plan– Create comprehensive security plan
– Implement yearly security audits
– Meet with cross-functional teams for input &
collaboration
– Critically assess customer sat, test center and
partner feedback or survey findings
– Schedule business reviews w/testing vendor
– Schedule security summits w/testing vendor
Cisco: Protect
• Support Industry Security Initiatives– Back industry’s leadership role to:
• Act as a clearinghouse
• Conduct outreach
• Police braindump activity
• Take enforcement actions
INVESTIGATE
Liz Burns
EMC Proven Professional
Program Manager
EMC
EMC Proven Professional
• Business realities that drive exam security
policies
– Global exam delivery
– VUE and Prometric for exam distribution
– Exam Volume
• 1000+ exams per month
– No authorized partner channel
– 50% EMC employees
EMC Investigative Approach
• To identify websites– Monthly
• Search EMC Exams on Google and eBay. Send take down request.
• Web Crawl report on new websites. Block from EMC network
• Forensic analysis on high volume exams. Various actions
– Daily• 24x7 web alerts. Various actions
• To identify individuals and test centers– Monthly
• Forensic analysis on high volume exams. Various actions
– Daily• 24x7 web alerts. Various actions.
Objective – Regular, Repeatable Processes and Measurable Results
Case Studies
• EMC sees a reduction in monthly exam
fraud incidents when we do regular
investigation and enforcement
• Case Study One – Using web alerts to
identify when, where and who steals exams
• Case Study Two – Using exams to identify
test takers who are using downloaded exams
1) EMC publishes to test vendors:
E20-500 on 9/10/07
E20-381 on 10/29/07
2) Both exams appear on web:
11/8/07
3) EMC orders
exams on 11/8/07
4) Monitor exam
registration activity
5) Exams stolen 11/10/08
from India testing center
Testing center
closed
Pros and Cons to Process• Cons
– Does not prevent current exam theft
• Pros– Does prevent future exam theft
• No new EMC exams on web since November 07
• When attempted to purchase other new exams the website operator replied:
“Dear:
Not technical problem, the reason from the test center. Whenever we download any EMC exam, after this EMC exam will be removed by prometric . So now we cannot got the emc exam data, so we also cannot do the exam for you. Thanks
Developed process to identify individuals
memorizing stolen exam content to pass exam
but who do not understand content
Monthly
forensics
Individuals Identified
and action taken
Pros and Cons to Process• Cons
– Again, does not prevent current exam theft
or cheating
• Pros
– Does identify individuals who should retest
or be banned from Program
– Does send a message to testing
community
ENFORCE
Peggy Crowley
Anti-Piracy Program Manager
Microsoft
Microsoft Learning APEducation
• To warn candidates of potential dangers
• To educate candidates of proper procedures
• To deter “bad guys”
Engineering• Striving to use innovations
in exam security
• Exploring other Microsoft technologies for delivery
Enforcement
• Legal Actions
• Security Issues
Intelligence
• Data Forensics
• Metrics to drive the other pillars
• Metrics to justify resources
Combating Piracy With…
Legal Enforcement Process
•Lead Tracking
•Lead Qualifying Process
•Test Purchase Program
•Actions•C&Ds
•Takedown Notices
•Internet Monitoring Program
•Auction Monitoring Program
•Lawsuits
Case Study: TestKing• Case filed August 2006
– Started as a John Doe suit
• Settled May 2007
– immediately cease marketing, selling, distributing, publishing, reproducing, disseminating, offering or otherwise knowingly transferring in any way any actual Microsoft Certification Exam content
– a permanent injunction prohibiting distribution of infringing material
– post the following notice on all affiliate websites: “Testking Materials do not contain actual questions and answers from Microsoft’s Certification Exams”
Questions?
Don’t forget to fill out your evaluations!