Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProctoringServiceAPISteveLay,Ques7onmark
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} Increasinginterestinproctoringofexamsdeliveredelectronically
} Anumberofproctoringservicestochoosebetween} ProctoringservicetypicallyseparatefromtheAssessmentManagementSystem§ Some7mesreferredtoasthe‘TestDriver’
} TheProctoringServiceAPIaimstostandardizethetouch-points
Introduc7on
Slide2
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} PhotoCaptureandCompare§ Theproctoringservicecapturesoneormorephotosofthecandidateduringtheexam
§ Photosarecomparedwithregistra7onphotosonfile§ Photosareavailableforlaterreview
} RecordandReview§ Theproctoringservicecapturesacon7nuousaudioandvideostreamofthecandidateduringtheexam
§ Dataisprocessedandflaggedautoma7callyforanomalies§ Summarydataandfullstreamsareavailableforlaterreview
} OnlineProctoring§ Thecandidateisconnectedwitharemotehumanproctoratthestartoftheexam
§ Theproctormonitorsthecandidatethroughouttheexamandrecordsanyanomalies
§ Summarydatafromproctorandsampleframesareavailableforlaterreview
ProctoringServiceExampleServiceTypes
Slide3
Slide5
Launching into an Assessment from the Proctoring PlatformWhat we’ve learned at ProctorExam
What are we trying to achieve?1. Security
a. The exam should only be accessible through the exam platformb. The exam attempt should always be correlated with the proctoring sessionc. The exam should not be accessible after the attempt
2. Ease of Usea. The student should take the least amount of steps to access the examb. The student should have explicit instructions to follow
3. Universalitya. The exam access procedure should be the same across exam platforms
The “Download Your Exam as a Document” method● Student can keep a copy of
the exam in any number of ways
● Teacher has to download exam document
● This isn’t our core competency!
The “Sign in to Blackboard” method● The student can access
the exam if the poke around blackboard enough
● The student can share their credentials with anyone
● Way too many steps!
The “Global Exam Password” method● Once the password leaks,
anyone can access the exam
● Exam session isn’t automatically linked to the proctoring session
The “Individual Password” method● Student can share their
individual credentials out of band if they are sneaky enough
The “Signed Exam Link” method● More difficult to
communicate through a side channel, as the link is fairly long
● Technically minded students can access credentials to use if there is also a login page
● Requires exam platform to provide mechanism to sign links
The famous “LTI Launch” method● Link points to url on
proctoring platform, which then redirects to a time limited(single use) signed LTI launch
● We can ensure that the student is sharing all streams before launching exam platform
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} ExamandQues7oncontent
} Candidateresponses} Detailednaviga7onpaths
} Timinginforma7on
} Photostakenduringtheexam} Audio/Videocapturedduringtheexam
} Screencaptures} Applica7onusageinforma7oncapturedfromopera7ngsystem
} Proctorincidentreports} Flagsfromautomateddetec7onofanomalies
PerformanceDatavsExamSessionEvidence
Slide13
PerformanceData ExamSessionEvidence
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProgramAdministratorRegistersCandidatesfor
theExamSetsProctoringRequirements
CandidateTakestheexam
ProctorOverseesExamSessionHelpscollectExamSessionEvidence
ReviewerReviewsExamSession
EvidenceMakesdecisionsabout
outcomes
Roles
Slide14
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} Site-wideProctoringOp7ons§ Configuringproctoringop7onsthatwillapplytoallassessmentsdeliveredwithintheins7tu7on
} AddProctoringRequirement§ Addproctoringrequirementtoanassessment
} AssessmentSpecificProctoringOp7ons§ Modifydefaultproctoringop7onsforanindividualassessment
} ScheduleanAppointment§ Pre-registerand/orscheduleanappointmentwithaliveproctororothertypeofresource-limitedproctoringservice
} ScheduleBlocking§ Requestabookingfromproctoringsystemforagivennumberofcandidatesand7meslot
UseCases:Pre-exam
Slide15
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} ValidateUserSystem§ ChecktechnologyrequirementforaspecificPC,laptoporotherdevice
} LaunchtheExam§ Launchtheexamthroughtheproctoringservice
} ProctorIntervenes§ Controlarunningexamtoprotectintegrityoftheprocess
} ExternalProctorLaunchofProctoringSystem§ Usersintheconsumersystemactaslive-onlineproctorsintheproctoringtool(agentprovidedownproctors)
UseCases:In-Session
Slide16
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} ExternalReviewerLaunchofProctoringSystem§ Reviewexcep7onsforanindividualassessment§ Reviewauditinforma7onforen7reassessmentprogram
UseCases:Post-Exam
Slide17
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
AnatomyofaProctoringSolu7on
Slide18
ProctoringAPI
CandidateCheck-in
Candidate’sBrowser
AMSCandidateExamDelivery
Proctor/ReviewerApplica7on
Proctor’swebbrowser
AMSProctorControls
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
APIDetails
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
IMSLearningToolsInteroperability(LTI)h_ps://www.imsglobal.org/ac7vity/learning-tools-interoperability
Slide20
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} AlreadyusedbyLMScommunity} Designedtosolvetheproblemof‘launching’onewebappfromanother§ ToolConsumerlaunchesToolProvider
} Onestop-shop:§ User,contextandroleinforma7on§ Resourceplacement(e.g.,mul7pleexamstosamegroup)§ Authen7ca7onviaOAuthsigning§ Returnlinksforsmoothhand-back
} What’smissing?§ Communica7onofworkflowinforma7on:
� Timeconstraintsonresource(exam)availabilitysetbyAMS� Stateofproctoringservice,e.g.,appointmentbookingphase,appointmentreadiness
§ SafewaytocommunicatetheExamLaunchinforma7on
WhyLTI?
Slide21
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProctoringSolu7onCandidateflowinWebBrowser
Slide22
CandidateCheck-intoProctoringService
LTILaunch Redirect
Candidate’sBrowser
AMSCandidateExam
Delivery
AMSCandidatePortal
ProctoringServiceWS
GET
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProctoringSolu7onCandidateflowwithSpecialProctoredApp
Slide23
CandidateCheck-intoProctoringService
LTILaunch Redirect
Candidate’sBrowser
AMSCandidateExam
Delivery
AMSCandidatePortal
ProctoringServiceWS
GET
AppLaunch
ProctoringApp
AppLandingPage
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProctoringSolu7onProctorsregisteredwithProctoringSolu7on
Slide24
iframeorwindow
Proctor’sBrowser
AMSProctorControls
ProctoringServiceProctorPageProctorControlslinkreceived
fromCandidatelaunch
CandidateCheck-intoProctoringService
LTILaunch Redirect
Candidate’sBrowser
AMSCandidateExam
Delivery
AMSCandidatePortal
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProctoringSolu7onProctorsregisteredwithAMS
Slide25
ProctoringServiceProctorPage
LTILaunchAMS
ProctorPortal
ProctoringServiceWS
GET
iFrameorPop-up
Proctor’sBrowser
AMSProctorControls
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
ProctoringSolu7onReviewerflow
Slide26
ProctoringServiceReviewPage
LTILaunchAMS
ReviewerPortal
ProctoringServiceWS
GET
Reviewer’sBrowser
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
} Reviewuse-casedocument:§ h_ps://drive.google.com/file/d/0B-xMrCxugbJxUDJHaHVXaUtoRmM/view
} Fillinyourpriori7es:§ h_ps://docs.google.com/spreadsheets/d/1I6Dg6sqysumQk1XVGaX6FPfu2ramAsi804B2hGPbu1Q/edit?usp=sharing
Whatnext?
Slide27
Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.
Jointheteam…
Slide28