Proctoring Service API - IMS Global Learning Consortium · } Increasing interest in proctoring of...

transcript

Copyright©1995-2017Ques7onmarkCorpora7onand/orQues7onmarkCompu7ngLimited,knowncollec7velyasQues7onmark.Allrightsreserved.Ques7onmarkisaregisteredtrademarkofQues7onmarkCompu7ngLimited.Allothertrademarksareacknowledged.

ProctoringServiceAPISteveLay,Ques7onmark

}  Increasinginterestinproctoringofexamsdeliveredelectronically

}  Anumberofproctoringservicestochoosebetween}  ProctoringservicetypicallyseparatefromtheAssessmentManagementSystem§  Some7mesreferredtoasthe‘TestDriver’

}  TheProctoringServiceAPIaimstostandardizethetouch-points

Introduc7on

}  PhotoCaptureandCompare§  Theproctoringservicecapturesoneormorephotosofthecandidateduringtheexam

§  Photosarecomparedwithregistra7onphotosonfile§  Photosareavailableforlaterreview

}  RecordandReview§  Theproctoringservicecapturesacon7nuousaudioandvideostreamofthecandidateduringtheexam

§  Dataisprocessedandflaggedautoma7callyforanomalies§  Summarydataandfullstreamsareavailableforlaterreview

}  OnlineProctoring§  Thecandidateisconnectedwitharemotehumanproctoratthestartoftheexam

§  Theproctormonitorsthecandidatethroughouttheexamandrecordsanyanomalies

§  Summarydatafromproctorandsampleframesareavailableforlaterreview

ProctoringServiceExampleServiceTypes

Teddy Funger

ProctorExamteddy@proctorexam.com

Launching into an Assessment from the Proctoring PlatformWhat we’ve learned at ProctorExam

What are we trying to achieve?1. Security

a. The exam should only be accessible through the exam platformb. The exam attempt should always be correlated with the proctoring sessionc. The exam should not be accessible after the attempt

2. Ease of Usea. The student should take the least amount of steps to access the examb. The student should have explicit instructions to follow

3. Universalitya. The exam access procedure should be the same across exam platforms

The “Download Your Exam as a Document” method● Student can keep a copy of

the exam in any number of ways

● Teacher has to download exam document

● This isn’t our core competency!

The “Sign in to Blackboard” method● The student can access

the exam if the poke around blackboard enough

● The student can share their credentials with anyone

● Way too many steps!

The “Global Exam Password” method● Once the password leaks,

anyone can access the exam

● Exam session isn’t automatically linked to the proctoring session

The “Individual Password” method● Student can share their

individual credentials out of band if they are sneaky enough

The “Signed Exam Link” method● More difficult to

communicate through a side channel, as the link is fairly long

● Technically minded students can access credentials to use if there is also a login page

● Requires exam platform to provide mechanism to sign links

The famous “LTI Launch” method● Link points to url on

proctoring platform, which then redirects to a time limited(single use) signed LTI launch

● We can ensure that the student is sharing all streams before launching exam platform

}  ExamandQues7oncontent

}  Candidateresponses}  Detailednaviga7onpaths

}  Timinginforma7on

}  Photostakenduringtheexam}  Audio/Videocapturedduringtheexam

}  Screencaptures}  Applica7onusageinforma7oncapturedfromopera7ngsystem

}  Proctorincidentreports}  Flagsfromautomateddetec7onofanomalies

PerformanceDatavsExamSessionEvidence

PerformanceData ExamSessionEvidence

ProgramAdministratorRegistersCandidatesfor

theExamSetsProctoringRequirements

CandidateTakestheexam

ProctorOverseesExamSessionHelpscollectExamSessionEvidence

ReviewerReviewsExamSession

EvidenceMakesdecisionsabout

outcomes

}  Site-wideProctoringOp7ons§  Configuringproctoringop7onsthatwillapplytoallassessmentsdeliveredwithintheins7tu7on

}  AddProctoringRequirement§  Addproctoringrequirementtoanassessment

}  AssessmentSpecificProctoringOp7ons§  Modifydefaultproctoringop7onsforanindividualassessment

}  ScheduleanAppointment§  Pre-registerand/orscheduleanappointmentwithaliveproctororothertypeofresource-limitedproctoringservice

}  ScheduleBlocking§  Requestabookingfromproctoringsystemforagivennumberofcandidatesand7meslot

UseCases:Pre-exam

}  ValidateUserSystem§  ChecktechnologyrequirementforaspecificPC,laptoporotherdevice

}  LaunchtheExam§  Launchtheexamthroughtheproctoringservice

}  ProctorIntervenes§  Controlarunningexamtoprotectintegrityoftheprocess

}  ExternalProctorLaunchofProctoringSystem§  Usersintheconsumersystemactaslive-onlineproctorsintheproctoringtool(agentprovidedownproctors)

UseCases:In-Session

}  ExternalReviewerLaunchofProctoringSystem§  Reviewexcep7onsforanindividualassessment§  Reviewauditinforma7onforen7reassessmentprogram

UseCases:Post-Exam

AnatomyofaProctoringSolu7on

ProctoringAPI

CandidateCheck-in

Candidate’sBrowser

AMSCandidateExamDelivery

Proctor/ReviewerApplica7on

Proctor’swebbrowser

AMSProctorControls

APIDetails

IMSLearningToolsInteroperability(LTI)h_ps://www.imsglobal.org/ac7vity/learning-tools-interoperability

}  AlreadyusedbyLMScommunity}  Designedtosolvetheproblemof‘launching’onewebappfromanother§  ToolConsumerlaunchesToolProvider

}  Onestop-shop:§  User,contextandroleinforma7on§  Resourceplacement(e.g.,mul7pleexamstosamegroup)§  Authen7ca7onviaOAuthsigning§  Returnlinksforsmoothhand-back

}  What’smissing?§  Communica7onofworkflowinforma7on:

�  Timeconstraintsonresource(exam)availabilitysetbyAMS�  Stateofproctoringservice,e.g.,appointmentbookingphase,appointmentreadiness

§  SafewaytocommunicatetheExamLaunchinforma7on

WhyLTI?

ProctoringSolu7onCandidateflowinWebBrowser

CandidateCheck-intoProctoringService

LTILaunch Redirect

AMSCandidateExam

Delivery

AMSCandidatePortal

ProctoringServiceWS

ProctoringSolu7onCandidateflowwithSpecialProctoredApp

LTILaunch Redirect

AMSCandidateExam

Delivery

AMSCandidatePortal

ProctoringServiceWS

AppLaunch

ProctoringApp

AppLandingPage

ProctoringSolu7onProctorsregisteredwithProctoringSolu7on

iframeorwindow

Proctor’sBrowser

AMSProctorControls

ProctoringServiceProctorPageProctorControlslinkreceived

fromCandidatelaunch

LTILaunch Redirect

AMSCandidateExam

Delivery

AMSCandidatePortal

ProctoringSolu7onProctorsregisteredwithAMS

ProctoringServiceProctorPage

LTILaunchAMS

ProctorPortal

ProctoringServiceWS

iFrameorPop-up

Proctor’sBrowser

AMSProctorControls

ProctoringSolu7onReviewerflow

ProctoringServiceReviewPage

LTILaunchAMS

ReviewerPortal

ProctoringServiceWS

Reviewer’sBrowser

}  Reviewuse-casedocument:§  h_ps://drive.google.com/file/d/0B-xMrCxugbJxUDJHaHVXaUtoRmM/view

}  Fillinyourpriori7es:§  h_ps://docs.google.com/spreadsheets/d/1I6Dg6sqysumQk1XVGaX6FPfu2ramAsi804B2hGPbu1Q/edit?usp=sharing

Whatnext?

Jointheteam…

Proctoring Service API - IMS Global Learning Consortium · } Increasing interest in proctoring of...

Documents