9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
EHRUsabilityTestReport
Product:ChiroTouch
Version:7.0
Reportbasedon(NISTIR7742)CustomizedCommonIndustryFormatTemplateforElectronicHealthRecordUsabilityTesting
DateofUsabilityTest: October11,2017toOctober13,2017
DateofReport: October16,2017
ReportPreparedBy:
TheUsabilityPeople,LLC.
703-687-6300 [email protected] 4000LegatoRoad,Suite1100 Fairfax,VA22033
Note:ThefollowingstudywasdevelopedusingtheNISTIR7742templateasaguideforreportingourfindings:CustomizedCommonIndustryFormatTemplateforElectronicHealthRecordUsabilityTesting.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ExecutiveSummaryAusabilitytestofChiroTouchVersion7.0,acompleteEHRwasconductedonOctober11toOctober13th,2017InavirtualenvironmentbyIntegratedPracticeSolutions,Inc.Thepurposeofthistestwastotestandvalidatetheusabilityofthe2015EHRuserinterface,andtoprovideevidenceofthisusabilityintheChiroTouchsoftware.
Duringtheusabilitytest,8chiropracticprovidersand2healthcareprofessionals(EHRusers)servedasparticipantsandusedtheEHRinsimulated,butrepresentativetasks.
Thisstudycollectedperformancedataon18taskstypicallyconductedonanEHR.Thetasksfocusedonthetwelve(12)2015EditioncertificationcriteriaspecifiedbyONC:
• Section170.315(a)(1)Computerizedproviderorderentry–medications• Section170.315(a)(2)Computerizedproviderorderentry–laboratory• Section170.315(a)(3)Computerizedproviderorderentry–diagnosticimaging• Section170.315(a)(4)Drug-drug,drug-allergyinteractionchecks• Section170.315(a)(5)Demographics• Section170.315(a)(6)Problemlist• Section170.315(a)(7)Medicationlist• Section170.315(a)(8)Medicationallergylist• Section170.315(a)(9)Clinicaldecisionsupport• Section170.315(a)(14)Implantabledevicelist• Section170.315(b)(2)Clinicalinformationreconciliationandincorporation• Section170.315(b)(3)Electronicprescribing
Duringthe60to90-minuteone-on-oneusabilitytest,eachparticipantwasgreetedbytheadministratorandaskedtoreviewandsignaninformedconsent/releaseform;theywereinstructedthattheycouldwithdrawatanytime.
MostparticipantshadpriorexperiencewiththeEHR.Theywerealsoprovidedtrainingmaterials(usermanualdocumentation)priortothetesttointroducethemtonewconcepts.
Theadministratorintroducedthetest,andinstructedparticipantstocompleteaseriesoftasks(givenoneatatime)usingtheChiroTouchsoftware.Duringthetesting,theadministratortimedthetestand,alongwiththedatalogger(s)recordeduserperformancedataonpaperandelectronically.Theadministratordidnotgivetheparticipantassistanceinhowtocompletethetask.Eachsessionwasrecordedforsubsequentanalysis.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TestLogisticsAllparticipantsweretestedontheChirotouchsystemduringremoteconferencingsessionsusingGotoMeeting.
TheGotoMeetingusabilitytestsessionwasconductedbyatestadministratorfromthetestingorganization(TheUsabilityPeople,LLC)workingfromasmallconferenceroomatTheUsabilityPeople’sFairfax,VAlocation.Adataloggerfromthetestorganizationalsotookdetailednotesoneachsession,includingusercommentsandotherratingsfollowingeachtask.Duringasessionboththetestadministratorandthedatalogger(s)couldseeonlytheparticipant’sscreenandheartheparticipant’scomments,questions,andresponses.
ChiroTouchwasevaluatedforeffectiveness,efficiencyandsatisfaction(ISO9241-11)asdefinedbythefollowingmeasurescollectedandanalyzedforeachparticipant:
• Numberoftaskssuccessfullycompletedwithintheallottedtimewithoutassistance• Timetocompletethetasks• Numberandtypesoferrors• Pathdeviations• Participant’sverbalizations
Participant’ssatisfactionratingsofthesystem.
ThefollowingisasummaryoftheperformanceandratingdatacollectedontheEHR.
Deviations Satisfaction
Task Participants TaskSuccess
TimetoComplete(Avg)
Total AverageAverage(outof5)
Task1DemographicInformation10 100% 1:49
60.60 4.20
Task2ProblemListView/Intervention10 100% 0:42
111.10 4.20
Task3ProblemListIntervention-Remove10 100% 0:57
40.40 3.40
Task4ClinicalInfoRecon/Incorporation10 70% 1:49
191.90 3.10
Task5MedicationAllergy-Add10 90% 1:33
171.70 3.50
Task6Drug-AllergyInteractionCheck10 100% 1:02
50.50 3.50
Task7ElectronicPrescribe10 100% 1:49
20.20 3.80
Task8eRXCancel/Change10 90% 2:46
111.10 3.90
Task9eRX-Renew10 100% 0:45
40.40 4.60
Task10MedOrder;Drug-DrugIntervention10 100% 1:39
10.10 4.30
Task11Medicationlist-Stop10 100% 0:21
00.00 4.80
Task12MedAllergyList-Remove10 100% 0:27
00.00 4.70
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TheresultsfromtheSystemUsabilityScalescoredtheoverallsubjectivesatisfactionwiththesystembasedonperformancewiththesetaskstobe:53.
Inadditiontotheperformancedata,thefollowingqualitativeobservationsweremade:Inadditiontotheperformancedata,thefollowingqualitativeobservationsweremade:
1. MajorFindingsTheoveralltaskperformanceforthisstudywasveryhigh,withanoveralltaskcompletionrateofninety-seven(97)percent.Althoughtheperformancedatawasveryhigh,andtheindividualtaskperformanceratingwasalsohigh,thesubjectiveusabilityratingscollectedviatheSUSscalewerelowerthanexpected.
2. AreasforImprovementThissystem,likemanyotherEHRswouldbenefitfromseveralminorenhancementstotheuserexperience.ThetaskthatcreatedthemostnumberofissuesfortheparticipantswastheClinicalInformationReconciliationtask.Thistask,andseveralothers,couldbemadetobeeasiertoperformbyadditionsomeadditionalworkflowindicationtohelp“guide”theuserstowardscompletionofthetask.Severalotherareasforimprovementarediscussedlaterinthisdocument
Task13CPOEDiagImaging10 100% 1:44
10.10 4.10
Task14CPOEDiagImaging-Modify10 100% 1:18
60.60 4.30
Task15CPOELabOrders10 100% 1:36
10.10 4.20
Task16CPOELabOrders-Modify10 100% 0:59
30.30 4.70
Task17ImplantableDevice-Add10 100% 0:59
90.90 4.20
Task18ImplantableDevice-Modify10 100% 0:48
101.00 4.20
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
IntroductionTheEHRtestedforthisstudywasChiroTouchVersion7.0,acompleteEHR.Thissystemisdesignedtorepresentacompleteandcomprehensivedigitalversionofapatient’spaperchartwithreal-time,patient-centeredrecordsandpresentmedicalinformationtohealthcareprovidersinanambulatorysetting.ThissystemcomplieswithallONCcertificationcriteriaandisbeingsubmittedforverificationofcompliancewith2015certificationcriteria.Theusabilitytestingwasconductedwitheveryefforttorepresentrealisticexercisesandconditions.
Thepurposeofthisstudywastotestandvalidatetheusabilityofthecurrentuserinterface,andprovideevidenceofusabilityintheChiroTouchEHRsoftware.Tothisend,measureofeffectivenessandefficiency(timetoperformtasks;totalnumberofdeviations;totalnumberoferrors;etc.)werecapturedduringtheusabilitytesting.
Method
ParticipantsAtotalof10participantsweretestedontheChiroTouchEHRSoftware.Participantsinthetestwereindividualsthatworkwithinanambulatoryhealthcareenvironment.ParticipantswerecontactedbyChiroTouchdirectlyandfromadatabaseofmedicalprofessionalsmaintainedbythetestingorganization(TheUsabilityPeople,LLC).Inaddition,participantshadnodirectconnectiontothedevelopmentofororganizationproducingtheChiroTouchEHRSoftware.ParticipantswerenotfromIntegratedPracticeSolutions,Inc.Participantsweregiventheopportunitytohavethesameorientationandleveloftrainingastheactualenduserswouldhavereceived.
Forthetestpurposes,end-usercharacteristicswereidentifiedandtranslatedintoarecruitmentscreenerusedtosolicitpotentialparticipants.AnexampleofascreenerisprovidedinAppendix1.
Recruitedparticipantshadamixofbackgroundsanddemographiccharacteristicsconformingtotherecruitmentscreener.Thefollowingisatableofparticipantsbycharacteristics,includingdemographics,professionalexperience,userrole,userneedsforassistivetechnology,andproductexperience.ParticipantnameswerereplacedwithParticipantIDssothatanindividual’sdatacannotbetiedbacktoindividualidentities.
ParticipantID
Age Education Gender ProfessionalExperience
AssistiveTechnologyNeeds
UserRole ComputerExperience(years)
ProductExperience(Years)
1 P0101 50to59
Bachelor’sdegree Female 20
None Officemanager
353
2 P010250to59
Somecollegecredit,nodegree Female 30
None
Officemanager
20
23 P0103
40to49
Tradetechnicalvocationaltraining Female 22
None
Officemanager
15
44 P0104 50to
59Doctorate
degree(e.g., Female 35None Chiropractor
154
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Atotaloftwelve12participantsrecruitedforthetestshoweduptoparticipateinthetest.Two(2)participantsfailedtoshowforthestudy.
Participantswerescheduledforsixty(60)toninety(90)minutesessionswithuptothirty(30)minutesinbetweeneachsessionfordebriefbytheadministrator(s)anddatalogger(s),andtoresetsystemstopropertestconditions.Aspreadsheetwasusedtokeeptrackoftheparticipantschedule,andincludedeachparticipant’sdemographiccharacteristicsasprovidedbythetestingfirm.
StudyDesignOverall,theobjectiveofthistestwastouncoverareaswheretheapplicationperformedeffectively,efficiently,andwithsatisfaction,andareaswheretheapplicationfailedtomeettheneedsoftheparticipants.ThedatafromthistestmayserveasabaselineforfuturetestswithanupdatedversionofthesameEHRand/orcomparisonwithotherEHRsprovidedthesametasksareused.Inshort,thistestingservesasbothameanstorecordorbenchmarkcurrentusability,butalsotoidentifyareaswhereimprovementsmustbemade.
Duringtheusabilitytest,participantsinteractedwithone(1)EHR.Eachparticipantwasprovidedthesamesetofinstructions.Thesystemwasevaluatedforeffectivenessandefficiencyasdefinedbymeasurescollectedandanalyzedforeachparticipant.
• Timetocompletethetask• Numberandtypesoferrors• Pathdeviations• Participantsverbalizations• Taskscompletedintheallottedtime• Participant’ssatisfactionratingsofthesystem• Numberoftaskscompletedsuccessfullywithintheallottedtimewithoutassistance
MD,DNP,DMD,PhD)
5 P0105 20to29
Bachelor’sdegree Female 5
None OfficeManager
152
6 P0106 40to49
Associatedegree Female 12
None
ChiropracticAssistant
202
7 P0107
50to59
Doctoratedegree(e.g.,MD,DNP,DMD,PhD) Male 27
None
Owner/chiropractor
15
38 P0108 20to
29 NursingDegree Female 9None RN
150
9 P0109
60to69
Highschoolgraduate,
diplomaortheequivalent Female 7
None
Ins.ClerkandCA
17
310 P0110 40to
49MastersDegree Male 4
None HealthITInstructor
150
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
AnumberoftaskswereconstructedthatwouldberealisticandrepresentativeofthekindsofactivitiesausermightdowiththisEHR,including:
• ModifyPatientDemographicinformation• Addanitemtopatientproblemlist• Removeanitemfrompatientproblemlist• Reconcileandintegrateinformationfromanotherprovider• AddamedicalAllergy• Prescribeadrug/ViewCDS• Prescribeadrugw/oCDSIntervention• CancelaneRX• RenewaneRX• Orderamedication;viewCDS• Stopamedication• Removeanallergy• Createimagingorders• ModifyImagingorder• CreateLaborder• ModifyLaborder• AddanImplantableMedicalDevice• ModifyanImplantableMedicalDevice
Taskswereselectedbasedontheirfrequencyofuse,criticalityoffunction,andthosethatmaybemosttroublesomeforusers.
ProceduresUponcommencementoftest,participantsweregreeted,remotelyloggedin,andtheiridentitywasverified.ParticipantswerethenassignedaparticipantID.Allparticipantssignedaninformedconsentformpriortothetesting(SeeAppendix1).Thetestadministratorprovidedtheinstructionsforeachtest,notedallcommentsfromtheparticipants;whilethedataloggernotedalltimes,pathdeviations,comments,successes,anderrors.
Participantswereinstructedtoperformthetasks:
• Afterlisteningtotheinstructionsfromthetestingadministrator• Asquicklyaspossiblewithasfewdeviationsorerrorsaspossible• Withoutassistance–participantswereallowedtogiveclarificationontasks,butnotguidanceonuse• Withoutusingathinkaloudtechnique
Foreachtask,theparticipantswereshownacopyofthetaskonthescreenandwerefreetoswitchback-and-forthbetweenthetaskandtheEHR.Tasktimingbeganafterthecompletionoftheverbalinstructionsfromtheadministrator;andafteranacknowledgementfromtheparticipantthattheywerereadytobegin.Thetasktimewasstoppedoncetheparticipantindicatedtheyhadsuccessfullycompletedthetask.
Followingthetest,theadministratorgavetheparticipantthepost-testquestionnaire;andthenthankedthemfortheirtime.Participants'demographicinformation,tasksuccessrate,timeontask,errors,deviations,verbalresponses,andpost-testquestionnairewererecordedintoaspreadsheet.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TheUsabilityPeople,LLCstaffmemberswereresponsibleforloggingdatarecordedallparticipants’demographicinformation,tasksuccessrate,timeontask,errors,comments,anddeviationsintoaspreadsheet.
TestLocationThetestwasadministeredinaremotesettingwhereparticipantswereisolatedfromotherparticipantsinthestudy.Onlythetestadministratorandloggerwereonthecallwiththeparticipantswhilethestudywasbeingadministered.
TestEnvironmentTheChiroTouchEHRSoftwarewouldbetypicallybeusedinaprivatepractice,andinthisinstancethetestingwasconductedremotelyineachparticipant’sactualpractice.Althoughthetypeofcomputer,operatingsystemanddisplayresolutionoftheremoteparticipantsystemwasunknown,thesystemthatwasusedbythetestadministratorandcontrolledbytheremoteparticipantwasaDellInspironLaptoprunningtheWindows10operatingsystemataresolutionof1366x768pixels.UsersalsousedamouseandkeyboardwhileinteractingwiththeEHR.
TheChiroTouchapplicationisaprimarilyclient-serversolutionwithsomewebcomponents;andallcomputerswererunningonhighspeedinternetconnections.Allparticipantsindicatedthatsystemperformanceduringthetestwaswhattheywereusedtoseeingduringtheirtypicalworkday.Additionally,participantswereinstructednottochangeanyofthedefaultsystemsettings(suchascontroloffontsize).
TestFormsandToolsDuringtheusabilitytest,variousinstrumentsanddocumentswereused,including:
• Informedconsent• ModeratorGuide• Post-testquestionnaire• Study
ExamplesofthesedocumentscanbelocatedintheAppendixsection.Theparticipant’sinteractionwiththeChiroTouchEHRSoftwarewascapturedandrecordeddigitallywithscreencapturesoftwarerunningonthemoderator’smachine.
ParticipantInstructionsTheAdministratorreadthefollowinginstructionsaloudtoeachparticipant:
“Thankyouforparticipatingintoday’susabilitystudyofChiroTouch.Inafewminutes,youwillbeaskedtoperformaseriesoftasksandcompleteausersurvey.Pleaseattempttocompletethesetasksasquicklyaspossible.ThepurposeofthisstudyisforChiroTouchtoobtaininformationonwhereenhancementsareneededintheapplicationbasedonhowquicklyandeasilythetaskisbeingperformedintheChiroTouchEHRsystem.
Pleasenotethatwearenottestingyou;wearetestingthesystem.Thereforeifyouhaveanydifficultythismaymeanthatsomethingneedstobeimprovedinthesystem.Iwillbehereincaseyouneedspecifichelp,butIamnotabletoinstructyouorprovidehelpinhowtousetheapplication.
Overall,weareinterestedinhoweasy(orhowdifficult)thissystemistouse,whatinitwouldbeusefultoyou,andhowwecouldimproveit.Ididnothaveanyinvolvementinitscreation,sopleasebehonestwithyouropinions.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Alloftheinformationthatyouprovidewillbekeptconfidentialandyournamewillnotbeassociatedwithyourcommentsatanytime.Shouldyoufeelitnecessaryyoumaywithdrawatanytimeduringtesting.”
Participantswerethengiven18taskstocomplete.TasksarelistedintheDataAppendix3.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
UsabilityMetricsAccordingtotheNISTGuidetotheProcessesApproachforImprovingtheUsabilityofElectronicHealthRecords,EHRsshouldsupportaprocessthatprovidesahighlevelofusabilityforallusers.Thegoalisforuserstointeractwiththesystemeffectively,efficiently,andwithanacceptablelevelofsatisfaction.Tothisend,metricsforeffectiveness,efficiencyandusersatisfactionwerecapturedduringtheusabilitytesting.
Thegoalsofthistestweretoassess:
1. TheefficiencyofChiroTouchbymeasuringthelengthoftimeittakesforausertocompletethetask;andthesuccessoftaskcompletion.
2. TheefficiencyofChiroTouchbymeasuringthepathdeviationstakenbytheuserduringthetask.3. TheefficiencyoftheChiroTouchbymeasuringtheaveragetasktimeandpathdeviations.4. TheeffectivenessofChiroTouchbymeasuringthenumberandtypesoferrorsexperiencedbytheuserduringthetask.5. ThesatisfactionoftheuserwithChiroTouchbyloggingtheircommentsonthetask.
DataScoringThetablebelowdetailshoweachtaskwasscored.
Measure RationaleandScoring
TaskTime Timingstartedwhentheadministratorsaid‘Begin’.Thetimeendedwhentheparticipantsaid‘Done’.Intheeventthattheparticipantfinished,anddidnotsay‘Done’,theadministratorstoppedtheclockwhenitwascleartheparticipanthadcompletedthetask.Tasktimewasonlycountediftheparticipantcompletedthetaskintheallottedtime.Averagetimepertaskwascalculatedforeachtask.
Errors Thetaskresultedinanerroriftheparticipant:failedtofinishthetaskintheallottedtime;or,iftheybecame‘stuck’andcouldnotproceedwithoutaskingforassistance.Tasktimewasnotcountedwhenthetaskresultedinanerror.
PathDeviations Pathdeviationswererecordedasactionstakenduringthetaskthatwerenotpartofthenecessaryactionsneededtocompletethetask.
Wecalculatedpathdeviationsbytakingthetotalnumberofobserveddeviationsanddividingthatnumberbythetotalnumberofstepstakenusinganoptimalpathtoprovidearatioofpathdeviation.
TaskSuccess Thetaskwasconsideredasuccessiftheparticipantcompletedthetaskintheallottedtimewithoutassistance.
TaskRating Participant’ssubjectiveimpressionoftheeaseofusewasmeasuredbyadministeringapost-sessionquestionnaireandapost-taskquestion.Theparticipantswereaskedtoratetheoveralleaseofeachtask on a scale of 1 (Very Difficult) to 5 (Very Easy) To measure participants’ confidence in andlikeabilityoftheoverallsystem,thetestingteamadministeredapost-testquestionnaire.Questionsincludedthoseaddressingtheeaseofuseofthesystemandparticipants’thoughtsonhowquicklyausercouldlearnthesystem.SeefullquestionnaireinAppendix2.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Results
DataAnalysisandReportingTheresultsoftheusabilitytestwerecalculatedaccordingtothemethodsspecifiedintheUsabilityMetricssectionabove.
AsummaryofthetestingresultsforChiroToucharedetailedbelow.Thetablebelowidentifiesthetasksperformedandtheperformancelevelandsatisfactionforeachtask.
Task MeanTaskTime
SD
CompletionRate(%)
Mean#PathDeviations
SD
MeanTaskSatisfaction
SD
Task1DemographicInformation 1:49 0:54 100% 0.60 0.66 4.20 0.98
Task2ProblemListView/Intervention 0:42 0:10 100% 1.10 0.70 4.20 0.75
Task3ProblemListIntervention-Remove 0:57 0:37 100% 0.40 0.92 3.40 1.20
Task4ClinicalInfoRecon/Incorporation 1:49 1:01 70% 1.90 1.81 3.10 1.45
Task5MedicationAllergy-Add 1:33 0:40 90% 1.70 1.62 3.50 1.02
Task6Drug-AllergyInteractionCheck 1:02 0:21 100% 0.50 0.81 3.50 1.20
Task7ElectronicPrescribe 1:49 0:26 100% 0.20 0.40 3.80 1.08
Task8eRXCancel/Change 2:46 0:40 90% 1.10 1.87 3.90 0.94
Task9eRX-Renew 0:45 0:42 100% 0.40 0.92 4.60 0.92
Task10MedOrder;Drug-DrugIntervention 1:39 0:22 100% 0.10 0.30 4.30 0.64
Task11Medicationlist-Stop 0:21 0:07 100% 0.00 0.00 4.80 0.40
Task12MedAllergyList-Remove 0:27 0:12 100% 0.00 0.00 4.70 0.64
Task13CPOEDiagImaging 1:44 0:28 100% 0.10 0.30 4.10 0.83
Task14CPOEDiagImaging-Modify 1:18 0:21 100% 0.60 0.66 4.30 0.64
Task15CPOELabOrders 1:36 0:43 100% 0.10 0.30 4.20 0.75
Task16CPOELabOrders-Modify 0:59 0:15 100% 0.30 0.64 4.70 0.46
Task17ImplantableDevice-Add 0:59 0:24 100% 0.90 0.83 4.20 0.60
Task18ImplantableDevice-Modify 0:48 0:20 100% 1.00 1.18 4.20 0.75
Asthetableaboveshows,relativetooptimalperformancestandardsasdefinedbyChiroTouchandTheUsabilityPeople,
participantperformanceintheChiroTouch EHRusabilitytestwasquitesatisfactory.Theoverallaveragetaskcompletionrate
wasninety-seven(97)percent.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ReliabilityDuringtheentiredatacollectionphase,itwasobservedthatthesystemprovidedaconsistentandreliableinterfacetoeachparticipantastheycompletedtheirtasks.Aseachparticipantcompletedtheirassignedtasks,thesystemprovidedthesameinformationandrespondedtotheirinputwiththesameverbiageandusingthesamemodeofcommunication(e.g.Pop-upmessage,orembeddedassistance).
RiskAssessmentThefollowingtablepresentsaprioritizedlistoftasks(alongwithadescriptionofthetaskandanidentificationofthecriteria
evaluated)prioritizedbytheriskoferrorasobservedduringthetesting.
TaskCriteria
Evaluated DescriptionPercentComplete RiskStatus
Task1DemographicInformation a(5) ModifyPatientDemographicinformation 100% None
Task2ProblemListView/Intervention a(6) Addanitemtopatientproblemlist 100% None
Task3ProblemListIntervention-Remove a(6) Removeanitemfrompatientproblemlist 100% Low
Task4ClinicalInfoRecon/Incorporation b(2)Reconcileandintegrateinformationfromanotherprovider 70% Moderate
Task5MedicationAllergy-Add a(8) AddamedicalAllergy 90% Low
Task6Drug-AllergyInteractionCheck a(1);a(9) Prescribeadrug/ViewCDS 100% None
Task7ElectronicPrescribeb(3);a(4);
a(9) Prescribeadrugw/oCDSIntervention 100% None
Task8eRXCancel/Change b(3) CancelaneRX 90% Low
Task9eRX-Renew b(3) RenewaneRX 100% None
Task10MedOrder;Drug-DrugInterventiona(7);a4;a(9) Orderamedication;viewCDS 100% None
Task11Medicationlist-Stop a(7) Stopamedication 100% None
Task12MedAllergyList-Remove a(8) Removeanallergy 100% None
Task13CPOEDiagImaging a(3) Createimagingorders 100% None
Task14CPOEDiagImaging-Modify a(3) ModifyImagingorder 100% None
Task15CPOELabOrders a(2) CreateLaborder 100% None
Task16CPOELabOrders-Modify a(2) ModifyLaborder 100% None
Task17ImplantableDevice-Add a(14) AddanImplantableMedicalDevice 100% None
Task18ImplantableDevice-Modify a(14) ModifyanImplantableMedicalDevice 100% None
EffectivenessParticipantsinthestudyexperiencederrorsinallbuttwo(2)tasks.However,thebulkoftheerrorsoccurredinthreeareas:“ClinicalInformationReconciliation”,“AddanAllergy”,and“Cancelanelectronicprescription.”ErrorsfoundintheseareaswereofnosurprisetoChiroTouch.TheClinicalInformationReconciliationapplicationisanewercomponenttoChiroTouchusers.Thisfeature,aswellastheallergyandeRXfeaturesarenotusedtypicallybythethoseintheChiropracticprofession.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
EfficiencyParticipantswhosuccessfullycompletedtasksgenerallycompletedthosetaskswithinanacceptabletime.Sometaskswerecompletedmorequicklythanthecalculatedoptimaltime,whileseveraltaskstookslightlylongerthanexpected.Thetasksthattookthelongestrequiredtheparticipantstonavigatetoaparticularunfamiliarportionofapage,interactwithaworkflow,locateandselectspecificactions.
Someparticipantsmadeerrorswhenattemptingtonavigatetowardsolvingtheirassignedtasks.TheseerrorsmaybeassociatedwiththoseparticipantsnotbeingfamiliarandnotunderstandingthepresentedinformationarchitectureoftheEHRsystem.Asnotedabove,priorexperiencewithEHRsystemswasrelatedtosuccessfultaskcompletion.Similarly,experienceandpracticewiththegivensystemmayhavepositiveeffectswithregardtouserefficiency.
SatisfactionIndividualTaskSatisfaction
Participantsverballyindicatedtheirsatisfactionwiththeeaseofuseforeachtaskusingascaleof“1”(“VeryDifficult”)to“5,”
(“VeryEasy”).Asthefigurebelowshowsindividualtasksatisfactionrangedfromalowof3.1outof5onTask4(Clinical
InformationReconciliation)toahighof4.8outof5onTasks11(Medicationlist–Stopamedication).Theoverallaveragetask
satisfactionwas4.09indicatingthatoveralltheparticipantswerequitesatisfiedwiththeirtasks.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
SystemUsabilityScale
TheSystemUsabilityScale(SUS)isasimple,10-itemLikert-typeattitudescaleprovidingaglobalsubjectiveassessmentof
usabilityfromtheuser’sperspective(JohnBrookeatDigitalEquipmentCompanydevelopedtheSUSin1986).TheSUSscaleis
scoredfrom0to100;scoresunder60representsystemswithlessthanoptimalusability,scoresover80areconsideredbetter
thanaverage.
ThemeantotalSUSscorefortheChiroTouch EHRwasfifty-three(53)andrangedfromalowofthirty(30)andahighofninety
(68).Overall,participant-usersratedtheirsatisfactionwiththeChiroTouch EHRsystemtobelowerthantheexpectedrange
giventhehighindividualsatisfactionratingsandexcellenttaskperformancedata.
ThefollowinggraphshowstheSUSscorebyeachparticipant:
0.00 1.00 2.00 3.00 4.00 5.00
Task1DemographicInformationTask2ProblemList…Task3ProblemList…Task4ClinicalInfo…
Task5MedicationAllergy- AddTask6Drug-AllergyInteraction…
Task7ElectronicPrescribeTask8eRXCancel/Change
Task9eRX- RenewTask10MedOrder;Drug-Drug…Task11Medicationlist- Stop
Task12MedAllergyList-…Task13CPOEDiagImagingTask14CPOEDiagImaging-…
Task15CPOELabOrdersTask16CPOELabOrders- ModifyTask17ImplantableDevice- Add
Task18ImplantableDevice-…
4.204.20
3.403.103.503.503.803.904.604.304.804.70
4.104.304.204.70
4.204.20
AverageTaskSatisfaction
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Onthepost-studyquestionnaire4out8ofexperiencedChirotouchusersexpressedtheywere“VeryorSomewhatSatisfied”withtheChiroTouchsystem.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
MajorFindings
Theoveralltaskperformanceforthisstudywasveryhigh,withanoveralltaskcompletionrateofninety-seven(97)percent.Althoughtheperformancedatawasveryhigh,andtheindividualtaskperformanceratingwasalsohigh,thesubjectiveusabilityratingscollectedaspartoftheSystemUsabilityScale(SUS)weremuchlowerthanexpected.Thisseemstoimplythatalthoughthesystemiseasytouse,andthatparticipantsareabletosuccessfullycompletetheirtasks,somethingisnegativelyinfluencingtheiroverallsatisfactionwiththesystem.
AreasforImprovement
Theoveralltaskperformancerateforthistestwasveryhigh,atninety-seven(97)percent.Thisindicatesthatamajorityofthesystemishighlyusableanddoesnotrequireanysignificantimprovement.Therearealways,however,severalareasthatcouldbeenhancedtoachieveevenhighertaskcompletionrates,andperhapscouldincreaseoverallparticipantsatisfaction.
• Indicationoftaskworkflowo Inseveralareasoftheproduct,theusermustnavigateacrossseveraltabsorotheruserinterfaceelementinto
ordertocompletetheirtask.Insteadofpresentingtheseitemsacrosscommontabs,theworkflowcouldbeindicatedusinga1->2->3typearrangementthatwouldservetoguidetheusertocompletionofthetask.
o Inaddition,someembeddedassistancecouldbeprovidedtohelpusersunderstandandcompletetheirtasks.
• Buttons;PrimaryandSecondarybuttonso Acrossthesystemtherewasaninconsistentuseofuseractionbuttons,links,buttonnamesandoccasionally
buttonarrangement.Thesystemcouldbenefitfromprovidingamoreconsistentuseofbuttonsorlinksandperhapsamoresalientindicationofprimaryvssecondaryactionbuttons.
• Betterintegrationof3rdpartytoolso Thesystemtakesadvantageofseveral3rdpartytoolsinordertoachieverequiredfunctionality.Unfortunately,
eachofthesetoolspresentsadifferentuserinterface,andthiscanhavenegativelyimpactedtheperceptionoftheoveralluserexperience.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Appendix1:SampleConsentForm
ChiroTouchUsabilityStudyPracticeConsent
ChiroTouchwouldliketothankyouforconsideringparticipationinthisstudy.ThepurposeofthestudyistoevaluatetheusabilityoftheChiroTouchEHRsystem.YourparticipationinthisstudywillincludeperformingaspecifictaskwithinChiroTouch;andcompletingashortsurveyfollowingthestudy.Thestudyshouldtakeapproximately60-90minutes.TheinformationcollectedbyChiroTouchduringthestudyisforresearchpurposesonly.Yourparticipationinthisstudyisvoluntary,soyouarefreetowithdrawatanypointduringthestudy.
Pleaseanswerthefollowingscreeningquestions:
YesNo
☐☐ Doyou,oranyoneinyourhome,haveacommercialorresearchinterestinanEHRsoftwareorconsultingcompany?☐☐ Doyou,ordoesanyoneinyourhome,workinmarketingresearch,usabilityresearch,webdesign[…etc.]?☐☐ Doyourequireanyassistivetechnologiestouseacomputer?☐☐ Areyouactivelypracticingasalicensedmedicalproviderwithinyourclinic?
NameofPractice
NameofParticipant
Age ☐23to39|☐40to59|☐60-to74|☐75andolder
Gender ☐Male☐Female☐Other
YearsinPractice(ifDC)
Position ☐Provider☐Other ☐FrontOffice☐BackDesk/ProviderAssistants
Experiencew/ChiroTouch(Yrs)
Location
Bysigningbelow,Iagreetoparticipateinthestudy.
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
SignatureDate PrintedNameDate
Appendix2:SamplePost-TestQuestionnaire
ChiroTouchUsabilityStudyPost-TestQuestionnaire
1. Howwouldyoupreferwecommunicatenewenhancementstoyourpractice?�Email �MembersAreaPage �Newsletter �Wedonotcaretobenotified
2. WhatisyourpreferredmethodofworkingwiththeChiroTouchSupportdepartment?�Phone�Email �OnlineChat(ifthiswereavailable)
3. Whatisyourpreferredmethodforcreatingaprescription?�DrugSearch �Favorites
4. HowlongdoesittaketoenteranewpatientinChiroTouch(Demographicdataonly)?�0-3minutes �3-5minutes �>5Minutes
5. IfyoucouldchangeonepartofChiroTouch,whatwouldyouchange
6. DoyoufeelthattheChiroTouchsystemmakesreportingandmaintainingcomplianceeasy?
7. Whatcompliancefeature(s)wouldyouaddtotheChiroTouchreportingsystemtomakeiteasierforyoutoreport?
8. Onascaleof1-5,howwouldrateyouroverallsatisfactionwithChiroTouch?�1(VeryDissatisfied) �2(SomewhatDissatisfied) �3 �4(SomewhatSatisfied) �5(VerySatisfied)
9. WhichofthefollowingfunctionsareavailablethroughChiroTouch?(Youmaycheckmorethan1option)�PatientEducationMaterials�ElectronicallyPrescribeControlledSubstances�PrescriptionCouponsfordistributingtoPatients�Secureemailmessaging
10. DidyoufindtheChiroTouchsystemoranypartofitunnecessarilycomplexorcumbersome?Pleaseexplain.
11. IfyoucouldaddonepieceoffunctionalitytoChiroTouch,whatwouldyouadd?
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Appendix3Data
1. TasksPerformedDuringStudyPreparation Task DesignatedTaskTime OptimalPaths
Task1DemographicInformation
ModifyPatientDemographicinformation
120(seconds)7
Task2ProblemListView/Intervention
Addanitemtopatientproblemlist 30(seconds)
5Task3ProblemListIntervention-Remove
Removeanitemfrompatientproblemlist
30(seconds)
2Task4ClinicalInfoRecon/Incorporation
Reconcileandintegrateinformationfromanotherprovider
180(seconds)
11Task5MedicationAllergy–Add
AddamedicalAllergy 60(seconds)9
Task6Drug-AllergyInteractionCheck
Prescribeadrug/ViewCDS 120(seconds)
5Task7ElectronicPrescribe Prescribeadrugw/oCDSIntervention 180(seconds) 6Task8eRXCancel/Change CancelaneRX 180(seconds) 12Task9eRX-Renew RenewaneRX 30(seconds) 2Task10MedOrder;Drug-DrugIntervention
Orderamedication;viewCDS 60(seconds)6
Task11Medicationlist-Stop Stopamedication 30(seconds) 2Task12MedAllergyList–Remove
Removeanallergy 30(seconds)2
Task13CPOEDiagImaging Createimagingorders 120(seconds) 7Task14CPOEDiagImaging-Modify
ModifyImagingorder 60(seconds)
6Task15CPOELabOrders CreateLaborder 60(seconds) 7Task16CPOELabOrders–Modify
ModifyLaborder 60(seconds)
6Task17ImplantableDevice–Add
AddanImplantableMedicalDevice 60(seconds)
5Task18ImplantableDevice–Modify
ModifyanImplantableMedicalDevice 30(seconds)4
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
2. ParticipantsDemographicsGender
Male 2Female 8Total 10
ChiroTouchUserRoleProvider 2BackDesk/Prov.Asst. 1FrontDesk 5Other 2
YearsofExperiencew/ChiroTouch<1 2<2 3>2 5
YearsinPractice<1 02-5 25-10 2>10 6
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Appendix4:ModeratorGuide
ChiroTouchUsabilityStudyModeratorGuide
Prior to testing § Confirm schedule with Participants § Ensure the client’s ChiroTouch software is running properly
Prior to each participant:
§ Reset application § Start session recordings with Morae
Prior to each task:
§ Reset application to starting point for next task
After each participant: § End session recordings with Morae
After all testing
§ Back up all video and data files
Orientation (5 minutes)
Thank you for participating in this study. Our session today will last 60 to 90 minutes. I will ask you to complete a few tasks using this system and answer some questions. We are interested in how easy (or how difficult) this system is to use, what in it would be useful to you, and how we could improve it. You will be asked to complete these tasks on your own trying to do them as quickly as possible with the fewest possible errors or deviations. Do not do anything more than asked. If you get lost or have difficulty I cannot help you with anything to do with the system itself. Please save your detailed comments until the end of a task or the end of the session when we can discuss freely.
I did not have any involvement in its creation, so please be honest with your opinions.
The product you will be using today is describe the state of the application, i.e., production version, early prototype, etc. Some of the data may not make sense as it is placeholder data.
We are recording our session today. All of the information that you provide will be kept confidential and your name will not be associated with your comments at any time.
Do you have any questions or concerns?
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK1:DemographicInformation (120 Seconds)
Direct the participant to the starting point for the task.
Yourpatient,KerryKennedy,hasarrivedfortheirscheduledappointment.Beforebeginningtheirtreatment,youneedtoverifyorentersomeofthedemographicinformationthatisstoredwithinthesystem.NavigatetotheDemographicinformationsectionfromtheFrontDeskapplicationandmakesurethatthefollowinginformationisstoredintheEHR:
Asyoureviewthepatienthistory,theymentionthattheyprefertospeakSpanishasherfirstlanguageandsheisnotascomfortablereadingEnglish.
• SetthepatientLanguageto‘Spanish’• SetthepatientRaceas‘African-American’• SetthepatientEthnicityas‘HispanicorLatino’
Asyoucontinuethroughtheirsocialhistory,theyinformyouthatwhilehersexassignedatbirthwasmale,shehaslegallychangedhernametoRandyandnowidentifiesasafemale.Documentthefollowinginformationinthechart.
• SexualOrientationas'Choosenottodisclose'• IndicatetheGenderIdentityas‘Male-to-Female(MTF)transsexual'
EnterandsavethisinformationintotheEHR.
Success:
¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK2:ProblemListViewandIntervention. (30 Seconds)
Direct the participant to the starting point for the task.
Yourassistanthasmeasuredthepatient’sbloodpressureandrecordedthefollowingvalues:150/90.Consideringtheseresults,youhavediagnosedyourpatientwiththefollowing:
401.9-Hypertension
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK3:ProblemListIntervention--Remove. (30 Seconds)
Direct the participant to the starting point for the task.
Youalsonoticethattheyhavealisteddiagnosisof“(M54.5)LowBackPain“ofwhichtheyclaimthattheynolongersufferfrom.
• Removethe“(M54.5)LowerBackPain“diagnosis.”
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK4:ClinicalInformationReconciliation/Incorporation (180 Seconds)
Direct the participant to the starting point for the task.
ThepatienthascometoyourofficewithaCCDAdocumentwithmedicalinformationfromanotherprovider.Thisinformationhasalreadybeenpre-loadedintotheChiroTouchsystem.UsingtheReconciliationfeaturesforCCDAlocatedonthepatientsearchscreenonthePatientsapplicationpleasedothefollowing:
• AccesstheCCDDocumentforyourpatient• ReconciletheMedications• ReconciletheMedicationAllergies• ReconciletheProblems.
EnterandsavethisinformationintotheEHR.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK5:MedicationAllergyList-Addanallergy.(60 Seconds)
Direct the participant to the starting point for the task.
Asyoubeginyourconsultationthepatienttellsyouthattheyrememberthataftertakingamedicationthatcontained“sulfa”awhilebacktheyhadexperiencedsevereurticaria(hives).
• AddanallergyforSulfadixineorSulfa(SulfonamideAntibiotics)totheirrecord.o Youmayseeawarningmessage.
EnterandsavethisinformationintotheEHR.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK6:Drug-AllergyInteractionCheck.(120 Seconds)
Direct the participant to the starting point for the task.
Yourpatientsaidthatthemainreasontheycameintoyourofficeisthattheystartedtofeelpoorlyaftereatingasaladbowlfromalocalfast-foodMexicanrestaurantchain.Youthinkthattheymayhavecontractedane-Coliinfection.
YoudecidetoprescribeKeflex(cephalexin)capsule250MG.However,onaddingKeflexyounoticeadrug-allergyinteractionalertsoyoudecidetostopthisprescription.
EntertheinformationaboveintotheEHRandidentify/viewthedrug-allergyalert.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK7:ElectronicPrescribing. (180 Seconds)
Direct the participant to the starting point for the task.
Youstillneedtoprescribeamedicationtowhichthepatientisnotallergic.YoudecidetoorderDoxycyclineinstead.
Pleaseprescribe:
• DoxycyclineHyclate20MGtabletstobetakenonetablettwiceadayaftermealsfor5days.
• SendtheRXtothechosenpharmacyusingthefollowingSIG:1234
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Task8.ElectronicPrescribingCancel/Change. (180 Seconds)
Direct the participant to the starting point for the task.
Oops,youjustrealizedthatyouprescribedandsentelectronicallytothepharmacytheantibioticaboveat20mgdose,butyoureallywantedtoprescribethe100mgDoxycyclinetablet.Youneedtomakemodificationssuchthatthepharmacydispensesthecorrectdosage,andmakesurethepatientsmedicationlistisalsocorrect.
• Navigatetoareportoftheprescriptionsthatweresentelectronicallyandcancelthe20mgtablet.• PrescribeDoxycyclineHyclate100MGtabletstobetakenonetablettwiceadayaftermealsfor5days.
• MakesuretheMedicationlistreflectsthischange.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Task9.ElectronicPrescribing–renew. (30 Seconds)
Direct the participant to the starting point for the task.
YoupatienthasbeentakingLipitor(atorvastatin)forseveralyearswithoutanyproblemsandithasbeendoingagoodjobcontrollingtheircholesterollevels.Theyhaveaskedthatyourenewtheircurrentprescription.
• ElectronicallyrenewtheexistingRXforLipitor(atorvastatin)
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK10:MedicationOrder,ViewDrug-DrugInteractions,MedicationListInterventions (60 Seconds)
Direct the participant to the starting point for the task.
SeveraltasksagoyoudiagnosedyourpatientwithHypertension.Inordertotreatyourpatient’selevatedbloodpressure,youdecidetoaddAccupriltotheirhypertensiondiagnosis.
Enteraprescriptionfor:
• AccuprilOralTablet10MGtotakeonetablettwiceadayfor30days.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Task11:Stopamedication. (30 Seconds)
Direct the participant to the starting point for the task.
Uponexaminationofthepatient’scurrentmedicationlist,younoticethatthedrugNaproxenislistedasanactivemedication.YoumayalsonoticethatthisdrugisassociatedwithareactiontoapatientAllergy.YouneedtostopthepatientfromtakingNaproxenandindicatethisstopintheirrecord.
• EnteramedicationstopforNaproxen.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK12:MedicationAllergyList-Removeanallergy. (30 Seconds)
Direct the participant to the starting point for the task.
Duringtheconsultation,yourpatienthastoldyouthattheynolongerhaveanallergytoAmoxicillin.Youdecidetoremovethisallergyfromthepatientrecord.
• RemovetheAmoxicillinallergyfromthepatientrecordintheEHR.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Task13:RecordanoutpatientRadiologyorder. (120 Seconds)
Direct the participant to the starting point for the task.
Yourpatienthasbeencomplainingofanklepainafterinjuringthemselveswhileplayingbasketball.Uponfurtherexamination,youdecidetoorderanx-ray,usingtheCPOEfeaturesofthesystem.EnterthefollowingintotheEHR:Diagnosis:ICD-10code:S93.01XASubluxationofRightAnkleJointTest:Radiologicexaminationankle2viewsReview:RequestingPhysicianCraigPalmer
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Task14:ModifyoutpatientRadiologyorder. (60 Seconds)
Direct the participant to the starting point for the task.
YourPatienthasalsocomplainedofKneepainafterinjuringthemselveswhileplayingbasketball.Youdecidethatyouwanttoaddanadditionalitemtotheradiologyorderthatyouplacedabove.Modifytheorderyourplacedinthetaskabovetoincludethefollowingtest:
• Radiologicexaminationknee3views
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK15:LaboratoryOrders. (60 Seconds)
Direct the participant to the starting point for the task.
Yourpatienthasbothhighbloodpressureandelevatedcholesterollevels.Youwanttoordersomeotherbasiclabworktomakesurethatthepatientdoesn’thaveanyotherunderlyingproblems,suchasakidneyissue.UsingtheCPOEsystempleaseenteranorderforlabworkasfollows:
• Diagnosis:EssentialPrimaryHypertension• Tests:
o Creatinine,24-Hoururineo Fastingglucoseinserumorplasma
EnterthelaborderintotheEHR.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK16:LaboratoryOrders–Modify (60 Seconds)
Direct the participant to the starting point for the task.
.
Uponreviewofthelaborderthatyoujustcompletedintheprevioustask,youdecidetochangethislaborderandaddanadditionaltest.Addthefollowinglabtesttothepreviousorder.
• Urobilinogen,Semi-Qn
Makethenecessarychangessothattheadditionaltestisalsoordered.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK17:ImplantabledevicelistTASK7:ElectronicPrescribing. (60 Seconds)
Direct the participant to the starting point for the task.
.
Thepatienttellsyouthatafewyearsbacktheyhada“Calciumcompoundbonevoidfiller”implantedintotheirleftarm.UponexaminationoftheimplantabledevicelistpresentedintheEHR,thereisnorecordofthismedicaldevice.
• AddamedicaldevicewiththefollowingUDIcode.
=/W4146EB0010T0475=,000025=A99971312345600=>014032=}013032&,1000000000000XYZ123(Note:thisinformationisstoredonyourclipboardandsavedasa.txtfileontheWindowsDesktop)
• Parsetheinformationassociatedwiththeimplantedmedicaldeviceandverballystatetheexpirationdate.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
TASK18:Implantabledevice-Modify. (30 Seconds)
Direct the participant to the starting point for the task.
Theimplanteddevicethatyouenteredintheprevioustaskisexpiredandshouldnolongerclassifiedas“active.”ModifytheinformationfortheBoneVoidFillersothattheDeviceStatusissettoINACTIVE.
Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:
Task Time: Seconds
Comments:
Observed Errors and Verbalizations:
Rating: Overall, this task was:
Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)
Administrator / Notetaker Comments:
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
Appendix5.SpecificTaskData
ParticipantNumber
Task1DemographicInformation
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 1:54 Success 1 4P0102 1:23 Success 0 3P0103 2:08 Success 0 5P0104 2:42 Success 2 2P0105 1:06 Success 0 5P0106 1:45 Success 0 5P0107 1:47 Success 0 5P0108 0:20 Success 1 4P0109 3:48 Success 1 4P0110 1:18 Success 1 5
ExpectedTimeonTask 2:00 (SD) AverageTimeonTask 1:49 0:54 AverageTaskSatisfaction 4.20 0.98 Average#PathDeviations 0.60 0.66 PercentSuccess 100%
ParticipantNumber
Task2ProblemListView/Intervention
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:26 Success 1 5P0102 0:29 Success 1 4P0103 0:45 Success 0 5P0104 0:34 Success 0 4P0105 0:50 Success 1 5P0106 0:49 Success 2 4P0107 0:39 Success 1 3P0108 0:37 Success 1 3P0109 0:59 Success 2 5P0110 0:48 Success 2 4
ExpectedTimeonTask 0:30 (SD)
AverageTimeonTask 0:42 0:10 AverageTaskSatisfaction 4.20 0.75 Average#PathDeviations 1.10 0.70 PercentSuccess 100%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task3ProblemListIntervention-Remove
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:22 Success 0 5P0102 0:49 Success 0 4P0103 2:33 Success 3 1P0104 1:03 Success 0 3P0105 1:25 Success 0 3P0106 0:31 Success 0 2P0107 0:16 Success 0 4P0108 0:58 Success 1 3P0109 0:45 Success 0 5P0110 0:51 Success 0 4
ExpectedTimeonTask 0:30 (SD)
AverageTimeonTask 0:57 0:37 AverageTaskSatisfaction 3.40 1.20 Average#PathDeviations 0.40 0.92 PercentSuccess 100%
ParticipantNumber
Task4ClinicalInfoRecon/Incorporation
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 4:23 Success 3 1P0102 1:50 Success 1 4P0103 3:15 Fail 3 1P0104 2:09 Fail 6 1P0105 1:17 Success 0 4P0106 0:44 Success 0 5P0107 1:16 Success 0 4P0108 1:42 Success 2 3P0109 2:06 Fail 3 4P0110 1:31 Success 1 4
ExpectedTimeonTask 3:00 (SD)
AverageTimeonTask 1:49 1:01 AverageTaskSatisfaction 3.10 1.45 Average#PathDeviations 1.90 1.81 PercentSuccess 70%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task5MedicationAllergy-Add
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 1:20 Success 1 4P0102 1:48 Success 2 3P0103 1:13 Success 1 5P0104 2:29 Success 3 2P0105 0:58 Success 1 5P0106 1:33 Success 1 2P0107 1:38 Success 0 4P0108 1:31 Success 1 3P0109 3:20 FAIL 6 3P0110 1:24 Success 1 4
ExpectedTimeonTask 1:00 (SD)
AverageTimeonTask 1:33 0:40 AverageTaskSatisfaction 3.50 1.02 Average#PathDeviations 1.70 1.62 PercentSuccess 90%
ParticipantNumber
Task6Drug-AllergyInteractionCheck
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:35 Success 0 3P0102 1:02 Success 0 3P0103 0:33 Success 0 5P0104 1:53 Success 2 1P0105 1:02 Success 0 2P0106 1:07 Success 0 4P0107 0:52 Success 0 4P0108 0:57 Success 0 4P0109 1:09 Success 2 4P0110 1:13 Success 1 5
ExpectedTimeonTask 2:00 (SD)
AverageTimeonTask 1:02 0:21 AverageTaskSatisfaction 3.50 1.20 Average#PathDeviations 0.50 0.81 PercentSuccess 100%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task7ElectronicPrescribe
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 2:15 Success 0 2P0102 2:22 Success 1 4P0103 1:45 Success 0 5P0104 2:12 Success 1 2P0105 1:05 Success 0 4P0106 1:37 Success 0 3P0107 1:48 Success 0 4P0108 1:14 Success 0 4P0109 2:19 Success 0 5P0110 1:34 Success 0 5
ExpectedTimeonTask 3:00 (SD)
AverageTimeonTask 1:49 0:26 AverageTaskSatisfaction 3.80 1.08 Average#PathDeviations 0.20 0.40 PercentSuccess 100%
ParticipantNumber
Task8eRXCancel/Change
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 2:28 Success 0 3P0102 3:48 Fail 6 3P0103 2:13 Success 0 5P0104 3:26 Success 0 2P0105 1:43 Success 0 4P0106 3:27 Success 1 4P0107 2:25 Success 0 5P0108 2:23 Success 0 4P0109 3:15 Success 3 5P0110 3:32 Success 1 4
ExpectedTimeonTask 3:00 (SD)
AverageTimeonTask 2:46 0:40 AverageTaskSatisfaction 3.90 0.94 Average#PathDeviations 1.10 1.87 PercentSuccess 90%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task9eRX-Renew
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 1:48 Success 0 4P0102 0:25 Success 0 5P0103 0:22 Success 0 5P0104 2:25 Success 3 2P0105 0:18 Success 0 5P0106 0:38 Success 1 5P0107 0:22 Success 0 5P0108 0:21 Success 0 5P0109 0:21 Success 0 5P0110 0:26 Success 0 5
ExpectedTimeonTask 0:30 (SD)
AverageTimeonTask 0:45 0:42 AverageTaskSatisfaction 4.6 0.92 Average#PathDeviations 0.40 0.92 PercentSuccess 100%
ParticipantNumber
Task10MedOrder;Drug-DrugIntervention
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 1:31 Success 0 4P0102 1:38 Success 0 4P0103 1:12 Success 0 5P0104 1:26 Success 0 3P0105 1:24 Success 1 4P0106 1:21 Success 0 5P0107 2:11 Success 0 4P0108 1:24 Success 0 4P0109 2:16 Success 0 5P0110 2:03 Success 0 5
ExpectedTimeonTask 1:00 (SD)
AverageTimeonTask 1:39 0:22 AverageTaskSatisfaction 4.3 0.64 Average#PathDeviations 0.1 0.30 PercentSuccess 100%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task11Medicationlist-Stop
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:11 Success 0 5P0102 0:21 Success 0 5P0103 0:08 Success 0 5P0104 0:27 Success 0 4P0105 0:15 Success 0 5P0106 0:22 Success 0 5P0107 0:29 Success 0 5P0108 0:28 Success 0 4P0109 0:31 Success 0 5P0110 0:21 Success 0 5
ExpectedTimeonTask 0:30 (SD)
AverageTimeonTask 0:21 0:07 AverageTaskSatisfaction 4.80 0.40 Average#PathDeviations 0.00 0.00 PercentSuccess 100%
ParticipantNumber
Task12MedAllergyList-Remove
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:15 Success 0 5P0102 0:57 Success 0 5P0103 0:29 Success 0 5P0104 0:41 Success 0 3P0105 0:28 Success 0 5P0106 0:17 Success 0 5P0107 0:21 Success 0 5P0108 0:15 Success 0 4P0109 0:24 Success 0 5P0110 0:22 Success 0 5
ExpectedTimeonTask 0:30 (SD)
AverageTimeonTask 0:27 0:12 AverageTaskSatisfaction 4.70 0.64 Average#PathDeviations 0.00 0.00 PercentSuccess 100%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task13CPOEDiagImaging
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 2:05 Success 0 5P0102 1:54 Success 0 4P0103 1:54 Success 0 3P0104 1:44 Success 0 4P0105 1:04 Success 0 5P0106 0:48 Success 0 4P0107 1:50 Success 0 3P0108 1:25 Success 0 3P0109 2:15 Success 0 5P0110 2:19 Success 1 5
ExpectedTimeonTask 2:00 (SD)
AverageTimeonTask 1:44 0:28 AverageTaskSatisfaction 4.10 0.83 Average#PathDeviations 0.10 0.30 PercentSuccess 100%
ParticipantNumber
Task14CPOEDiagImaging-Modify
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 1:08 Success 0 5P0102 1:41 Success 1 4P0103 1:03 Success 0 5P0104 0:51 Success 0 4P0105 0:49 Success 0 5P0106 1:14 Success 0 4P0107 1:26 Success 1 4P0108 1:57 Success 2 3P0109 1:36 Success 1 5P0110 1:13 Success 1 4
ExpectedTimeonTask 1:00 (SD) AverageTimeonTask 1:18 0:21 AverageTaskSatisfaction 4.30 0.64 Average#PathDeviations 0.60 0.66 PercentSuccess 100%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task15CPOELabOrders
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 1:47 Success 0 5P0102 0:48 Success 0 4P0103 0:48 Success 0 3P0104 1:15 Success 0 4P0105 1:28 Success 0 5P0106 2:15 Success 0 5P0107 1:44 Success 0 5P0108 1:19 Success 0 3P0109 3:22 Success 1 4P0110 1:14 Success 0 4
ExpectedTimeonTask 1:00 (SD)
AverageTimeonTask 1:36 0:43 AverageTaskSatisfaction 4.20 0.75 Average#PathDeviations 0.10 0.30 PercentSuccess 100%
ParticipantNumber
Task16CPOELabOrders-Modify
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:49 Success 0 5P0102 1:05 Success 0 5P0103 1:01 Success 1 5P0104 0:38 Success 0 4P0105 1:15 Success 0 5P0106 0:55 Success 0 5P0107 0:43 Success 0 5P0108 0:51 Success 0 4P0109 1:33 Success 2 5P0110 0:58 Success 0 4
ExpectedTimeonTask 1:00 (SD)
AverageTimeonTask 0:59 0:15 AverageTaskSatisfaction 4.70 0.46 Average#PathDeviations 0.30 0.64 PercentSuccess 100%
9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com
ParticipantNumber
Task17ImplantableDevice-Add
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:59 Success 0 4P0102 1:45 Success 3 4P0103 0:39 Success 1 5P0104 0:50 Success 1 4P0105 0:32 Success 0 5P0106 1:08 Success 1 5P0107 0:35 Success 1 4P0108 0:50 Success 1 3P0109 1:38 Success 1 4P0110 0:55 Success 0 4
ExpectedTimeonTask 1:00 (SD)
AverageTimeonTask 0:59 0:24 AverageTaskSatisfaction 4.20 0.60 Average#PathDeviations 0.90 0.83 PercentSuccess 100%
ParticipantNumber
Task18ImplantableDevice-Modify
TaskTime Outcome#Path
DeviationsTask
SatisfactionP0101 0:51 Success 0 5P0102 0:13 Success 0 5P0103 0:53 Success 1 4P0104 1:20 Success 2 3P0105 0:36 Success 0 4P0106 0:29 Success 0 5P0107 0:44 Success 1 3P0108 0:34 Success 1 4P0109 1:18 Success 4 5P0110 1:01 Success 1 4
ExpectedTimeonTask 0:30 (SD)
AverageTimeonTask 0:48 0:20 AverageTaskSatisfaction 4.20 0.75 Average#PathDeviations 1.00 1.18 PercentSuccess 100%