Page 1 of 26
Sugarloaf Corporate Center 2170 Satellite Blvd
Suite 435 Duluth, Georgia 30097
770-497-1560 www.mdlogic.com
EHR USABILITY TEST REPORT
MD Logic EHR v7.0
This reports is based on the NISTIR 7742 Customized Common Industry Format Template For Electronic Health Record Usability Testing
The information in this document is the confidential information of MD Logic, Inc MD Logic and the MD Logic Logo are registered trademarks of MD Logic, Inc.
© 2017 MD Logic, Inc. All rights reserved
Page 2 of 26
TABLE OF CONTENTS 1. Executive Summary Page 3
2. Introduction Page 4
3. Method Page 4
Participants / Study Design Page 4-5
Tasks Page 5-6
Procedure Page 6
Test Location Page 6
Test Environment Page 6-7
Test Forms and Tools Page 7
Participant Instructions Page 7
Usability Metrics Page 7
Data Scoring Page 7-8
4. Results Page 8
Data Analysis and Reporting Page 8
§170.315(a)(1) Computerized Provider Order Entry – Medications Page 9
§170.315(a)(2) Computerized Provider Order Entry – Laboratory Page 9-10
§170.315(a)(3) Computerized Provider Order Entry – Diagnostic Imaging Page 10
§170.315(a)(4) Drug-Drug, Drug-Allergy Interactions Checks Page 10-11
§170.315(a)(5) Demographics Page 11-12
§170.315(a)(6) Problem List Page 12
§170.315(a)(7) Medication List Page 13
§170.315(a)(8) Medication Allergy List Page 13-14
§170.315(a)(9) Clinical Decision Support Page 14-15
§170.315(a)(14) Implantable Device List Page 15-16
§170.315(b)(2) Clinical Information Reconciliation and Incorporation Page 16
§170.315(b)(3) Electronic Prescribing Page 16-17
Overall Results Page 17
5. Appendices Page 17
Appendix 1: Sample Recruiting Screener Page 17-18
Appendix 2: Participant Demographics Page 19-20
Appendix 3: Non-Disclosure Agreement / Informed Consent Form Page 21
Appendix 4: Sample Orientation Page 22
Appendix 5: User Protocol Tasks Page 22-25
Appendix 6: System Usability Scale Questionnaire Page 25-26
Page 3 of 26
1. EXECUTIVE SUMMARY
A usability test of the MD Logic EHR, v 7.0, an Ambulatory EHR was conducted on 04/21/2017 and 04/22/2017 via
online meeting by eMarketMD and the MD Logic End User Test Group. The purpose of this study was to test and
validate the usability of the current user interface, and provide evidence of usability in the EHR under Test (EHRUT).
During the usability test, ten (10) healthcare providers matching the target demographic criteria served as
participants and used the EHRUT in simulated, but representative tasks. This study collected performance data on 48
tasks typically conducted on an EHR:
§170.315(a)(1) Computerized Provider Order Entry – Medications
§170.315(a)(2) Computerized Provider Order Entry – Laboratory
§170.315(a)(3) Computerized Provider Order Entry – Diagnostic Imaging
§170.315(a)(4) Drug-Drug, Drug-Allergy Interactions Checks
§170.315(a)(5) Demographics
§170.315(a)(6) Problem List
§170.315(a)(7) Medication List
§170.315(a)(8) Medication Allergy List
§170.315(a)(9) Clinical Decision Support
§170.315(a)(14) Implantable Device List
§170.315(b)(2) Clinical Information Reconciliation and Incorporation
§170.315(b)(3) Electronic Prescribing
During the 120 minutes one-on-one usability test, each participant was greeted by the administrator online and asked
to review and sign an informed consent/release form (included in Appendix 3); they were instructed that they could
withdraw at any time. Participants did have prior experience with the EHR. The administrator introduced the test,
and instructed participants to complete a series of tasks (given one at a time) using the EHRUT. During the testing,
the administrator timed the test and, along with the data logger(s) recorded user performance data on paper and
electronically. The administrator did not give the participant assistance in how to complete the task.
The following types of data were collected for each participant:
Number of tasks successfully completed within the allotted time without assistance Time to complete the tasks Number and types of errors Path deviations Participant’s verbalizations Participant’s satisfaction ratings of the system System Usability Scale
All participant data was de-identified – no correspondence could be made from the identity of the participant to the
data collected. Following the conclusion of the testing, participants were asked to complete a post-test questionnaire.
Participants were mailed a $25 gift card in appreciation for their time. Various recommended metrics, in accordance
with the examples set forth in the NIST Guide to the Processes Approach for Improving the Usability of Electronic
Health Records, were used to evaluate the usability of the EHRUT. In addition, results include the following
observations:
Major Findings
Areas for Improvement
Participant’s verbalizations
Page 4 of 26
2. INTRODUCTION
The EHRUT tested for this study was the MD Logic EHR v7.0 engineered to present medical information to healthcare
providers in ambulatory settings. The intended users of the MD Logic EHR are doctors, physical therapists, nurse
practitioners, physician assistants, nurses, medical assistants including anyone entering or accessing clinical data at an
ambulatory medical practice. The EHRUT consists of electronic health records for order entry, review of past health
data and navigation of EHR. The usability testing attempted to represent realistic exercises and conditions. The
purpose of this study was to test and validate the usability of the current user interface, and provide evidence of
usability in the EHR under Test (EHRUT). The purpose of this study was to test and validate the usability of the current
user interface and provide evidence of usability in the EHRUT. To this end, measures of effectiveness, efficiency, and
user satisfaction were captured during the usability testing, such as task success, time on task, and task group ratings.
3. METHOD
PARTICIPANTS
A total of 10 participants were tested on the EHRUT(s). Participants in the test were physicians and nurses in
ambulatory settings. The Participants ranged in years of experience with the MD Logic EHR and varied in specialty.
Participants were recruited by eMarketMD. The request sent to participants described the end-user roles required for
testing. Participants either responded directly or were identified by administrators of their health care organization. For the
test purposes, end-user characteristics were identified and translated into a recruitment screener used to solicit potential
participants; an example of a screener is provided in Appendix 1 - Sample Recruiting Screener. A description of the
participants is listed in Table 1. The Participants had no direct connection to the development of or organization
producing the EHRUT(s). Participants were given the opportunity to have the same orientation as the actual end users
would have received. For the test purposes, end-user characteristics were identified and translated into a recruitment
screener used to solicit potential participants.
Recruited participants had a mix of backgrounds and demographic characteristics conforming to the recruitment
screener. The following is a table of participants by characteristics, including demographics, professional experience,
computing experience and user needs for assistive technology. Participant names were replaced with Participant IDs
so that an individual’s data cannot be tied back to individual identities. Participant demographics are detailed in
Appendix 2 - Participant demographics.
The total number and Description of Participants
Ten (10) participants were recruited and participated in the usability test. The description of the participants are listed
in the table below:
Table 1: Participants
Page 5 of 26
STUDY DESIGN Overall, the objective of this test was to uncover areas where the application performed well – that is, effectively,
efficiently, and with satisfaction – and areas where the application failed to meet the needs of the participants. The
data from this test may serve as a baseline for future tests with an updated version of the same EHR and/or
comparison with other EHRs provided the same tasks are used. In short, this testing serves as both a means to record
or benchmark current usability, but also to identify areas where improvements must be made. During the usability
test, participants interacted with the MD Logic EHR Software. The system was evaluated for effectiveness, efficiency
and satisfaction as defined by measures collected and analyzed for each participant:
Number of tasks successfully completed within the allotted time without assistance
Time to complete the tasks
Number and types of errors
Path deviations
Participant’s verbalizations (comments)
Participant’s satisfaction ratings of the system
TASKS
A number of tasks were constructed that would be realistic and representative of the kinds of activities a user might
do with this EHR. Tasks were selected for this study based upon the twelve (12) ONC CEHRT 2015 certification criteria.
Frequency of use, potential risk for patient safety and the critical function were all taken into consideration. The
safety-enhanced (SED) tasks for the twelve ONC CEHRT 2015 certification included:
Page 6 of 26
Scenario 1: Medication Allergy List
Scenario 2: Problem List
Scenario 3: Implantable Device List
Scenario 4: Clinical Decision Support
Scenario 6: Computerized Physician Order Entry (CPOE)
Scenario 7: Drug Utilization Review
Scenario 8: Clinical Information Reconciliation & Incorporation
Scenario 9: Demographics (Clerical)
Scenario 10: DUR Settings (Administrative)
Scenario 11: Electronic Prescriptions
PROCEDURES Upon arrival, participants were greeted; their identity was verified and matched with a name on the participant
schedule. Participants were then assigned a participant ID. Each participant reviewed and signed an informed consent
and release form (See Appendix 3). To ensure that the test ran smoothly, two staff members participated in this test,
the usability administrator and the data logger. The usability testing staff conducting the test was experienced
usability researchers. The administrator moderated the session including administering instructions and tasks. The
administrator also monitored task times, obtained post-task rating data, and took notes on participant comments. A
second person served as the data logger and took notes on task success, path deviations, number and type of errors,
and comments. Participants were instructed to perform the tasks (see specific instructions below):
As quickly as possible making as few errors and deviations as possible.
Without assistance; administrators were allowed to give immaterial guidance and clarification on tasks, but not
instructions on use.
Without using a think aloud technique.
For each task, the participants were given a written copy of the task. Task timing began once the administrator finished
reading the question. The task time was stopped once the participant indicated they had successfully completed the task.
Following the session, the administrator gave the participant the post-test questionnaire and thanked each individual for
their participation.
Participants' demographic information, task success rate, time on task, errors, deviations, verbal responses, and post-test
questionnaire were recorded into a spreadsheet.
Participants were thanked for their time.
TEST LOCATION The test was taken online by the participants. Only the participant, the administrator and data logger were online
during the test. All observers and the data logger worked next to the administrator. To ensure that the
environment was comfortable for users, background noise levels were kept to a minimum.
TEST ENVIRONMENT
The EHRUT would be typically be used in a healthcare office or facility. In this instanc e, the testing was conducted
online using the MD Logic Data Center. For testing, each participant used their own personal computer to access
the MD Logic EHR software. The MD Logic Data Center controlled the size and screen resolution of the MD Logic
software ensuring that each participant viewed the same image and resolution in size for software. The
application itself was running on a server based platform accessed via online meeting.
Page 7 of 26
TEST FORMS AND TOOLS During the usability test, various documents and instruments were used, including:
Informed Consent
Usability Protocol
Moderator’s Guide
Post-test Questionnaire
Examples of these documents can be found in Appendices 3-6 respectively. The Moderator’s Guide was devised so as
to be able to capture required data.
PARTICIPANT INSTRUCTIONS The administrator reads the following instructions aloud to the each participant:
Thank you for participating in this study. Your input is very important. Our session today will last approximately 120 minutes.
During that time you will use an instance of MD Logic EHR software and answer some questions. You should complete the
tasks as quickly as possible making as few errors as possible. Please try to complete the tasks on your own following the
instructions very closely. Please note that we are not testing you we are testing the system, therefore if you have difficult y all
this means is that something needs to be improved in the system. I will be here in case you need specific help, but I am not
able to instruct you or provide help in how to use the application. Overall, we are interested in how easy (or how difficult) this
system is to use, what in it would be useful to you, and how we could improve it. I did not have any involvement in its creation,
so please be honest with your opinions. All of the information that you provide will be kept confidential and your name will not
be associated with your comments at any time. Should you feel it necessary you are able to withdraw at any time during the
testing.
Following the procedural instructions, participants were shown the MD Logic EHR software and as their first task, were given
time 10 minutes to explore the system and make comments. Once this task was complete, the administrator gave the
following instructions:
For each task, I will read the description to you and say “Begin.” At that point, please perform the task and say “Done” once
you believe you have successfully completed the task. I would like to request that you not talk aloud or verbalize while you are
doing the tasks. I will ask you your impressions about the task once you are done.
USABILITY METRICS According to the NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records, EHRs
should support a process that provides a high level of usability for all users. The goal is for users to interact with the
system effectively, efficiently, and with an acceptable level of satisfaction. To this end, metrics for effectiveness,
efficiency and user satisfaction were captured during the usability testing. The goals of the test were to assess:
Effectiveness of MD Logic EHR by measuring participant success rates and errors
Efficiency of MD Logic EHR by measuring the average task time and path deviations
Satisfaction with MD Logic EHR by measuring ease of use ratings
DATA SCORING
Rationale and Scoring
Effectiveness: Task Success
Task success was determined by assigning numeric weights for various levels of task success, as follows:
Page 8 of 26
Success (without assistance) = 1.0
Partial success = 0.5
Failure = 0.0
A task was counted as a “Success” if the participant was able to achieve the correct outcome within the time
allotted on a per-task basis and a “Partial success” if the participant was able to achieve the correct outcome with
minimal assistance. A success score for each task was calculated by averaging the scores for ach task. The results
are provided as a percentage.
Task times were recorded for successes. Observed task times divided by the optimal time for each task were
calculated as a measure of optimal efficiency. Optimal task performance time, as benchmarked by expert
performance under realistic conditions, was recorded when constructing tasks.
Effectiveness: Task Failures
If the participant abandoned the task, did not reach the correct answer, performed it incorrectly, or reached the end
of the allotted time before successful completion, the task was counted as a “Failure”. No task times for failed tasks
or tasks that exceeded the target task time were used in calculations. A qualitative account of the observed errors
and error types was collected.
Effectiveness: Task Time
Each task was timed from when the administrator said “Begin” until the participant said “Done.” If the participant
failed to say “Done,” the time was stopped when the participant ceased performing the task. Only task times for
tasks that were successfully completed and tasks that were completed at or under the target time were included in
the average task time analysis. Average times per task and variance measures were calculated for each task for use
in the results analysis.
Satisfaction: Task Rating
Participant’s subjective impression of the ease of use of the application was measured by administering both a
simple question on completion of each scenario and a post-session questionnaire. After each scenario, the
participant was asked to rate “Overall, these tasks were:” on a scale of 1 (Very Difficult) to 5 (Very Easy). These data
were averaged across participants. To measure participants’ confidence in and likeability of MD Logic EHR overall,
the testing team administered the System Usability Scale (SUS) post-test questionnaire. Questions included, “I think
I would like to use this system frequently,” “I thought the system was easy to use,” and “I would imagine that most
people would learn to use this system very quickly.”
4. RESULTS
The results of the usability test were calculated according to the methods specified in the Usability Metrics (see
above). The charts representing each area tested and results recorded are listed below. No Participants failed to
follow the session and task instructions. The usability testing results for the EHRUT are detailed below. The
results should be seen in light of the objectives and goals. The data should yield actionable results that, if
corrected, yield material, positive impact on user performance.
Criteria 170.315(a)(1) CPOE – Medications
DATA ANALYSIS AND REPORTING
Page 9 of 26
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Record a New Medication Order (E4) 10 1.00 0.00 8.8 0.92 1 0 0.00 0.00 Change an Existing Medication Order (E3) 10 1.00 0.00 41.49 2.08 1.05 .316 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Record a New Medication Order (E4) Change an Existing Medication Order (E3)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Recording a new medication ordered confirmed 10 participants completing the task in the
optimal number of steps. Changing an existing medication order resulted in 9 participants completing the
task in the optimal number of steps with 1 participant taking an extra step due to touching the wrong
button close to the desired button.
- Satisfaction: The average satisfaction rating for CPOE was 5 out of 5 points
- Major Finding: Medication orders and changes to existing medication orders were executed very quickly
and without hesitation
- Areas of Improvement: Consider offering an option to increase the font size of the medication.
Criteria 170.315(a)(2) CPOE – Laboratory
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Record a Laboratory Order (f4) 10 1.00 0.00 55.7 2.497 1.04 0.422 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Record a Laboratory Order (f4)
- Effectiveness: Success for this task was 100% as participants were easily able to complete the tasks
within the time allowed.
Page 10 of 26
- Efficiency: Recording a laboratory order confirmed 8 of 10 participants completing the task in the optimal
number of steps with the other 2 adding an extra step due to selecting the wrong button.
- Satisfaction: The average satisfaction rating for CPOE was 5 out of 5 points
- Major Finding: Recording a laboratory order was executed very quickly and without hesitation
- Areas of Improvement: Consider offering additional color options when selecting Laboratory orders.
Criteria 170.315(a)(3) CPOE – Diagnostic Imaging
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access order and Record New Diagnostic Imaging Order (f2) 10 1.00 0.00 42.1 1.5 1.05 0.632 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding Access order and Record New Diagnostic Imaging Order (f2)
- Effectiveness: Success for this task was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Recording a new diagnostic imaging order confirmed 9 of 10 participants completing the task
in the optimal number of steps with 1 participant adding extra 2 steps due to selecting the correct button
then deselecting the button only to reselect the same button again.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: Recording new diagnostic imaging order was executed very quickly and without hesitation
- Areas of Improvement: Consider offering additional color options when selecting diagnostic imaging
order.
Criteria 170.315(a)(4) Drug-Drug, Drug-Allergy Interactions Checks
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Identify High-Priority Drug Interactions (g3) 10 1.00 0.00 13.8 1.55 1.1 0.316 0.00 0.00
View Full Details of Drug Interaction (g2) 10 1.00 0.00 16.9 0.994 1.1 0.316 0.00 0.00 Process Drug-Drug, Drug Allergy Interaction Alert (g1) 10 1.00 0.00 16 1.826 1.1 0.316 0.00 0.00 Adjust Severity Level of Interventions – Limited Permissions (j2) 10 1.00 0.00 9.4 0.699 1.1 0.316 0.00 0.00
Page 11 of 26
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Identify High-Priority Drug Interactions (g3)
View Full Details of Drug Interaction (g2)
Process Drug-Drug, Drug Allergy Interaction Alert (g1)
Adjust Severity Level of Interventions – Limited Permissions (j2)
- Effectiveness: Success for this task was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Identifying high priority drug interactions, viewing details of full drug interaction, processing
drug-drug/drug allergy interaction and adjusting the level of interventions confirmed 9 of 10 participants
completing the task in the optimal number of steps with 1 participant adding 1 step due to selecting an
incorrect button.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: All tasks were executed very quickly and without hesitation
- Areas of Improvement: No known areas of improvement identified at this time.
Criteria 170.315(a)(5) Demographics
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access and Record Demographics (i1) 10 1.00 0.00 85.8 3.645 1.01 0.316 0.00 0.00
Change Demographics (i2) 10 1.00 0.00 17.7 1.159 1.03 .316 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Access and Record Demographics (i1)
Change Demographics (i2)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Accessing and recording demographic information confirmed 9 of 10 participants comp leting
the task in the optimal number of steps with 1 participant adding 2 extra steps due to selecting the
incorrect button and having to deselect the button. Changing demographic information confirmed 9 of 10
participants completing the task in the optimal number of steps with 1 participant adding 2 extra steps
due to selecting the incorrect button and having to deselect the button.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
Page 12 of 26
- Major Finding: Recording and changing demographic information was executed very quickly and without
hesitation
- Areas of Improvement: Consider increasing the bolding of required fields to draw more attention to the
end user.
Criteria 170.315(a)(6) Problem List
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access and Active Problem (b1) 10 1.00 0.00 8.7 0.949 1.05 0.316 0.00 0.00 Change an Existing Problem from a Different Encounter (b2) 10 1.00 0.00 26.4 2.459 1.02 0.316 0.00 0.00
Reactivate Inactive Problem (b3) 10 1.00 0.00 27.2 1.751 1.02 0.316 0.00 0.00
Record a New Problem (b4) 10 1.00 0.00 26.9 1.853 1.02 0.316 0.00 0.00
Find Problem SNOMED Code (b5) 10 1.00 0.00 13.9 2.601 1.1 0.632 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Access and Active Problem (b1)Change Demographics (i2)
Change an Existing Problem from a Different Encounter (b2)
Reactivate Inactive Problem (b3)
Find Problem SNOMED Code (b5)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: All tasks confirmed 9 of 10 participants completing the task in the optimal number of steps
with a single participant in each task selecting an incorrect button.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: Each of these tasks were executed very quickly and without hesitation
- Areas of Improvement: Consider adding the color red to Snomed codes so they stand out when viewing
on the screen.
Criteria 170.315(a)(7) Medication List
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access Active Medication List (e1) 10 1.00 0.00 8.5 0.707 1.05 .316 0.00 0.00
Page 13 of 26
Access Full Medication List (e2) 10 1.00 0.00 8.6 0.699 1 0 0.00 0.00
Change an Existing Medication (k6) 10 1.00 0.00 9.3 0.823 1.05 .316 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Access Active Medication List (e1)Change Demographics (i2)
Access Full Medication List (e2)
Change an Existing Medication (k6)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Accessing active medication confirmed 9 of 10 participants completing the task in the optimal
number of steps with 1 participant adding extra steps due to selecting the incorrect button. Accessing full
medication list confirmed 10 of 10 participants completing the task in the optimal number of steps.
Change existing medication confirmed 9 of 10 participants completing the task in the optimal number of
steps with one participant taking an extra step due to selecting the incorrect button.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: Accessing and changing medication list information was executed very quickly and
without hesitation
- Areas of Improvement: No known areas for improvement were noted at this time.
Criteria 170.315(a)(8) Medication Allergy List
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access Active Medication Allergies (a1) 10 1.00 0.00 18.1 1.969 1.05 0.316 0.00 0.00 Change an Existing Medication Allergy (a3) 10 1.00 0.00 26.2 3.521 1.05 0.632 0.00 0.00
Record a New Medication Allergy (a4) 10 1.00 0.00 41.6 3.406 1.02 0.316 0.00 0.00 Access Inactive Allergies in Medication Allergy List (a5) 10 1.00 0.00 12.9 2.234 1.05 0.316 0.00 0.00 Reactivate an Inactive Medication Allergy (a2) 10 1.00 0.00 22.2 2.201 1.1 0.422 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Access Active Medication Allergies (a1)Access Full Medication List (e2)
Page 14 of 26
Change an Existing Medication Allergy (a3)
Record a New Medication Allergy (a4)
Access Inactive Allergies in Medication Allergy List (a5)
Reactivate an Inactive Medication Allergy (a2)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Accessing active medication allergies confirmed 9 of 10 participants completing the task in the
optimal number of steps. Changing and existing medication allergy confirmed 8 of 10 participants
completing the task in the optimal number of steps. Recording a new medication allergy confirmed 9 of
10 participants completing the task in the optimal number of steps. Accessing inactive medication
allergies confirmed 9 of 10 participants completing the task in the optimal number of steps. Reactivating
an inactive medication allergy confirmed 8 of 10 participants completing the task in the optimal number
of steps. Participants that did not complete the task in the optimal number of steps simply selected an
incorrect button when trying to complete the task.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: These tasks were executed very quickly and without hesitation
- Areas of Improvement: No known areas for improvement were noted at this time.
Criteria 170.315(a)(9) Clinical Decision Support
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec) Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access CDS Interventions (d1) 10 1.00 0.00 8.7 1.059 1.05 0.316 0.00 0.00 Identify Due / Overdue CDS Interventions (d2) 10 1.00 0.00 8.3 0.949 1.1 0.316 0.00 0.00 Identify the guidelines for CDS Interventions (d3) 10 1.00 0.00 42 2.708 1.03 0.316 0.00 0.00 Locate Reference Information for a CDS Intervention (d4) 10 1.00 0.00 8.9 0.8756 1.1 0.316 0.00 0.00
Configure a CDS Guideline (j1) 10 1.00 0.00 29.5 4.007 1.03 0.316 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding Access CDS Interventions (d1)
Identify Due / Overdue CDS Interventions (d2)
Identify the guidelines for CDS Interventions (d3)
Locate Reference Information for a CDS Intervention (4)
Configure a CDS Guideline (j1)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
Page 15 of 26
- Efficiency: All tasks confirmed 9 of 10 participants completing the task in the optimal number of steps
with a single participant in each task selecting an incorrect button.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: Each of these tasks were executed very quickly and without hesitation
- Areas of Improvement: No known areas of improvement were identified at this time.
Criteria 170.315(a)(14) Implantable Device List
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access Current Device (c1) 10 1.00 0.00 26.6 3.204 1.025 0.316 0.00 0.00 Change an Existing Implantable Device (c2) 10 1.00 0.00 25.9 1.792 1.025 0.316 0.00 0.00
Record a New Implantable Device (c3) 10 1.00 0.00 26 1.414 1.05 0.632 0.00 0.00 Change device list to show current Device List (c5) 10 1.00 0.00 12.5 1.509 1.1 0.316 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Access Current Device (c1) Change an Existing Implantable Device (c2)
Record a New Implantable Device (c3)
Change device list to show current Device List (c5)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Accessing current devices confirmed 9 of 10 participants completing the task in the optimal
number of steps. Changing and existing implantable device confirmed 9 of 10 participants completing the
task in the optimal number of steps. Recording a new implantable device confirmed 8 of 10 participants
completing the task in the optimal number of steps. Changing device list to show current device list.
confirmed 9 of 10 participants completing the task in the optimal number of steps. Participants that did
not complete the task in the optimal number of steps simply se lected an incorrect button when trying to
complete the task.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: These tasks were executed very quickly and without hesitation
- Areas of Improvement: Consider making the button Font for Devices larger in the future.
Criteria 170.315(b)(2) Clinical Information Reconciliation and Incorporation
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Page 16 of 26
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Access Retrieved C-CDA Document (h1) 10 1.00 0.00 8.6 0.966 1.033 0.316 0.00 0.00
Go to import CCD Option (h2) 10 1.00 0.00 9.4 0.699 1.1 0.316 0.00 0.00
Import C-CDA Data into Chart (h3) 10 1.00 0.00 86.9 5.131 1.02 0.316 0.00 0.00
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Discussion of Finding
Access Retrieved C-CDA Document (h1)
Go to import CCD Option (h2)
Import C-CDA Data into Chart (h3)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Accessing retrieved CCD doccument confirmed 9 of 10 participants completing the task in the
optimal number of steps. Go to import CCD option confirmed 9 of 10 participants completing the task in
the optimal number of steps. Importing CCD document into chart confirmed 9 of 10 participants
completing the task in the optimal number of steps. Participants that did not complete the task in the
optimal number of steps simply selected an incorrect button when trying to complete the task.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: These tasks were executed very quickly and without hesitation
- Areas of Improvement: Consider making the screen heading for Import CCD much larger.
Criteria 170.315(b)(3) Electronic Prescribing
DATA ANALYSIS AND REPORTING
Task Scores N Task Success Task Time (sec)
Path Deviation Errors
Task # Mean
(%) (SD) Mean (sec) (SD)
Deviations Observed/
Optimal (SD) Mean (SD)
Create New Electronic Prescription (k3) 10 1.00 0.00 42.5 4.601 1.02 0.316 0.00 0.00 Send Prescription to specific pharmacy (k4) 10 1.00 0.00 18.6 1.174 1.05 0.316 0.00 0.00 Process Electronic Rx Request from Pharmacy (k8) 10 1.00 0.00 9.5 2.068 1.1 0.316 0.00 0.00
Discussion of Finding
Create New Electronic Prescription (k3)
Task Scenario Ratings Mean (SD)
Task Group Familiarity Ratings (5=very familiar) 5 0
Task Group Satisfactory Ratings (5=very familiar) 5 0
Page 17 of 26
Send Prescription to specific pharmacy (k4)
Process Electronic Rx Change Request from Pharmacy (k8)
- Effectiveness: Success for these tasks was 100% as participants were easily able to complete the tasks
within the time allowed.
- Efficiency: Creating electronic prescription, sending electronic prescription to specific pharmacy and
processing electronic request from pharmacy confirmed 9 of 10 participants completing these tasks in the
optimal number of steps. Participants that did not complete the task in the optimal number of steps
simply selected an incorrect button when trying to complete the task.
- Satisfaction: The average satisfaction rating for was 5 out of 5 points
- Major Finding: These tasks were executed very quickly and without hesitation
- Areas of Improvement: No known areas of improvement were identified at this time.
Overall Results
In addition to the specific findings and areas of improvement, the overall results of the test were excellent. This was
due in part to the participants being were well versed in using the MD Logic EHR software on a daily basis in clinic. As
test participants completed each task, there overall ability to efficiently navigate the EHR system increased with each
completed task. No known major issues were encountered with the test group. Test participants were effectively able
to follow each task assigned with total effectiveness.
As expected, the EHR software was designed for easy adoption in the healthcare industry. The EHR software has been
used in the healthcare industry for the past 20 years. The EHR Company develops its entire suite of software around
touch screen technology making it simple for uses to navigate. The study confirmed the EHR is intuitive, simple to
navigate and saves time. Having larger easy to touch buttons simplifies the learning curve and makes using the
software simple and easy.
APPENDICES The following appendices include supplemental data for this usability test report. Following is a list of the appendices provided: 1: Sample recruiting screener
2: Participant demographics
3: Non-Disclosure Agreement (NDA) and Informed Consent Form
4: Example Moderator’s Guide
5: System Usability Scale Questionnaire
6: Acknowledgment form
Appendix 1: SAMPLE RECRUITING SCREENER
Hello, my name is (callers name goes here) calling for MD Logic. We are recruiting individuals to participate in a usability study for an electronic health record. We would like to ask you a few questions to see if you qualify and if would like to participate. This should only take a few minutes of your time. This is strictly for research purposes. If you are interested and qualify for the study, you will be paid to participate. Can I ask you a few questions?
1. Are you male or female?
2. Have you participated in a focus group or EHR usability test in the past xx months? [If yes, Terminate]
3. Do you, or does anyone in your home, work in marketing research, usability research, web design […etc.]? [If yes,
Terminate]
Page 18 of 26
4. Do you, or does anyone in your home, have a commercial or research interest in an electronic health record software or
consulting company? [If yes, Terminate]
5. Which of the following best describes your age? [23 to 39; 40 to 59; 60 - to 74; 75 and older]
6. Do you require any assistive technologies to use a computer? [if so, please describe]
7. What is your current position and title? (Must be healthcare provider)
RN: Specialty
Physician: Specialty _
Resident: Specialty
Administrative Staff
Other [Terminate]
8. How long have you held this position?
9. Which of the following describes your highest level of education? [e.g., high school graduate/GED, some college, college
graduate (RN, BSN), postgraduate (MD/PhD), other (explain)]
Computer Expertise Customize this to reflect what you know about your EHR’s audience
1. Besides reading email, what professional activities do you do on the computer?
2. About how many hours per week do you spend on the computer? [Recruit according to the demographics of the
intended users, e.g., 0 to 10, 11 to 25, 26+ hours per week]
3. What computer platform do you usually use? [e.g., Mac, Windows, etc.]
4. What Internet browser(s) do you usually use? [Firefox, IE, Chrome, etc.]
5. Are you familiar with online meetings?
6. In the last month, how often have you used an electronic health record?
7. How many years have you used an electronic health record?
8. How many EHRs do you use or are you familiar with?
9. How does your work environment manage patient records?
On paper Some paper, some electronic All electronic
Contact Information If the person matches your qualifications, ask:
Those are all the questions I have for you. Your background matches the people we're looking for. Would you be able to participate on [date, time]? [If so collect contact information]. May I get your contact information?
Name of participant: Address: City, State, Zip: Daytime phone number: Evening phone number:
Before your session starts, we reserve the right to have you sign a release form giving us the option of videotaping your session.
The videotape would only be used internally for further study if needed. Will you consent to be videotaped? This study will take
place online. I will confirm your appointment a couple of days before your session and provide you instructions on how to login for
the test. What time is the best time to reach you?
Page 19 of 26
Appendix 2: PARTICIPANT DEMOGRAPHICS
The report should contain a breakdown of the key participant demographics.
Page 20 of 26
Page 21 of 26
Appendix 3: Non-Disclosure Agreement / Informed Consent Form
Non-Disclosure Agreement
THIS AGREEMENT is entered into as of ____/____/2017, between (“the Participant”) and the testing
Organization eMarketMD.
The Participant acknowledges his or her voluntary participation in today’s usability study may bring the Participant into possession
of Confidential Information. The term "Confidential Information" means all technical and commercial information of a proprietary
or confidential nature which is disclosed by Test Company, or otherwise acquired by the Participant, in the course of today’s study.
By way of illustration, but not limitation, Confidential Information includes trade secrets, processes, formulae, data, know-how,
products, designs, drawings, computer aided design files and other computer files, computer software, ideas, improvements,
inventions, training methods and materials, marketing techniques, plans, strategies, budgets, financial information, or forecasts.
Any information the Participant acquires relating to this product during this study is confidential and proprietary to Test Company
and is being disclosed solely for the purposes of the Participant’s participation in today’s usability study. By signing this form the
Participant acknowledges that s/he will receive monetary compensation for feedback and will not disclose this confidential
information obtained today to anyone else or any other organizations.
Participant Printed Name: __________________________________
Participant Signature: __________________________________
Date: ____________
Recording Consent Form
The session today will be recorded for to allow our team to go back and review your tasks completed. If you agree to the terms
below, the please sign and date the form.
I understand that my usability test session will be recorded. I grant eMarketMD the permission to use this recording for internal
use only, for the purpose of improving the products being tested.
Participant Printed Name: __________________________________
Participant Signature: __________________________________
Date: ____________
Informed Consent Form
We would like to thank you for participating in this study. The purpose of this study is to evaluate an electronic
health records system. If you decide to participate, you will be asked to perform several tasks using the prototype
and give your feedback.
Agreement
I understand and agree that as a voluntary participant in the present study conducted by eMarketMD. I am free to withdraw consent or discontinue participation at any time. I understand and agree to participate in the study conducted and give my consent to be videotaped.
I understand and consent to the use and release of videotape. I understand that the information and videotape is for research purposes only and that my name and image will not be used for any purpose other than research. I relinquish
Page 22 of 26
any rights to the videotape and understand the videotape may be copied and used for research. I understand and agree that the purpose of this study is to make EHR software applications more useful and usable in the future.
I understand and agree that data confidentiality is assured, because only de-identified data – i.e., identification
numbers not names – will be used in analysis and reporting of the results.
I agree to immediately raise any concerns or areas of discomfort with the study administrator. I understand that I can leave at any time.
Please check one of the following:
YES, I have read the above statement and agree to be a participant.
NO, I choose not to participate in this study.
Participant Printed Name: __________________________________
Participant Signature: __________________________________
Date: ____________
Appendix 4: Sample Orientation Introduction Thank you for taking the time today to participate in this study. You will be helping us evaluate workflows pertaining to the ONC’s EHR vendor certification requirements. I will be asking you to complete a variety of tasks in the MD Logic EHR software. Please remember that the reason for the study is to review the MD Logic EHR software and how users relate to each task assigned. If during a task you get to a point where you are not sure what to do please use your best judgement to proceed. Our goal is to observe how you handle each task. I have consent forms that I need you to sign before beginning the study? Once signed, we can begin the study. You will be asked to perform specific tasks in a workflow and can begin each task when I say “begin”. Once you have completed the task please say the word “done”. We will be recoding the times it takes to complete each task.
Appendix 5: User Protocol Tasks Scenario 1: Medication Allergy List
1. Zach Jones has come in today for a follow -up visit for his prehypertension, but he now complains of ringing in his left ear and bilateral sinus pressure
2. Open Zach Jones chart so that you can review his known allergies with him 3. Locate Zach’s active allergies (Begin Now) 4. Zach then states that he is not allergic to Iodine as noted in his health record. 5. Please update Iodine to Inactive in his chart. (Begin Now) 6. Zach says his reaction to Penicillin is not correct. Zach says he get a rash when he takes penicillin but does
NOT get shortness of breath. 7. Your task is to update the adverse reaction for Penicillin to rash. (Begin Now) 8. Zach also states that he gets a rash when takes Sulfa Drugs. 9. Your task is to add Sulfa drugs and the adverse reaction as rash. (Begin Now) 10. After adding Sulfa drugs Zach suddenly remembers that it was not Sulfa drugs he had a reaction too. It was
Latex. 11. Your task is to Inactivate Sulfa drugs(Begin Now) 12. Zach confirms he gets a rash with Latex. 13. Your task is to add Latex and the adverse reaction of rash. (Begin Now)
Page 23 of 26
Scenario 2: Problem List It has been several months since Zach has visited the practice. During the past few months Zach has lost 10lbs but
BMI is still high.
1. Go to Zach’s Problem list. (Begin Now) 2. Zach’s states that he controlling his prehypertension with eating healthy and daily exercise. 3. Update the Assessment for prehypertension to “Resolved”. (Begin Now) 4. Zach’s is having a recurrent problem with Tinnitus. 5. Locate Zach’s Inactive Problem List. Change the Status of Tinnitus to Active. (Begin Now) 6. After careful review, it is confirmed that Zach’s cholesterol is elevated / high. Add a problem for
Hyperlipidemia with an onset for today’s date. (Begin Now) 7. For the Hypertension (essential) problem, find the SNOMED code. (Begin Now)
Scenario 3: Implantable Device Since the last visit Zach had Cystoscopy (Balloon) for Gross Hematuria. Today, Zach was working on his car and
accidently cut his forearm. After examination you decide he requires and suture and implant a non-absorbable
suture.
1. Add Cystoscopy Balloon in Zach’s Chart. (Begin Now) 2. Since the balloon was used but not implanted, update the Cystoscopy Balloon as inactive. (Begin Now) 3. Add Zach’s suture to the device list. (Begin Now) 4. Change the Device List setting to show inactive. (Begin Now) 5. Change the Device List setting to show Current. (Begin Now)
Scenario 4: Clinical Decision Support To ensure the best quality of care is being provided, you need to check Zach’s health reminders.
1. Locate Health Reminders for Zach. (Begin Now) 2. Identify which reminders are due now. (Begin Now) 3. Identify which guidelines trigger the PSA reminder. (Begin Now) 4. You are not sure how often the patient should complete PSA. Look for the reference information for the PSA
reminder. (Begin Now) 5. The U.S. Preventive Services Task Force (USPSTF) recommends against prostate -specific antigen (PSA)-based
screening for prostate cancer. In MD Logic maintenance remove the PSA testing guideline for Males 50 to 60 years of age from the reminder.
Scenario 5: Medication List Next let’s review Zach’s Medications.
1. Locate Zach’s current medication list. (Begin Now) 2. After asking Zack about the current medications he is taking he confirms he is taking Crestor. Review Zach’s
past medications history to see if there are any other medications that have been prescribed to Zach. (Begin Now)
3. You identify Bactrim in Zach’s chart and he confirms that he is no longer taking the medication. Discontinue Bactrim. (Begin Now)
4. Zach confirms that he started taking Nexium OTC 20MG once a day. Please add Nexium to his Mediation List. (Begin Now)
5. If asked by the patient, how would you find the generic name for Prilosec.(Begin Now)
Scenario 6: Computerized Physician Order Entry (CPOE)
Page 24 of 26
Upon examination of Zach, he is complaining of bilateral pain in the upper sinus area and constant drainage (greenish
color). You diagnose Sinusitis and decided to create an order for a Nasal Endoscopy and order antibiotics to treat the
sinus infection. Do not save the order – simply add the orders.
1. Electronically order Augmentin 875mg twice a day for seven days to be taken with food. (Begin Now) 2. Next order a CT sinus. (Begin Now) 3. Next renew Zach’s prescription for Crestor. (Begin Now) 4. Finally, order a Lipid Panel. Please stop when the system confirms that you want to order the Lipid Panel.
(Begin Now)
Scenario 7: Drug Utilization Review Now you will save the orders
1. Select “save” the prescription for Augmentin and identify the priority warnings. (Begin Now) 2. Review the drug-allergy interaction and describe your options displayed. (Begin Now) 3. Review all warning and explain your options for the highest drug interaction warnings. (Begin Now)
Scenario 8: (CIRI) Clinical Information Reconciliation & Incorporation Part of the Meaningful Use measures requires interoperability and the exchange of clinical data between healthcare
providers. The data is available electronically or printed on paper and is referred to as a CCD document and
commonly referred to as a CCDA document. CCDA standard for Consolidated Clinical Document Architecture. The idea
behind the CCDA standards is to automate the sharing of clinical information (between doctors in different clinic) on a
patient. Your next patient has arrived and his name is Jeff Garmon. Jeff has brought his CCD file on a flash drive
provided by his other doctor. The medical assistant has already opened Jeff’s chart and imported the available health
data that you can import into Jeff’s electronic chart.
1. Open up Jeff’s chart. (Begin Now) 2. Got to Import CCD option. (Begin Now) 3. Now import and reconcile the new clinical information not already in Jeff’s chart from the CCD file. (Begin
Now) 4. After importing the clinical information, please demonstrate where you would go to view Health Reminders
that may be triggered after importing new health information into Jeff’s chart. (Begin Now). As part of meaningful use, data imported from CCD documents that meet the criteria for health reminders should trigger the reminders.
Scenario 9: Demographics – Clerical Samantha Smith is referred to you by a mutual friend and stops in to see if she can be seen today. Your staff collects
her insurance information and adds her basic demographic information (name, address) into the system. Normally
the staff would hand the patient a touch screen table where the patient would to the scree and answer demographic
questions required by meaningful use. This automation is provided by the MD Logic Kiosk software. Since Samantha is
a work-in patient, there are no available tablets for her to use to answer the meaningful use questions.
1. Add the following demographic information for this patient. (Begin Now)
Date of Birth 02-21-1982
Race White
Ethnicity Non-Hispanic or Latino
Sex Female
Sexual Orientation Hetero Sexual
Gender Identity Female
Language English
2. Upon completing the history, the patient questions Ethnicity and tells you she does not want to answer the question. Change her answer from Non-Hispanic or Latino to Patient Declined. (Begin Now)
Page 25 of 26
Scenario 10: (DUR) Drug Utilization Review Settings – Administrative For this example you are a system administrator with the practice. The doctors have brought it to you attention that
the (DUR) is reporting drug interactions that are insignificant and request the level of interactions be changed.
1. Go to DUR Configuration Settings. (Begin Now)
2. Explain how you could change the configuration so that the minimum severity of interactions reported
includes only major drug interactions? (Begin Now)
Scenario 11: Electronic Prescriptions Sending drug prescriptions electronically in the state of New York is the law. Doctor can no longer generate paper
prescriptions. Chris Turner is a patient you have not seen in 6 months. Chris has come back to the clinic complaining
of recent heart burn and reported that taking OTC Nexium 20MG is not working as well as it used to and wants to
know if there is something else he can take for heartburn. You explain to Chris that he can take (2) Nexium 20MG pills
each day. As an option, you explain that you can write him a prescription for Nexium 40Mg. Chris decides that he
wants the prescription for Nexium 40Mg.
1. Open Chris’s chart and view his Current Medications. (Begin Now)
2. Update Nexium 20MG as discontinued. (Begin Now)
3. Order an electronic prescription for Nexium 40MG to be taken once a day for 90 days. (Begin Now)
4. Chris has asked you to send the prescription to his new pharmacy (Publix off State Bridge Road). (Begin Now)
Chris returns back the clinic after in 30 days complaining of a sore throat and post nasal drip. Chris explains to you
that he looked into the cost of the Nexium 40MG prescription and said it was more than taking 2 OTC Nexium 20MG
tablet each day so he decided not to use the Nexium 40MG prescription. Void the Nexium 40MG prescription and
send electronic cancellation to his Pharmacy.
5. Update Chris’s charts that he is taking Nexium 20MG / 2 tablets once a day. (Begin Now)
6. Update Nexium 40MG as discontinued. (Begin Now)
The next patient is Kurt Kenyon and he is in the office today for right knee pain. After the examination, you decide
you are going to write Kurt a prescription for NDAIDS (anti-inflammatory medication).
7. Review his medication history to see if he has been prescribed NDAIDS (anti-inflammatory medication) in the
past. (Begin Now)
Your last patient just before lunch is a no show so you decide to review your Prescriptions
8. Find the prescription renewal request for Flomax for Eric Moore. Approve the prescription renewal. (Begin Now)
Appendix 6: System Usability Scale Questionnaire Please answer the following questions based upon your overall experience with the tasks you were asked to complete in the user Study.
1. I thought the system was easy to navigate
Strongly Disagree 1 2 3 4 5 Strongly Agree
Page 26 of 26
2. I found the system complex and hard to use
Strongly Disagree 1 2 3 4 5 Strongly Agree
3. I would like to use this system in a clinic setting Strongly Disagree 1 2 3 4 5 Strongly Agree
4. I found the system workflow was easy to follow Strongly Disagree 1 2 3 4 5 Strongly Agree
5. I believe users would quickly be able to learn the system Strongly Disagree 1 2 3 4 5 Strongly Agree
6. I did not feel confident using the system Strongly Disagree 1 2 3 4 5 Strongly Agree
7. I found the workflow of the system to be cumbersome Strongly Disagree 1 2 3 4 5 Strongly Agree
8. I believe a user would need a lot of training before being able to use this system Strongly Disagree 1 2 3 4 5 Strongly Agree
9. I believe the speed of the system was very fast Strongly Disagree 1 2 3 4 5 Strongly Agree
10. I believe that users could begin using this system with minimal training