Date post: | 27-Jan-2017 |
Category: |
Healthcare |
Upload: | gilberto-crespo |
View: | 73 times |
Download: | 1 times |
A Comparative Study of Nurses Accessing Electronic Patient Record Systems
with PDAs and Tablet PCs
Gilberto Crespo Pérez
Electrical & Computer Engineering DepartmentUniversity of Puerto Rico at Mayagüez
December 2005
Hospitals in Puerto Rico & United States keep their patients’ records in paper form.– Problems with information management effectiveness.– Handwriting may produce errors due to misunderstanding of
written information. Compromise patients’ health.
PDAs & Tablet PCs are two technological devices with potential for accessing and capturing clinical data at bedside. – Allow greater speed and effectiveness to compile patient’s
information.– Reduce the risk of errors during treatment of patients.– Improve physicians and nurses’ effectiveness and productivity
at point-of-care, as well as the quality of patient care.
Introduction
A review of the literature reveals that there are no formal studies comparing the use of PDAs and Tablet PCs for accessing electronic patient records.
We believe that such study could help determine which of the two technologies fits better for collecting and accessing clinical information at the point-of-care.
Introduction (Cont.)
Previous Work From 44,000 to 98,000 people die annually in U.S.
hospitals as a result of medical mistakes -- more than half of them preventable [Rosenbloom03].
Trying to correct some of these errors, the Veteran’s
Administration (VA) has taken effective measures through electronic system controls and implementation [Rosenbloom03]
– Over a five-year test period, the medication error rate dropped 70%
Previous Work (Cont.)
The Use of Portable Devices in Health Care: E-prescribing, ordering, checking labs tests, dictation
notes among others - [Fischer03, Ying03, Rosenbloom03, Barret02,].
Clinical Applications for Portable Devices: Drug references, pharmacopoeias, medical
calculators, and patient trackers among others -[Adatia03, Rosenbloom03, Choi00, Kimura03, van der Velde01, Berner04].
Previous Work (Cont.)
Usability Studies of Portable Devices in Health Care: PDA vs Laptop & PDA vs Paper Based [Rodríguez02,
Rodríguez03, Rodríguez04, Staggers00, Stausberg03]. Users performance, subjective satisfaction, and preference were
dependable variables.
Usability of Tablet PC [Narayan04, Andon04]. Physical Aspects.
No more studies related to other usability issues of Tablet PCs have been found, which indicates that this field is in its early stages of research.
Objectives
Conduct a usability study to compare the use of PDA and Tablet PC-based applications to support nurses’ tasks at point-of-care.– These studies compare both systems in terms of
performance parameters: Time to complete tasks.
Number of tasks completed. Subjective user satisfaction.
– Identify: Potential usability problems. Interaction differences. Advantages and Disadvantages.
Systems Tablet PC version
– Implemented on a Gateway tablet.– Windows XP Tablet PC OS.– Uses a Stylus as a pointing device.– A soft keyboard provided by OS was used for text input.
PDA version– Implemented on a HP iPaq 5500.– Windows Pocket PC 2003 OS.– Uses a small stylus as a pointing device.– A soft keyboard provided by OS was used for Text input.
Systems (Cont.)
The Tablet PC version– Developed in Java.
The PDA version – Developed in C Sharp.
Both applications use:– MSSQL to store and retrieve the patient’s clinical
information in a database.– The standard IEEE 802.b to communicate with the
database server.
Graphical User Interface
Patient List Windows
Graphical User Interface (Cont.)
Medicine Windows
Graphical User Interface (Cont.)
Intake/Output Windows
Participants
20 staff nurses– Selected on a first-come first-served basis from those that
responded a call for participation.– Experience as staff nurses ranged from 1 to 27 years
(Mean=12.3 years).– Experience with computers ranged from 0 to 9 years
(Mean=4.6 years).– On average, they used computers on their job for 3.8 hours
per day.– None of them had prior experience with the systems used
for the study or with any similar application.– None of them had experience using PDAs or Tablet PCs.
Experimental Design
Participants were asked to fill out a questionnaire about their work experience and their experience using computers.
They were asked to sign a letter of consent. An orientation script was read to each participant
explaining the objective of the test. Nurses were given a short tutorial session on the graphical
system.– 18 minutes for the PDA version.– 13 minutes for the Tablet PC version.– They were allowed to use the system by themselves and were
guided to practice using each of the functions of the system.
Half of the nurses performed the tasks first on one system and the other half started on the other system.
Once the participants complete all the task in both versions, they were asked to fill a user satisfaction questionnaire.
Experimental Design (Cont.)
Tasks1. Indicate patient’s age and weight. Say them aloud.2. Indicate the most recent registered patient’s temperature.3. Look for the most recent nurse note and read it aloud.4. Acknowledge any pending medication order as administered.5. Enter the following set of vitals signs:
Temp: 37.0o C; BP: 130/90; Pulse: 71; RR: 18; O2 Sat: 96%. 6. Indicate the total balance of intake/output of fluids in the last
24 hours.7. Look for a Dr. Colón note.8. Enter the following patient assessment information provided.9. Enter the following I/O information:
Intake PO: 50 ml; Output Urine: 650 ml;
10. Enter the following text as a note:Patient presents fever and a large lung mass on left upper lobe consistent on CT with multi-focal pneumonia
11. Enter the following patient’s pain information.Classification: Four; Location: Right frontal shoulder; Description: Hurt; Therapy: Massage; Administered medicine.
12. Look for most recent physician consult order and acknowledge it as that was notified by phone.
13. Show where is indicated in the record the reason why a dose of Roboxine was omitted.
Tasks (Cont.)
User Satisfaction Questionnaire
Interaction Aspects:– Look up patient information– Acknowledge medication orders– Acknowledge consult order– Enter vitals signs– Document intake/output– Document patient assessment– Document pain assessment– Read a note– Write a note
Nurses were asked to rate on a 1 – 7 scale (1 being poor & 7 being excellent)
System Aspects:– Record organization– Trustworthiness of information– Precision of Information – Accessibility of information– System security
Physical Aspects:– Use of the stylus– Use of the screen keyboard– The screen– The weight– The portability
Level of Satisfaction with each system The system they would prefer for doing their
nursing documentation.
User Satisfaction Questionnaire (Cont.)
Statistical Analysis
Dependent variables were:– Task completion time.– Number of tasks completed.– Subjective user satisfaction.
Time analysis – Paired t Test User Satisfaction – Wilcoxon Signed Ranks Number of Task Completed – Wilcoxon Signed
Ranks
Results: Overall Completion Times
656.2 667.3
191.7215.6
0.0
100.0
200.0
300.0
400.0
500.0
600.0
700.0
Seco
nds
Mean Std. Deviation
PDATablet PC
Dependent Sample t test did not reveal significant difference between average time it took the participants to complete all the tasks on both system.
Results: Overall Completion Time (Cont.)
0
50
100
150
200
250
Com
plet
ion
Tim
e (
Seco
nds)
1 2 3 4 5 6 7 8 9 10 11 12 13Tasks
PDATablet PC
Results: Completion Times for Each Task on Both Versions of the System
Dependent Sample t test did not reveal significant difference in the time it took the participants to complete each of the tasks on both versions of the system.
Results: Completion Times for Each Task on Both Versions of the System (Cont.)
Results: Number of Participants that Complete Each Task on Each Version of the System
0
2
4
6
8
10
12
14
16
18
20N
umbe
r of
Par
ticip
ants
1 2 3 4 5 6 7 8 9 10 11 12 13Tasks
PDA
Tablet PC
Wilcoxon Signed Ranks did not reveal significant difference in the total number of tasks completed by the participants.
The participants completed an average of 12.0 tasks on the PDA and 11.7 on the Tablet PC
Results: Number of Participants that Complete Each Task on Each Version of the System (Cont.)
Results: Average Satisfaction Ratings for Individual Interaction Aspects
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
Rating
Look up patient informationAcknowledge medication orders
Acknowledge consult orderEnter vitals signs
Document intake/outputDocument patient assessment
Document pain assessmentRead a noteWrite a note
Inte
ract
ion
Aspe
cts
PDA Tablet PC
Wilcoxon test did not reveal significant a difference on each individual interaction aspects considered.
Also, Wilcoxon Test did not reveal a significant difference in the overall average satisfaction rating given to the 9 interaction aspects.
Results: Average Satisfaction Ratings for Individual Interaction Aspects (Cont.)
Results: Average Satisfaction Ratingsfor Individual System Aspects
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
Rating
Recordorganization
Trustworthinessof information
Precision ofInformation
Accessibility ofinformation
Systemsecurity
Syst
em A
spec
ts
Tablet PCPDA
Wilcoxon Test did not reveal a significant difference in the overall average satisfaction rating given to the 5 system aspects.
The overall average satisfaction rating for the PDA and the Tablet PC was 6.8
Results: Average Satisfaction Ratingsfor Individual System Aspects (Cont.)
Results: Average Satisfactions Ratings for Individual Physical Aspects
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
Rating
Use of the stylus
Use of the screen keyboard
The screen
The weight
The portability
Phys
ical
Asp
ects
Tablet PCPDA
Wilcoxon Test revealed a significant difference in overall satisfaction with the five physical aspects considered.
Wilcoxon Test also revealed significant user satisfaction rating for:– The use of the Stylus– The weight of the device– The portability of the device
Results: Average Satisfactions Ratings for Individual Physical Aspects (Cont.)
Participants were more satisfied with:– The PDA computer (Average Rating = 6.6) than with the
Tablet PC computer (Average Rating = 5.2).– The use of the stylus on the PDA (Average Rating = 6.7)
than on the Tablet PC (Average Rating = 6.2). – The weight of the PDA (Average Rating = 6.5) than with
the weight of the Tablet PC (Average Rating = 4.1).– The PDA portability (Average Rating = 6.6) than with the
Tablet PC portability (Average Rating = 3.5).
Results: Average Satisfactions Ratings for Individual Physical Aspects (Cont.)
Overall Results
No significant differences were found in the No significant differences were found in the overall satisfaction rating given by the overall satisfaction rating given by the participants to each version of the system.participants to each version of the system.
However, the majority of the participants However, the majority of the participants preferred the PDA over the Tablet PC version preferred the PDA over the Tablet PC version for performing their duties.for performing their duties.
Discussion No significant difference was found in the overall time it
took nurses to complete tasks on both portable devices.– It contradicts our expectations that nurses would be faster using
a larger user interface than a smaller one. Our results are consistent with the results of the study by
Rodríguez et. al. [Rodríguez03]– When the task of writing a note was not considered, nurses were
able to complete nursing tasks in similar overall times on the PDA and on Laptop.
The use of a stylus, the screen size, and resolution is not a factor in nurses’ performance.
Discussion (Cont.) Nurses were as fast on a PDA as they would on a
Tablet PC.
Younger nurses tend to take shorter times in completing tasks.
Due to the short training given to nurses to teach them how to use the systems and the high percentage of tasks completed:– Results indicate that both nursing documentation systems
are very easy to learn .– Similar results were found in the [Rodríguez03] study.
Discussion (Cont.)
The word “document” in tasks T4, T6, T8, and T12 seems to create confusion in some of the nurses to the point that they did not find the way to complete the task successfully.
Nurses expressed similar high levels of satisfaction with the documentation activities performed in both, the PDA and Tablet PC systems.
They were significantly more satisfied with the physical aspects of the PDA than those of the Tablet PC
Discussion (Cont.) Many of the participants (12 out of 20) expressed
discomfort in holding the Tablet PC in a standing position during the test session. – This result is consistent with the studies conducted by:
Michael A. Narayhan [Narayhan04] Christopher L. Andon [Andon04]
Fourteen of the nurses indicated that they preferred the PDA version for performing nursing documentation tasks while only four preferred the Tablet PC.
Conclusions Our results supports that it is possible to design PDA-
based applications that allow nurses to achieve similar performance and satisfaction levels as with a Table PC-based.
Screen size and display resolution of the PDA are no factors that limit nurses’ performance and satisfaction in comparison to Tablet PCs.
Nurses were significantly more satisfied with the physical aspects of the PDA than with those of the Table PC.– Aspects such as the use of the stylus, the weight, and portability
are factors that influence nurses’ satisfaction with the system.
Conclusions (Cont.) These applications exhibited a high degree of
learnability. Differences on user interfaces have not affected users
satisfaction. Usability engineering principles and guidelines
– Are critical part of the graphical user interface success.– Proved to be powerful and important tools for measuring the
efficiency of the system. The age seems to be a factor that affects nurses’
performance..
Conclusions (Cont.)
Considering that:Considering that:– Satisfaction with the physical aspects of the systems Satisfaction with the physical aspects of the systems
was the only dependent variable for which a was the only dependent variable for which a significant difference was found in favor of the PDAsignificant difference was found in favor of the PDA
– That 14 out of 20 participants selected the PDA over That 14 out of 20 participants selected the PDA over the Tablet PC the Tablet PC
– And the relatively low costs of the PDAAnd the relatively low costs of the PDA
We can conclude that PDAs are a better alternative We can conclude that PDAs are a better alternative for supporting nursing documentation tasks at for supporting nursing documentation tasks at bedside than Tablet PCs.bedside than Tablet PCs.
Future Work
Other usability studies should be conducted:– After the users have had a year of experience with the system.– With physicians interacting with each version of the system used
on this research.
Application improvement:– Incorporation of printing and supplemental language options
capabilities.– Speech recognition functionality.– Convert to a web based application.
Questions/CommentsQuestions/Comments
Usability Concepts Definitions
Usability: the extent to which the intended user can meet his or her goals using the system being tested.
Learnability: easy to learn -> time a novice user takes to complete tasks using the system.
Efficiency: high level of productivity -> time that users with certain expertise take to complete typical tasks.
Memorability: easy to remember. Errors: Low error rate and easy to recover. Satisfaction: users are subjectively satisfied -> ask users opinions.
When replies from multiple users are average, the result is objective. -questionnaire with Likert scale.
General Concepts Definitions
Ethical Aspects with Human Subjects:– Respect– Comfortable environment– Confidentiality– Early success experience
Statistical Tools Definitions Dependent-samples t test: used to compare the
time to complete the tasks -> same users, 2 measurements.
Wilcoxon Signed Rank test: compare differences in user satisfaction -> dependent samples.
Correlation analysis: determine associations between time to complete the tasks, computer literacy, typing skills, age, and eye glasses, among others.
Linear regression: evaluate the learning effect between 2 systems.