j ourna l homepage: www.e lsev ie r .com/ locate /apnr
Original Articles
Challenges and facilitators to nurse use of a guideline-based nursing informationsystem: Recommendations for nurse executives
Paulina S. Sockolow, DrPH, MBA a,⁎, Michelle Rogers, PhD b, Kathryn H. Bowles, RN, PhD c,Kristin E. Hand, BSN, RN d, Jessie George, RN, MSN e
a Drexel University College of Nursing and Health Professions, 245 N 15th St, Philadelphia, PA, USAb Drexel University College of Information Science and Technology, Philadelphia, PA, USAc University of Pennsylvania School of Nursing, Room 340 Fagin Hall 418 Curie Blvd. Philadelphia, PA, USAd University of Pennsylvania School of Nursing, Philadelphia, PA, USAe University of Pennsylvania Health System, Philadelphia, PA, USA
Received 25 February 2013Revised 17 July 2013Accepted 28 October 2013
Keywords:Evaluation studiesClinical information systemsEvidence-based nursingNursing informatics
Aims: The aims of this study were to develop empirical data on how nurses used an evidenced-based nursinginformation system (NIS) and to identify challenges and facilitators to NIS adoption for nurse leaders.Background: The NIS was part of the electronic health record with 200 evidence-based, interdisciplinaryclinical practice guidelines from which clinicians selected to guide the patient's care.Methods: A purposeful sample of 12 randomly selected nurses in three units across two hospitals participatedin scenario-testing. Sessions were audio-recorded, transcribed, content analyzed, and coded for themes.Results: Major themes emerged: computer placement in patient rooms; difficulty using NIS; documentationcompleteness; efficiency; time spent at the bedside; team communication; training; unintendedconsequences of workflow changes; perceived NIS value as challenge to adoption.
Conclusions: Nurse executives' opportunities to improve adoption include enhancing communication to/fromfront-line clinicians about the hospitals' goals, perceived NIS value at the bedside, and constructive feedbackespecially for patient care/safety and software functionality.
Nursing information systems (NISs) are promoted as a technologysupporting collaboration and improving health care decision makingat the point-of-care and ultimately health care outcomes. An NIScontains data collection and integration functionality for nurses andcould be used as a part of an electronic health record (EHR) or inconjunction with an electronic medical record (EMR). NISs have thepotential to improve the processes of obtaining patient history andcare planning and to increase nursing documentation completeness,readability, and availability. NISs also provide the means to decreasedouble documentation and assist with more precise compliance withlegal documentation requirements (Ammenwerth, Rauchegger,Ehlers, Hirsch, & Schaubmayr, 2011). However, a recent systematicreview found no evidence of measurable impact of nursing recordsystems on nursing practice and patient outcomes. The reviewincluded only two hospital studies of NIS: both assessed quality ofdocumentation (Urquhart, Currell, Grant Maria, & Hardiker Nicholas,2009). Due to the scarcity of hospital NIS studies, relatively little isknown about the impact of the increasing adoption and use of NISs inhospitals. There is a larger body of literature on EHRs. These systems
chronologically order patient clinical information captured bysystems. These include order entry and results reporting systemssuch as laboratory, pharmacy, and radiology, as well as medicationadministration systems. While nurses are the end users of EHRs, verylittle is known about how nurses are affected, and whether theyassociate EHRs with quality care and patient safety (Kutney-Lee,2011). Studies have shown that nurses are frustrated with theinconveniences of EHRs such as poor impact on nursing workflow(Stevenson, Nilsson, Petersson, & Johansson, 2010), increased work-load, and high frequency of irrelevant notifications or alerts (Sassen,2009; Sidebottom, Collins, Winden, Knutson, & Britt, 2012). As a resultof such frustrations, nurses are less likely to use the EHR as intended(Sockolow, Lehmann, Bowles, & Weiner, 2009). To address thisknowledge gap, the study described in this article focused on ahospital-based NIS, using a strong research design for the evaluation.
Implementation of NIS is relatively new, and due to a lack ofevaluation studies, it is not well understood. This study focused on anurban, non-profit, academic, health system that implemented an NISin its hospitals in 2011. The health system's goal in implementing thisNIS was to promote patient safety and improve patient outcomes by:(1) standardizing care and reducing variability in clinical practiceamong the clinical disciplines with evidence-based clinical practiceguidelines (CPGs), and (2) supporting nurse provision of patient-centered care. Nursing leadership expected that the NIS would save
26 P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
time and improve the quality of care provided. The health systemwanted to learn from the evaluation whether the NIS made adifference in nurse practice and whether nurses were satisfied withthe NIS. The research question was, what were the challenges andfacilitators to NIS adoption? The purpose of this paper is to presentempirical data on how nurses used an evidenced-based NIS and, basedon the study findings, provide practical guidance about adoption ofNIS designed to support clinical process and decision making.
1. Methods
The researchers conducted a qualitative study with staff nursesusing the NIS. The research team obtained approval from theresearchers' academic institutions' institutional review boards. Thestudy received strong backing from nurse leadership at the healthsystem level as well as in the individual hospitals.
1.1. The intervention
The NIS is a module within a nationally known electronic healthrecord (EHR) previously implemented throughout the hospitalsystem. The NIS provides approximately 200 evidence-based, inter-disciplinary clinical practice guidelines (CPGs) from which cliniciansselect to guide patient care.
Care plans, designed with information from CPGs, are furtherindividualized by nurses for each patient. Content from the care planpopulates throughout the assessment and education flowsheetsproducing a comprehensive and detailed assessment specific to thechosen plan of care, prompting nurses to recognize importantelements of the selected care plan. For example, for a patient admittedfor chemotherapy, a nurse selects a chemotherapy CPG for the plan ofcare which incorporates or embeds chemotherapy specific assess-ment elements in the assessment and education flowsheets. Theseembedded elements in the documentation fields are intended toreduce variability in nursing care. The NIS is accessible on newlyinstalled computers in each patient room.
1.2. Site
The study took place in two hospitals within a three-hospitalhealth system. The flagship hospital has 760 beds and the other has300 beds. Nurses attended 8 hours of training in advance of the NISimplementation. They were instructed to document almost allpatient-related issues in the NIS including the admission assessment,physical assessment, educational interventions, vital signs, intake andoutput measurements, medication administration, assessment find-ings, interventions completed, and significant event summaries. Twokey benefits emphasized in the training were: (1) reduction ofredundant questions addressed to patients by clinicians in variousdisciplines; and (2) support for patient-centered care. An example ofthe latter is eliciting from the patient an individualized statementindicating information about unique patient needs not readilyapparent from his or her diagnosis or CPGs. “Super-users” (nursesidentified by managers to have advanced user skills) providedongoing support on each unit.
1.3. Participants
Nurses were selected purposefully to insure variety in the studyparticipants. The nurses worked on units that met the followingcriteria: (1) were representative of most units in the hospital (e.g.,medical or surgical unit); (2) had a conference roomwith a computerwhere the study could be conducted; and (3) had a contact person onthe unit known to a research team member (to facilitate introductionof the study to the staff). Eligible participants were registered nurseswho provided and documented direct patient care. Nurses were
selected from among those working that day and were asked toparticipate if they could spare 20 minutes away from their work.Individually, a teammember obtained the nurse's consent outside theconference room. To protect the nurses' identities in such as smallsample, no demographic or identifying information was collected.
1.4. Procedures
The research team was composed of academic researchers andclinical nurses who conducted the evaluation from March to May of2012. The team used scenario-based user testing, presented as amodified think-aloud protocol (Nielsen, 1993) which is a standardmethodology used to elicit data about cognitive reasoning that occursduring a problem solving task. In a conference room with a computer,the research team presented the previously prepared scenarios to thenurses and simultaneously conducted follow-up interview questions(shown in Fig. 1) while observing the nurses using the system. Thescenarios were designed by the investigators to have the nursesinteract with the major components of the NIS. For example, onescenario asked the nurse to document a patient fall. A different groupof randomly selected multiple scenarios was presented to each nurse,ensuring that all scenarios and questions were asked at least once foreach unit. The 20 minute sessions were audio recorded andtranscribed. Three researchers (i.e., PS, MR, KB), independentlyperformed directed content analysis for challenges and facilitatorsto NIS adoption, analyzing the transcripts of the answers to thescenarios and interview questions. The inter-rater agreement goalwas 100% and discussion was held until it was reached. Similarly, thedata were individually coded for themes. After the coding wascompleted, the themesweremapped to the concepts and componentswithin the Health Information Technology Reference-based Evalua-tion Framework (HITREF). The HITREF is a comprehensive frameworkfirmly grounded in research evidence that provides a comprehensivelist of 20 evaluation criteria related to HIT characteristics (Sockolow etal., 2009). During the analysis process, questions that arose werereferred to the authors (i.e., KH,JG) who worked as nurses in thehospitals. These authors provided validation as well as facilitatedmember checking validation among their colleagues.
Following interpretation of the findings, the team developedsolutions to the identified challenges to adoption. The team alsoidentified existing facilitators to adoption and proposed avenues tosupport or enhance these facilitators. Health system nurse executivesreceived the team's final recommendations.
2. Findings
Participants were 12 registered nurses. They were from two unitsof the flagship hospital and one unit of an acute care hospital—fournurses from each unit. To protect their anonymity no socio-demographics were collected.
Transcripts from the scenario testing sessions revealed thefollowing themes. Further analysis and synthesis identified thechallenges and facilitators to NIS adoption.
1. Hardware referred to placement of the computer in the patientroom. Nurses reported they tended to chart in the roomespecially for complex patients and were less likely todocument at the bedside for uncomplicated patients. As onenurse stated, “I find especially withmy xxx patients, I really likeit because they have so much going on, they have a-lines, theyhave chest tubes, they have epidurals, and it's easier to be like,‘forgot to look at this’, and you can just peek at them real quickand finish your charting. And that part I do it in the room withthose patients, because I don't want to miss something. Withthese patients back here [observation patients] I don't usuallydocument at the bedside, cause it's pretty straightforward.”
Fig. 1. Scenarios and questions interview guide for NIS scenario testing.
27P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
Nurses reported they documented medications, vital signs,giving blood, and hourly rounds in the room. Nurses also usedthe NIS in the room when reviewing information with thepatient. One respondent noted that having the computer in theroom enabled him or her to look at the patient whiledocumenting. However, a number of participants mentionedthat they did not use the computer in the patient room for someor all their documentation. Universally nurses reported theydid not document admission assessments in the room due to
the time requirement (e.g., 20 minutes), as one nurse stated, “Ithink it's [in-room assessment documentation] fine, if thepatient's okay with it, and you have time to do it. I wouldn'twant to stand there for 20 minutes documenting.” Some nursesreported the patient or family talking to themwhile document-ing could distract them. For example, one nurse reported, “A lotof patients talk to you and interrupt your thought. And even if,in the beginning we were really encouraged just to do it at thebedside no matter what, and to just remind the patients that
28 P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
we were doing it. But some people just, they're bored, theywant to chat, or ask questions.”One participant said that he/shedid not like having her back to the patient when documenting,indicating computer placement was an issue. The authors (KH,JG) and their colleagues concurred that some nurses do notdocument the admission assessment in the room. However,some nurses do document in the room, and some begin theadmission assessment in the room and complete it on acomputer outside the room.An unintended consequence related to hardware placementwas that computers in patient rooms and hallways weredesigned for use while standing and did not offer users achance to sit and have some respite. As a nurse described, “Ifyou are standing there documenting for several minutes, younever get any reprieve, really…[get off your feet].”
2. Software Quality focused on software usability. Key topics werenavigation and whether the screen flow matched the clinicalworkflow, as described more fully elsewhere (Rogers, Sock-olow, Bowles, Hand, & George, in review). Briefly, respondentsreported the system was difficult to scroll through, requiredmuch scrolling to look back at previous documentation (as onenurse stated, “you have to scroll, scroll, scroll”), and the screenflow skipped among screens. As one nurse said, “I'm alwaysafraid I'm going to miss something, and [the system] tends tojump around.”Respondents described situations where screen flow did notsufficiently match clinical workflow such as where the NIScaused changes in patient discharge workflow. The systemintroduced one change perceived as increasing accountabilityand a second change that was viewed as introducing a delay inworkflow. The former change forced nurses to document in achecklist that they reviewed discharge plans/education withthe patient. As one nurse explained, “we used to also review theorders, the discharge orders, in the computer, and we can'treally do that: We have to wait until it prints out and then kindof like review it there, andmake sure everything's on it, so it's alittle more difficult…” The second change forced nurses to waitfor the physician to “click the box” for the discharge orderbefore the patient signature pages print and dischargeeducation could begin. The unintended consequence of thedelay introduced in this workflow change could lead to rusheddischarge education.Participants noted thedifficulty infinding their desiredguideline.Causes for difficulty included too many guidelines and too manypages of documentation. One nurse explained, “there are somany [CPGs], and the one that you want is so hard to find.”Respondents also noted that guidelines were not sufficientlyspecific or not focused on frequent conditions, such as hyper-glycemia and hypoglycemia. Nurses described their solution aschoosing a more general guideline for pain, or for diabetesinstead. This required themto spendmore timesearching theNISfor the appropriate guideline. One nurse described the followingsituation: “I've noticed that there aren't ones [guidelines] forhyperglycemia and hypoglycemia. I do see that a lot on the floor.So then I'm like stuck between a rock and a hard place. I knowthere's a diabetic oneat least, so I'll use that one. I'veused anotherone before, too, and I can't rememberwhich one I used. But I kindof go along with their main diagnosis and what they've come inwith. My fallback is acute pain. I'll be honest, if I can't findanything else I choose acute pain.”
3. Functionality related to the capabilities and features of thesoftware. Nurses appreciated specific functionality such as thepatient education functionality because it was easier to educatethe patient with the NIS in front of her/him. Another nurseacknowledged the helpfulness of memory prompts in thepresentation of the clinical guideline: “I guess it's helpful a little
bit, because it incorporates into my assessment sheet littlethings that I should look for, so it's helpful in that sense.Especially some of the times we get surgeries I'm not familiarwith.” Nurses also liked the “copy forward” functionality whichallowed nurses to review the patient's previous admissionassessment and copy any information that still applied. As onenurse said, “I do love the fact that when you have people thatcome in again and again, that their admission sheet's alreadydone. Copy forward is my favorite thing.”Respondents highlighted functionality deficits, including theNIS's inability to summarize what happened on the shift foreach patient in a designated area, leaving it to the nurse towrite a narrative summary. Another functionality deficit wasrelated to a key metric, patient falls. If a nurse missedcompleting the fall risk assessment, there was no reminder toindicate that area of the assessment was incomplete. Further-more the NIS did not have a designated procedure forcommunicating falls or other significant events. One respon-dent suggested having a tab for significant events instead of thenurse needing to scroll back, or more likely, depend on the shiftreport to receive the information. In addition, one nursementioned that he or she tended to enter information usingfree text instead of the structured check boxes: “I free text a lot,'cause sometimes not everything is in there that you need. Orother times it's just easier to free text towrite in something thatfits better, or makes more sense. 'Cause not everything is in the20 check boxes….” Respondents suggested other functionalityimprovements. Instead of looking in two places to find patients'height and weight entered by nursing assistants, informationcaptured on one screen should also populate the same datafields on other screens. Also, participants suggested additionalfunctionality to alert the physician or nurse practitioner aboutsignificant events.
4. Documentation Quality included the completeness and timeli-ness of the documentation. One nurse felt that documentationquality improved due to wide system use saying, “I don't reallyknow if a lot has changed as far as howwe give care, but we justdocument it better now.” However, the documentation'srichness and completeness may have been limited due tohow the system was used. Nurses reported they seldomselected more than one CPG, because each additional CPGselected added more fields requiring documentation: “Andthenwhen you do find it [CPG], there's like 17 pages of stuff.” Inaddition, the use of ‘copy forward’ functionality may compro-mise the correctness of the data. The concern is that nursesmust carefully review the previous information to see if it wasstill accurate. For example, if a patient had a wound on the lastadmission that is no longer present on the current admissionand a nurse copies all of the information forward withoutreading and editing, the current documentation would statethat the wound was still present.
5. Clinician Satisfaction was found with the NIS. Observationindicated that all nurses who provided direct patient care usedthe system. Overall, respondents reported being satisfied withthe NIS. They were unanimous in their preference fordocumenting in the system rather than using paper records:“I'm more comfortable with a computer, but I wouldn't want togo back to paper.”
6. Efficiency focused on ease of use. Nurses reported that patienteducation was easier and faster to document by using check-boxes. As one nurse stated, “it's [patient education] less timeconsuming when you have that stuff at your fingertips…It's justquicker, which in turn makes your patient care better, cause it'smore efficient.” However, nurses identified NIS-related changesthat decreased their efficiency. As noted above, the systemintroduced a step in the clinical process—forcing a nurse to wait
29P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
for the physician tofinalize discharge orders before documentingthe education of the patient. Nurses pointed out that this changecould lead to rushed discharge education. Another inefficiencyreported by nurses was documenting the same information intwo places in the system (e.g., goal section and discharge note).As a nurse explained, “When I review everything, then I do it inthe goal section, where we normally do our notes. I do like adischarge note that says I reviewed everything with them, andgave them all their prescriptions, answered all their question,and the patient left the unit at whatever timewith whoever. So Ido it in both places.”
7. Impact on Patient Care is related to whether the NIS changed theprocess related to how clinicians provided care. Nursesappreciated the availability of information relevant to patientcare (e.g., key expert resources). As one nurse stated, “Anothergood thing is when you are giving out meds there is expert info,you can right-click on that, and it tells you right in the room,you can go over the side effects and everything. That's nice, andit would help promote quality care, standardized care.”However, another nurse described the resources as havingtoomany pages to be usable. Another information resourcewasprior assessments, which one nurse reported was helpful to seeif the patient's condition had changed.A second aspect of Impact on Patient Care was the memoryprompt function in the system. Nurses acknowledged thebenefits of reminders about completing items in the patientassessment, as one nurse stated, “[the CPG] kind of triggerswhat I tend to miss in an assessment. It triggers what I actuallyshould be looking for.” On the other hand, some nursesreported they did not find the computerized CPGs useful asthey provided clinical care. One nurse observed that he or shechose a guideline only because it was required, not because itwas helpful. Another nurse explained that the NIS system didnot provide new information for the clinical process: “I thinkwe already kind of had that, we had the [guidelines], which Ithink, at least I hope, we've already been doing, not givingourselves credit for it. So I don't really know if a lot has changedas far as howwe give care. But we just document it better now, Ithink, cause you didn't have all those check boxes you filled outbefore.” A third nurse viewed the system as providing no inputinto the care process because the nurse did not incorporate theguideline into his or her care provision: “I kind of dictate myown care, and make [NIS] a part of it, I don't use [NIS] to dictatemy care.” In addition, nurses discussed their lack of criticalthinking as they quickly click through checkboxes whendocumenting patient education. As one nurse stated, “for allthis ‘education’ again you're clicking off, I can do it in my sleepyou know, sometimes, just 'cause I've been here for a while, youjust go through it and sometimes, you don't read it—you know?Just, it's second nature, and I'm not sure who's looking at thatbut literally you are just clicking off that you educated thispatient, which we do, but I think there should be a better way.”A third aspect of Impact on Patient Care was the time spent atthe bedside. Two nurses pointed out the advantages ofaccessing the NIS in the room. One nurse observed it waseasier to educate patients with the NIS in front of him or her.Another nurse stated that documenting in the room resulted inthe nurse spending more time at the bedside while charting: “Iprobably spend more time at the bedside because of it, cause Isign out my meds at the bedside, and I'm able to actually do alittle bit more charting, just on like rounding and stuff. Whereasbefore I couldn't find the bedside chart or something, then I'dforget about it, so it definitely makes it easier like that.”However, a different nurse described use of the NIS as timeconsuming such that nurses feel they spendmore timewith theNIS than with their patients: “…It's still time consuming to the
point where we are spending so much more time documentingthan we are with our patients.”An aspect of patient care, patient-centered care, was anintended focus of the NIS. However, nurses reported theywere frustrated having to spend time asking patients anddocumenting questions in the admission assessment intendedto support patient-centered care, the purpose of which wasunclear to them. For example, a question about the patient'spet, intended to identify a patient motivation for getting better,was a puzzle to nurses: “Like the pets thing…Have you hurtyour pets? Are your pets safe? What am I supposed to do, callthe SPCA?”
8. Team Communication is related to the quality and usefulness ofNIS related to teamwork. Nurses described how using the NISimproved team communication and relations in variety ofways.First, it was easier to find notes from other nurses and physicaltherapists. Also, nurses could more easily follow what otherdisciplines did. As one nurse noted, “So I think that's gottenbetter, I thinkwe can followmuch,much easier what's going onfrom the other disciplines cause they are in there. They areputting little things into our assessment, like the notes sectionsthat's going across, theywould say they saw our patient, andwecan see it really easy that they saw our patient, whereas before,we'd have to go back in, go into the chart, to see oh, yeah, theycame and they did this,…(before)…they had their own section,and half the time I couldn't even find their section, but now Ithink it's a lot easier.” A third point was that medical aides, bydocumenting in theNIS,were included in teamcommunication:“[NIS] allows the aides to get involved. And they feel like theyare more accountable because they have to enter documenta-tion into the computer system: vitals and things like that.” Afourth point was that the use of the NIS made the physician'swork easier which in turn made the nurses' work easier.However, nurses reported continued dependence on verbalcommunication. The NISwas not relied upon to relay importantevents (e.g., at shift change or to report to physicians): “There'salso that physician communication note, but you're supposed tocall the doc before you put that note in, so it's just documentingthat you called the doc.” Also, nurses found it easier to talk to aphysical therapist when the person was nearby rather thanlooking for a note: “What if I need to communicate with them[physical therapy]? Well, not in the computer, I probably haveto call 'em… I wouldn't rely on the computer.”In addition, respondents provided examples of theNIS not beingused sufficiently for team communication. One nurse reportedthat patients complained they were being asked the samequestions repeatedly: “Sometimes, I'll go into a room and I'll askthe person questions, and they'll say, I've already been askedthat, a couple times.” Also, nurses expressed uncertainty anddoubt about whether other clinical disciplines looked at thenurse's documentation for information. However, validationwith a discharge planner revealed that discharge planners referdaily to the patient information (e.g., vital signs and clinicalsummary). Nurses also reported that physicians were notaccessing the NIS, causing nurses to double document to enablephysicians to more readily see the nursing documentation. Asone nurse described the situation: “…Everything was alreadydone throughout the day, so that's how we were trained. Andthen our doctors, our attendings, came in and said, ‘where arethe notes?’ …We request that our nurses still do a detailed notebecause we are using them.” A challenge to clinicians using theNIS for team communication was the existence of separatesystems specific to various disciplines. For example, whiledischarge planners reported they retrieved data from the NIS,they documented in another system that did not interface withthe NIS.
30 P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
9. Patient Rapport focused on NIS related to patient communica-tions. A nurse reported that he or she did not like documentingin the NIS in the patient room because they perceived it wasimpersonal to have your back toward the patient (asmentioned above in Hardware) or to not give the patientyour full attention. As the nurse described his/her perception:“I believe I should be giving my patient undivided attention,exactly, and I want to look at the patient when I'm askingquestions. I want to see that facial [expression]—does he reallyhave pain or is he just fluffing it off likes it's nothing? So I reallybelieve in watching the patient when I'm assessing him, takingtime. And if I was doing it in the computer, then I don't get that.And I feel as though the patient feels that we are too busy doingother things, than giving him the attention.”
10. Training was focused on usefulness and NIS implementation.Nurses described three types of training related issues. First, allthe disciplines were supposed to participate in training.However, if the training is not completed by all NIS users, it isfrustrating for those who did complete the training. One nursedescribed the nurses showing a perturbed physician how toaccess the flow sheets: “This is one of our big attendings, and hewas flipping out. We sat down and showed him the trendingsheets, and he got a little bit more comfortable with that….”Second, nurses reported that NIS changes were introducedduring upgrades without sufficient communication or follow-up training. One nurse informed the researchers that he/shewas out for several weeks, but there was no update oreducation to let him or her know about changes since the lastlogin. Another nurse explained, “It just seems like every otherweek they come out with a new format for [NIS]. Like, first itwas to do the care provider notes. Then it was, change it, do itthis way, do this. And sometimes they just added certain things.And you're like, where did that even come from? So, it justneeds to be, if you are going to roll something out, at least let usknow what it is and tell us what we are supposed to be doing.”In addition, the researchers observed and nurses reported aninconsistency in standards across units that should have beenaddressed with training. The team observed variation in howmany or what types of guidelines were required. One nursestated, “People need to be on the same page as to how we'redocumenting each of our systems, what we are putting into[NIS], and what certain things mean, like our falls preventionprogram, what does it mean to be multi pharmacy or you knowwhat I mean? They just weren't understanding some of thelingo in the [NIS], so to speak.” Nurses also reported trainingissues involving lack of standards for documentation anddifferent interpretations of NIS terms. As one nurse observed,“There has to be a continuous ongoing training.”
11. Patient Outcomewas related to perceptions about NIS impact onthe outcome of the process of providing care. As mentionedabove, one nurse perceived the NIS had a positive impact onpatient outcomes, because clinicians across disciplines used thesystem. This nurse also felt the NIS had a positive impact onpatient safety because clinical roles, such asmedical aides, werenow documenting in the system.
12. Perceived NIS Value related to overall impact of the NISimplementation. Nurses were not aware of the value of theNIS. As one nurse stated, “[documenting in the NIS is] just onemore thing to do.” They viewed the NIS as a documentationsystem. For example, most respondents perceived that otherclinical disciplines did not view the NIS nursing documenta-tion, although a few nurses thought that physicians viewedit. Respondents identified one opportunity where the NIScould provide value: improve presentation of data by syn-thesizing clinical information into a summary, such as an endof shift summary.
3. Discussion
This is the first evaluation of a hospital point-of-care documenta-tion system designed for the interdisciplinary care team. The nurseleadership of the health system requested feedback on the studyresults. Therefore, of special interest were themes that emergedrelated to the institution's goals for the evaluation, standardizing careand supporting patient-centered care. Regarding the methods, thescenario-based user testing process was enriched by the participationof research teammembers who were knowledgeable, current users ofthe NIS to aid in question formation and interviewing.
The study yielded themes related to the health system's evaluationgoals including: (i) NIS impact on reducing variability in patient caretogether with nurse use of NIS clinical educational resources, and (ii)NIS effect on team communication. Nurses tended to use the NIS at thepoint-of-care (i.e., in the patient room) for someof their documentationtasks. Nurses perceived NIS as a task-oriented documentation recordwith limited impact on clinical decision making and quality of patientcare. Infrequently nurses mentioned the benefits of the availability ofeducational material or guideline memory prompts in clinical decisionmaking. Some nurses perceived that use of point-of-care systemsincreased the time spent documenting, thereby decreasing the amountof time for other patient care-related activities. Although not a directcomparison, DesRoches' recent survey of nurses and EHRs (whichweredescribed as at least minimally functional and may not have explicitlyincluded interdisciplinary care documentation) found that EHR use didnot change nurse time allocation during their workday (DesRoches,Donelan, Buerhaus, Potter, & Zhonge, 2008).
The NIS implementation was intended to reduce team members'repetitive questioning of patients by improving team communication.However, nurses perceived that the NIS was not used to support crossdisciplinary communication because other disciplines did not use theNIS. Furthermore, among nurses, as seen in studies of physicians(Coumou & Meijman, 2006), it was easier to ask the clinician at yourelbow than to look back through the documentation in the computer.
The assessment was intended to support patient-centered care bycapturing information pertinent to the patient's motivation to getbetter. However, nurses did not perceive the value of thesequestions, and instead found them puzzling. Other than thesequestions, it was not apparent whether or how the NIS supportedpatient-centered care. As Avgar, Litwin, and Pronovost (2012) note,patient-centered care requires fundamental reorganization of work;successful HIT implementations leverage the HIT as a tool toreinforce these work changes.
Themes related to challenges and facilitators to adoption alsoemerged, which should be of particular interest to nurse decisionmakers. All nurse participants expressed that they did not want to goback to a paper medical record. Nurses said that it was easier to findpatient information in the NIS compared to searching the patient'spaper charts. They conveyed that, even though the system hadnavigation and usability issues that reduced their efficiency, havingthe majority of the patients' information in one place, accessible fromany computer, was convenient. However, nurses also expresseddissatisfaction with NIS impact on workflow. Nurses experiencedusability issues that impeded their ability to document in a systematicway. For example, the location of the detail to be documented washidden beneath unexpanded headers forcing the clinician to searchfurther. A concern was that a less experienced clinician may forget togo back and document the detail. Instead of saving time as intended,navigation patterns that skipped important sections in the documen-tation added more time to the process and were confusing to thenurse. In addition, nurses experienced unintended consequences thatdisrupted workflow, benefited other groups (e.g., physicians), and didnot decrease their cognitive load. For example, the absence offunctionality that synthesized and summarized information (Collins,Bakken, Vawdrey, Coiera, & Currie, 2011) caused nurses to double
31P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
document a summary in a location where a physician could readilyview the information.
In addition to usability issues, a challenge to adoption was thatnurses perceived their use of NIS as a task to be completed withoutrecognizing value in the process. Nurses did not know that otherdisciplines (e.g., discharge planner) looked at their documentationand did not perceive that documenting in the system provided valueto the clinicians on the team, raising the nurses' visibility. In addition,the value of the data collected was not demonstrated to the cliniciansdue to the lack of feedback to clinicians providing direct patient care(e.g., quality indicators). As a result, nurses were unaware thatdocumenting via free text as compared to structured text negativelyimpacts the availability of data for operational improvements andresearch (Bowles et al., in review).
Significant challenges to adoption were problems with ongoingtraining and support. Nurses described the need for continuousongoing training due to upgrades as well as prolonged staff absences.In addition, there was inconsistency in NIS use among the units andthe nurses, underscoring insufficient communication and trainingrelated to NIS functionality.
3.1. Limitations
During the interview process, the data collected were based solelyon responses to the list of questions and scenarios. If anonymity canbe assured, other factors that should be considered include nurses'age, length of employment within the hospital, and familiarity andfrequency of computer use. These factors have been shown toinfluence nurses' perceptions of usability, support, and ease of usewith NIS (Seckman, Romano, Mills, Friedmann, & Johantgen, 2009). Inaddition to lack of participant demographic data, another limitationwas that scenario testing produced descriptive results which may notbe generalizable beyond early stage NIS implementations or academicmedical centers. Also, study findings are not able to be generalizedbecause the results are applicable to the participants and not to thepopulation as a whole.
4. Recommendations
Our recommendations to nurse leaders have included respondingto opportunities to improve communication with front-line clinicians.Fundamentally, nurses need to understand the goals of the systemand hear feedback about why the NIS is or should be providing value.For example, replace the units' hand written posters on a bulletinboard in a conference room illustrating trends of selectedmetrics witha display on the computer (e.g., a splash screen) showing trendsgenerated from data captured in the NIS. We acknowledge theorganizational challenges in using HIT strategically and analytically,yet doing so can help transform care delivery instead of merelyautomating documentation (Avgar et al., 2012).
Operationally, there is an opportunity to improve how the NIS isevaluated, that is, to promote constructive feedback from front lineclinicians that is provided to those who can make decisions regardingsystem design and implementation. Novak, Shilo, Gadd, and Lorenz(2012) described clinicians who served as mediators of adoption byconducting on-going organizationally-sanctioned “work at the nexusof the institutional setting, the clinical users of information systems,and the IT infrastructure, including developers.” (p. 1043). Among theresponsibilities ascribed to mediators are the following post-imple-mentation activities of: (i) problem resolution for new systems beingimplemented, and (ii) ongoing support including identifying issuesthat necessitate process and policy changes (Novak et al., 2012). Anexample is the continued reliance on verbal communication becausethe NIS cannot be relied upon to relay important events. This processof mediation of adoption and use is a key strategy for mitigatingunintended consequences of HIT implementation.
Further communication opportunities include flowing informationfrom NIS to front-line clinicians to improve clinical process related topatient care and patient safety. An example is regular emailsinforming nurses of issues that have been fixed in response to theirrequests. Closing the communication loop would demonstrate thatnurses' concerns were being addressed and that the NIS was beingimproved for their benefit. The intended result is that front line usersmay be more accepting of the system and more willing tocommunicate their questions and issues. Another recommendationis to keep training up to date to create and sustain clinicians'enthusiasm for the potential of the NIS.
In addition, acknowledging that the NIS was a relatively newimplementation, there is an opportunity to improve important NISfunctionality. Valuable patient information is collected in the NIS thatcould trigger the initiation of consults or referrals, for example, forhigh risk patients (Bowles, Hanlon, Holland, Potashnik, & Topaz, Inpress). Also, while the outcome summary enables nurses to reviewtheir day and possibly identify important events that weremissed, theoutcome summary's functionality could be enhanced. The NIS shouldbe able to perform a lower-level synthesis for the clinician, such aspull together what was done for the patient in one place where thesummary could be edited by the clinician. If decision makers do notwant clinicians relying on verbal communication, then outputsummaries should also have a place to capture and flag informationcurrently communicated via verbal report, such as significant events.
The results of this study would help front-line clinicians'application to practice in a number of ways. First, this study informsthese clinicians about the challenges and facilitators to NIS adoptionidentified at the study site. Clinicians can use this information toinform decisions if they are involved in the selection or implemen-tation of an NIS. Second, clinicians can use the study findings post-implementation to inform their communication with NIS implemen-tation decision makers, if a mechanism for feedback from front-lineclinicians is available and functional. Using such a communicationmechanism, clinicians could provide feedback on topics such as NISimpact onworkflow, as well as suggestions to improve presentation ofNIS information. In addition, our methods might be useful to others.Using scenarios to gain insights into how the system is used provedvaluable and rich for the researchers.
5. Conclusions
Opportunities for hospital leadership to improve NIS adoptioninclude enhancing two-way communication with front-line cliniciansabout: (1) the hospitals' goals; (2) perceived NIS value at the bedside;and (3) constructive feedback about design (e.g., functionality) andimplementation (e.g., training) especially about patient care/safety.During NIS selection, design, and implementation, nurse leadersshould also focus on the other highly problematic areas identified inthis study: software usability, implementation, and training. Areas notidentified as problematic warranted less investment while beingcarefully monitored. Clinicians were relatively satisfied with hard-ware placement, completeness of documentation, clinician efficiency,and appropriateness of patient care. Clinicians were neither satisfiednor dissatisfied with areas that included software functionality andteam communication. Addressing these opportunities may improvethe value and usefulness of the system to clinicians, hopefullyrealizing the implementation goals of supporting collaboration andimproving patient care outcomes.
Acknowledgments
We thank Barbara Granger for her editorial assistance and GioiaChilton for transcribing the recorded scenario testing sessions. Wealso thank the clinicians who participated in the study and the health
32 P.S. Sockolow et al. / Applied Nursing Research 27 (2014) 25–32
system nursing leadership who enabled the team to conduct thisresearch.
References
Ammenwerth, E., Rauchegger, F., Ehlers, F., Hirsch, B., & Schaubmayr, C. (2011). Effect ofa nursing information system on the quality of information processing in nursing:An evaluation study using the HIS-monitor instrument. International Journal ofMedical Informatics, 80, 25–38.
Avgar, A. C., Litwin, A. S., & Pronovost, P. J. (2012). Drivers and barriers in health ITadoption: A proposed framework. Applied Clinical Informatics, 3, 488–500.
Bowles K. H., Hanlon A. L., Holland D. E., Potashnik S., & Topaz, M. (in press). Impact ofdischarge planning decision support on time to readmission among older adultmedical patients. Professional Case Management, 19(1), 1–10.
Bowles K. H., Potashnik S. L., Rosenberg M. L., Ratcliffe S. J., Holmes J. H., Shih N., et al. (InReview). The realities of conducting research using the electronic health record(EHR) across four hospital systems: Semantic harmonization, implications foradministrators. Journal of Nursing Administration 43(6), 355–360.
Collins, S. A., Bakken, S., Vawdrey, D. K., Coiera, E., & Currie, L. (2011). Modeldevelopment for EHR interdisciplinary information exchange of ICU common goals.International Journal of Medical Informatics, 80(8), e141–e149, http://dx.doi.org/10.1016/j.ijmedinfo.2010.09.009.
Coumou, H. C. H., &Meijman, F. J. (2006). How do primary care physicians seek answers toclinical questions?A literature review. JournalMedical Library Association,94(1), 55–60.
DesRoches, C., Donelan, K., Buerhaus, P., Potter, V., & Zhonge, L. (2008). Registerednurses' use of electronic health records: Findings from a national survey. TheMedscape Journal of Medicine, 10(7), 164–173.
Kutney-Lee, A. (2011). The effect of hospital electronic health record adoption onnurse-assessed quality of care and patient safety. Journal of Nursing Administration,41(11).
Nielsen, J. (1993). Usability engineering. San Diego, CA: Academic Press.Novak, L. L., Shilo, A., Gadd, C. S., & Lorenz, N. M. (2012). Mediation of adoption and use:
A key strategy for mitigating unintended consequences of health IT implementa-tion. Journal of the American Medical Informatics Association, 19(6), 1043–1049,http://dx.doi.org/10.1136/amiajnl-2011-000575.
Rogers, M., Sockolow, P. S., Bowles, K. H., Hand, K. E., & George, J. (in review). Usability:A window into understanding nursing documentation in an NIS. InternationalJournal of Medical Informatics, [Sept 2 2013 online publication]. http://dx.doi.org/10.1016/j.ijmedinf.2013.08.007.
Sassen, E. J. (2009). Love, hate, or indifference. how nurses really feel about theelectronic health record system. Computers, Informatics, Nursing, 27(5), 281–287.
Seckman, C., Romano, C., Mills, M., Friedmann, E., & Johantgen, M. (2009). Clinicians'perceptions of usability of an electronic medical record (EMR) over time.Computers, Informatics, Nursing: CIN, 27(5), 334–335.
Sidebottom, A. C., Collins, B., Winden, T. J., Knutson, A., & Britt, H. (2012). Reactions ofnurses to the use of electronic health record alert features in an inpatient setting.Computers, Informatics, Nursing: CIN, 30(4), 218–226.
Sockolow, P. S., Lehmann, H. P., Bowles, K. H., & Weiner, J. P. (2009). Are nurses happierafter an electronic health record is implemented at a nurse-managed practice forthe elderly? Computers, Informatics, Nursing, 27(5), 330–331.
Stevenson, J. E., Nilsson, G. C., Petersson, G. I., & Johansson, P. E. (2010). Nurses'experience of using electronic patient records in everyday practice in acute/-inpatient ward settings: A literature review. Health Informatics Journal, 16(1),63–72, http://dx.doi.org/10.1177/1460458209345901.
Urquhart, C., Currell, R., Grant Maria, J., & Hardiker Nicholas, R. (2009). Nursing recordsystems: Effects on nursing practice and healthcare outcomes. : John Wiley & Sons,Ltd, http://dx.doi.org/10.1002/14651858.CD002099.pub2.
