Post on 27-Sep-2020
transcript
Team Healthcare
1 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Preserving Patient Safety after the Implementation of an Electronic
Health Record
Valerie Alley, Geoff Rhode, Elizabeth Martin, Angie Wolthuis
OPWL 529 Needs Assessment
Organizational Performance and Workplace Learning
Boise State University
Fall, 2013
Team Healthcare
2 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Contents Contents ........................................................................................................................................................ 2
Executive Summary ................................................................................................................................... 4
Project Overview ................................................................................................................................... 4
Goals ..................................................................................................................................................... 4
Data Collection ...................................................................................................................................... 4
Findings ................................................................................................................................................. 4
Recommendations ................................................................................................................................ 5
Alternate Recommendation.................................................................................................................. 5
Introduction .............................................................................................................................................. 6
Client Organization ................................................................................................................................... 6
Primary Contact in the Organization .................................................................................................... 7
Team Members and their Roles ............................................................................................................ 7
Purpose of Needs Assessment Project ..................................................................................................... 8
Key Changes to the Orders Management Process during a Transfer ................................................... 8
“Exemplary” Orders Management ........................................................................................................ 9
Client Concerns and Observations ........................................................................................................ 9
Importance of Maintaining Accurate Electronic Records ..................................................................... 9
Initial Client Plan to Close Performance Gap ...................................................................................... 10
Worth of Solving the Problem ............................................................................................................ 10
Identifying the “Need” ............................................................................................................................ 11
Analyzing the Cause of the Gap .............................................................................................................. 14
Intervention Selection ............................................................................................................................. 19
Methods in Intervention Selection ..................................................................................................... 19
Organizational Constraints in Selecting Interventions ........................................................................ 20
Primary Interventions Presented ........................................................................................................ 21
Priority of selected interventions ....................................................................................................... 23
Alternative Interventions Presented................................................................................................... 23
Future Recommendations Based on Findings .................................................................................... 24
Intervention Alignment with Organizational Goals ............................................................................ 25
Conclusion ............................................................................................................................................... 26
References .............................................................................................................................................. 27
Appendix A. Mission, Vision, Values of Holy Healthcare ....................................................................... 29
Appendix B. Models ................................................................................................................................ 30
Appendix C. Data Collection.................................................................................................................... 31
Nursing Director Interviews ................................................................................................................ 31
Team Healthcare
3 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Emergency Room Nurse Interviews .................................................................................................... 38
Clinical Informatics Department Trainer Interviews ........................................................................... 44
Clinical Informatics Department Liaison Interviews ........................................................................... 50
Observation Checklist and Data .......................................................................................................... 55
Informal Interview of Nurses Results .................................................................................................. 64
Results of Informal Poll with Nursing Directors and Clinical Educator ............................................... 65
ERIH Intervention Interviews .............................................................................................................. 66
Appendix D. Data Synthesis ................................................................................................................... 68
BEM Data Synthesis ............................................................................................................................ 68
Appendix E. Supporting Documents ....................................................................................................... 79
Training Agenda .................................................................................................................................. 79
Transfer Process Job Aid ..................................................................................................................... 87
Meeting OPWL Program’s Learning Outcomes .................................................................................. 89
Team Healthcare
4 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Executive Summary
Project Overview
The implementation of the Electronic Health Record (EHR) in May of 2013 changed both clinician
workflow and patient care processes at Holy Medical Center (pseudonym). The Holy Medical Center
Clinical Education Department initiated this needs assessment as a result of observed inconsistencies in
the patient transfer process, particularly when the Emergency Department transfers a patient to an
inpatient unit.
Originally, the team intended to analyze the entire transfer process. However, through observations
and feedback from data sources, the team noted improvements to the verbal handoff process following
the Clinical Skills Fair held in mid-October, at which time the Clinical Education department addressed
verbal handoffs, including the importance of chart review during the verbal handoff. Based on the
noted improvements, the team narrowed the scope of the project to focus on orders management
during transfers (admits) from the Emergency Room (ER) to inpatient units. The team specifically
examined the ER nurse’s performance related to orders management during a patient transfer from the
ER to an inpatient unit.
Goals
The goals of this needs assessment were as follows:
1. Assess opportunities for nurse performance improvement, related to orders management,
during the transfer process.
2. Develop recommendations to improve nurse performance related to orders management
during the transfer process.
Data Collection
The assessment team collected data from various sources that both directly and indirectly affect orders
management during the transfer process. The data sources were as follows:
Interviews with nurses, Nursing Directors, a Clinical Educator, Clinical Informatics Specialists,
and Clinical Informatics Liaisons
Review of related training and process documents
Observations performed as nurses completed the transfer process from the ED to inpatient units
The team used the interview data from the aforementioned sources, as well as a review of supporting
documents, to assess both individual and organizational performance.
Findings
Through the Needs Assessment process, the team found the following factors contributing to nursing
performance issues related to the orders management process during transfer:
Nurses are receiving incomplete and/or conflicting information regarding the nurse’s role in the
orders management process during transfer
The nurses’ lack the information regarding the tools available to support them on this process
and the Electronic Health Record in general
Team Healthcare
5 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Nurses do not receive adequate feedback regarding their performance and therefore they may
not realize there is a performance issue
Nurses experience a generalized lack of motivation or buy-in to the process because of lack of
information provided to them regarding their role in the process.
Recommendations
The team makes the following recommendations, which we believe, if executed as recommended, will
help fill the performance gap:
Provide clear and accurate information regarding task responsibility and expectations: In the
nursing huddles, the charge nurses and or the Unit Director should clarify to the nurses, their
role in orders management during the transfer process. The Clinical Education department will
build a short job aid to reiterate the nurse’s role as well as guide them in how to retrieve orders
discontinued or archived.
Inform nurses of the support available from the Clinical Informatics Department: Post in the
ER and throughout the hospital, and communicate in daily huddles information on:
how to contact the Clinical Informatics Department support line; and
how the support line can be of service and provide aid.
Provide frequent feedback that is accurate, timely, and specific: Director and/or charge nurses
should complete chart audits and give both positive and constructive feedback to nurses on
their performance related to the completion of orders management. Nurses should also give
peer-to-peer feedback regarding orders management completion, which will be timelier than
Director or charge nurse feedback.
Redesign and relocate support aids: The Clinical Informatics Department and the Clinical
Education Department should collaboratively assess which job aids are necessary and relevant
to nurses, redesign them to be user-friendly, and store them in a location that is easily
accessible and user-friendly.
Alternate Recommendation
Through the assessment process, the team learned that the hospital had the option to add the
Emergency Room Inpatient Holding (ERIH) area to the EHR but elected not to at the time of
implementation. The electronic system places a patient in the holding area, in the electronic record,
while the patient waits for the ER to transfer them to an inpatient unit. Once they leave the holding
area, all of their ER orders are no longer visible on the record and the attending inpatient physician
places new orders for the patient. Another hospital in the local system uses the ERIH, and has
reported no issues with the orders management process during patient transfers from the ER to an
inpatient unit. The team recommends that the hospital investigate the feasibility of this option,
which could potentially eliminate the related performance issues.
Team Healthcare
6 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Introduction In September 2013, as part of the Boise State University Organizational Performance and Workplace
Learning (OPWL) program, Team Healthcare formed with the intent of completing a Needs Assessment
(NA) for Holy Medical Center. Throughout the semester, the team completed a formal needs
assessment.
Client Organization Holy Medical Center started as a small community hospital in 1917 by the Sisters of Mercy —bringing
healthcare to the poor and underserved. Holy Medical Center is a not-for-profit, 152-bed acute care
hospital serving the medical needs of the community. In 2010, the healthcare system HC Regional
Medical Center purchased Holy Medical Center making a part of one of the nation’s largest Catholic
healthcare systems.
The associates of Holy Healthcare and its hospitals and medical facilities refer to its Mission, Vision, and Values to guide them as they make decisions for their organizations (see Appendix A for the organizational goals in their entirety). Mission
We serve together in Holy Health, in the spirit of the Gospel, to heal body, mind and spirit,
to improve the health of our communities
and to steward the resources entrusted to us.
Holy Medical Center
152 Bed
Hospital
2 Emergency Departments
HC Health System
4 Hospitals
Multiple medical clinics in 2 states
Core Values
Respect Social Justice Compassion Care of the Poor and
Underserved Excellence
Vision
Inspired by our Catholic faith tradition, Holy Healthcare will be distinguished by an unrelenting focus
on clinical and service outcomes as we seek to create excellence in the care experience. Holy
Healthcare will become the most trusted health partner for life.
Holy Healthcare
82 Hospitals and numerous medical clinics in 21 states
Team Healthcare
7 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Primary Contact in the Organization
Anne Young, a clinical educator at Holy Medical Center, was the primary contact and client within the
client organization. As a clinical educator, she provides and coordinates educational opportunities for
the clinical staff including, nurses, Certified Nursing Assistants (CNAs), laboratory technicians,
respiratory therapists, and radiologic technologists. As the primary clinical educator in a small hospital,
Anne is also responsible for offering recommendations for improving nursing performance.
Team Members and their Roles
The table below summarizes the project team. Outlined in the table are the roles and responsibilities
each team member assumed throughout the duration of the project.
Table1 Project Team Members
Name and Job
Title
Department Project Role Project Responsibilities
Anne Young, MSN
Clinical Educator
Clinical
Education
Client
Project SME
Liaison between Team Healthcare and Holy Medical Center
Coordinated efforts to collect data; conducted observations
Participated in decision-making on intervention selection
Clinical Informatics Liaisons
Clinical Informatics Department
SMEs on workflow process
Provided data through interviews and chart audits
Nursing Directors Nursing Director ICU/Medical Unit
Interview Respondents
Provided data through interviews
Clinical Informatics Specialists
Clinical Informatics Department
Interview Respondents SMEs on the EHR
Provided information regarding the training on the EHR
Provided training documents
Hospital Nurses Varied Interview Respondents
Provided data through interviews
Valerie Alley
Boise State
University (BSU)
OPWL
Member of Team Healthcare
Acted as the team interview question expert by creating interview questions and observation documents
Assisted with data coding Provided connections between data and possible
solutions by completing the data synthesis
Elizabeth Martin
BSU OPWL Member of Team Healthcare
Created the data coding system Primarily responsible for coding data Researched and guided intervention selection
Geoff Rohde
BSU OPWL Member of Team Healthcare
Aided the team in deconstructing overly complex problems
Researched and provided connections between the NA and healthcare
Angie Wolthuis
BSU OPWL Member of Team Healthcare
Main point of contact between the team and the client
Coordinated efforts to keep the project on track
Acted as the primary data collector
Team Healthcare
8 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Purpose of Needs Assessment Project This project was initiated by the clinical educator at Holy Medical Center because of her commitment in
maintaining the organization’s vision of distinguishing Holy Health Care by having “an unrelenting focus
on clinical and service outcomes as they seek to create excellence in the care experience” (Corporate
Headquarters - anonymized, 2013, para. 7.)
Holy Medical Center converted from a paper orders management system to an online Electronic Health
Record system (EHR) in May 2013. Since the implementation, the Clinical Educator has been concerned
with how nurses are adapting to the new orders management process required by the new system.
Key Changes to the Orders Management Process during a Transfer
Table 2 Former and Current Orders Management Process
Paper Process (up to April 2013)
Electronic Process (May 2013 to present)
With the paper process, when a patient transferred to
a new unit, the nurse in the receiving unit would turn
the page in the patient’s chart and start a new page of
orders. Neither the sending nurse nor the receiving
nurse was required to take action on the “old orders.”
Old orders (orders from a previous unit) were kept in
the patient’s chart and could be referred to easily by
the doctor or nurse, or any other clinical provider, by
simply turning back to the paper orders in the chart.
However, there was only one chart, which made it
impossible for multiple care providers to review the
patient’s information at one time
The new electronic system requires that the nurse from
the sending unit, either discontinue or complete the
orders. By discontinuing an order, the nurse is telling the
computer system that the order should no longer be
active by deleting it from the orders screen. The system
archives the order and marks it as “Discontinued.” By
completing an order, the nurse is telling the computer
system that the order is “complete”; that the required
action has been taken. The system also archives and
marks these orders as complete. Archived orders
disappear from the current screen, but can be found and
retrieved by users and are permanently a part of the
patient record.
Team Healthcare
9 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
“Exemplary” Orders Management
According to our client, an exemplary performer would discontinue and/or complete all Emergency
Room (ER) orders in the EHR that must not continue with the patient when the patient transfers to the
new unit. Exemplary charts would have only current and active orders on the orders screen. The
receiving unit could rely on the information in the chart and carry out the orders based on the
information on the orders screen.
Client Concerns and Observations
In August 2013, an incident occurred in the hospital and a near miss put a patient’s safety at risk. A
nurse did not discontinue an ER medication order and the order continued with the patient into the
inpatient unit, where a medication, now unsafe for the patient, was almost administered. Fortunately,
an alert nurse spotted the error and contacted the physician for clarification, where upon, the order was
discontinued in the system.
While a fatality did not occur, the client chose to investigate further in an attempt to determine where
in the process there was a breakdown. The client discovered that there was a miscommunication during
the transfer process that led to this error. She spoke with both nursing Unit Directors and nurses and
discovered that there was a confusion surrounding the transfer process, including the orders
management piece. At this point, she engaged our team to look at the performance issues related to
the transfer process.
As per the client, in order for the transfer process to be effective the nurses must complete a thorough
verbal handoff. She suggested that if the nurses referred to the patient’s electronic chart during the
verbal handoff, they would not have to go from memory and would avoid not relaying details critical to
patient care and safety.
At the outset, the team was investigating the entire transfer process, including the verbal handoff.
However, in October, the Clinical Educator used a nursing skills fair as a forum to communicate the
importance of opening the chart during a transfer in order to review patient information. Observations
completed by the Clinical Education team following the skills fair, indicated that the nurses were
opening the charts during their verbal handoffs. At that time, the team, in collaboration with the client,
determined this was no longer an issue worthy of investigation and instead decided to focus on the
orders management piece of the transfer process.
Importance of Maintaining Accurate Electronic Records
Because patients transfer successfully from one unit to another every day, Team Healthcare asked the
question: Why was investigating this performance gap important to the hospital? Using the Lean
principle of the “5 Whys” to interview the client, allowed the team to identify the value and importance
of investigating this need.
Why is it important to maintain accurate electronic records?
This decreases the chances of patient mortality and morbidity due to medical errors.
Team Healthcare
10 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Why is it important for the hospital to avoid deaths due to medical errors?
It is the mission and vision of the Holy Medical to do no harm to patients and to seek to
create excellence in the care experience. The hospital wants to do no harm as both a
way to fulfill our mission of keeping patients safe, but also as a way to avoid costly
litigation.
What could happen if the hospital does not create an excellent care experience and harms
patients?
The Joint Commission could identify this as a violation and the Joint Commission could
put the hospital on probation or the hospital could lose its accreditation. The hospital
could also get involved in expensive litigation, which costs money, and takes resources
from our mission of caring for the poor and underserved.
Why does Holy Medical care if they lose their accreditation?
Because the members of the community would not have a full-service healthcare facility
in their immediate area and they would have to travel outside of their community for
full-service healthcare.
Why do community members need health facilities in their community?
Without emergency services in the community, sick or injured community members
might choose to delay or avoid health case, or in some instances die on the way to
finding another facility to access.
Asking these questions helped the team better understand why maintaining an accurate electronic
record is important for both the hospital and the patients they serve.
Initial Client Plan to Close Performance Gap
Originally, the client suggested that additional nursing training could address the performance gap.
With advanced EHR system training already scheduled, she felt this was a simple way to solve the issue.
However, as Dean (1993), and Mager and Pipe (1988) pointed out, worker lack of skill is not a common
cause of performance problems and rarely the only cause of performance problems. For this reason,
Team Healthcare helped the client understand that a skill deficiency is only one possible cause of the
problem and that an in-depth needs assessment would likely uncover additional causes of this problem
as well as provide evidence for such findings (Watkins, Meiers, & Visser, 2012). By pushing back on a
“training first” approach, the team was able to help the client understand there are many other causes
of, and solutions for performance gaps (ISPI, 2002).
Worth of Solving the Problem
Team Healthcare began its performance assessment by asking a very important question “Is this
problem worth solving?” Chyung (2008) suggests, “Performance becomes worthy when the value of an
Team Healthcare
11 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
accomplishment exceeds the costs needed for changing behavior in order to produce the valuable
accomplishment” (p. 109).
Because Team Healthcare is committed to adding value to the client’s organization, the team began the
needs assessment process, by asking, “What would be the cost of not doing anything about this
performance problem?” Team Healthcare recognized that if Holy Medical Center was not maintaining
accurate, timely medical records, patient safety – and maybe life itself - could be compromised. Team
Healthcare then looked closely at the costs associated with death due to hospital negligence for both
the hospital and for the survivors of the victim.
Cost of time and money associated with malpractice or wrongful death lawsuits could
have a very negative impact on the hospital. The Society of Actuaries published a study which
noted that while the average cost of inpatient medical error is about $22,000, the cost varies
widely by age of patient, type of harm (minor, major or death). The most costly errors range
from an average of over $40,000 to well over $600,000 (Shreve et. al., 2010).
Cost of mental suffering a family feels when someone passes due to the negligence of
others is immeasurable. Anger, frustration, and pain can last for years and have an effect on
both families and friends. According to Hospice Foundations of America (2013), anger,
frustration, pain can last for years and have an effect on numerous families and friends, thus
lowering quality of life for survivors.
Because of the cost of not doing anything about this performance problem was so great, Team
Healthcare identified this as a worthy project. The team believed that the cost of any recommended
intervention would be less than a potential lawsuit and/or the pain and suffering inflicted on survivors of
the victim.
Identifying the “Need” Watkins et al. (2012) note that the first step in a needs assessment is to identify the gap between
desired and current results. Below are the methodical steps Team Healthcare used in determining the
client’s needs.
Step #1 Established relationships and boundaries with client
In support of the ISPI code of ethics (2002) Team Healthcare set clear expectations with the client about
the systematic process we planned to follow, before proceeding with the needs assessment. To build
trust with the client, and to conduct our assessment within the bounds of the ethical standards set forth
by ISPI, the team did the following:
Obtained official written permission from our client to conduct our study
Agreed to keep conceal the identity of the organization and interviewees
Agreed to keep the client informed of key findings before moving to each process
Established a professional relationship with all parties involved when collecting data
Team Healthcare
12 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Step #2 Used a systematic approach in identifying the need
Team Healthcare began the investigation of identifying the performance need by using Harless’s (1987)
13 questions as a way to determine if there was a performance problem. It was the team’s desire, as
suggested by Harless, not to “waste” (p. 7) the hospital’s money investigating things that are not
problems. To begin our investigation, the team set out to answer Harless’s first three questions:
1. Do we have a problem?
2. Do we have a performance problem?
3. How will we know if the problem is solved?
Step #3 Collected data to establish the need
To investigate the existence of a gap in performance, Team Healthcare purposely looked at various
sources of information to triangulate our findings. According to Miles and Huberman (1984)
“Triangulation increases our confidence about a finding by helping us determine if other sources echo it”
(as cited in Rossett, 2009, p. 3).
The team used several data sources, as well as several methodologies by which to collect and triangulate
data to verify that there was a performance gap. Table 3 outlines the data the methodologies and data
sources the team used to assess the gap and verify the need for a complete NA. The impact of the data
is provided following table 3.
Table 3 Data Collection Methods
Data Collection Methods
5 Individual Interviews
Physician Liaison- Clinical Informatics Department
Orderset Coordinator- Clinical Informatics Department
Clinical Liaison- Clinical Informatics Department
Clinical Educator- Clinical Education Department
Charge Nurse- Emergency Room
Audit of 14 files Due to concerns of violating patient confidentiality, Team Healthcare was unable to review charts. Hospital personnel audited the charts.
Step #4 Analyzed data and summarize key findings
Evidence from both patient chart audits and interviews with key personnel, prove there is a gap
between the required and current state of discontinuing the electronic orders.
As outlined in table 3, of the 14 charts audited, the auditors classified approximately 70% as not up to
hospital standards. An interview with the Clinical Educator indicated that one nearly fatal mistake
occurred, due to improper orders management, but fortunately, competent staff diverted a possible
fatality. Furthermore, in an informal interview with Medical Unit and Intensive Care Unit (ICU) nurses,
the team asked nurses to rate, on a scale of 1 to 5 (1 being not well at all and 5 being really well), how
well the orders are cleaned up in a chart when they receive a transfer patient. The nurses answered
Team Healthcare
13 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
with an average of 2.17, which further points to an issue with orders management during transfer. The
client indicated that average of 4 would be expected and 5 would be optimal.
Table 4 Overview of Collected Data
Data Collection
Method
Reviewed by or
Interviewee
Data
Audit 1 Physician Liaison 3 out 4 files from the Emergency Department were not cleaned
up as they should have been
Audit 2 Orderset Coordinator 3 out 5 files were clearly not updated properly before the
transfer
Audit 3 Charge Nurse 4 out of 5 charts are not cleaned up when charge nurse puts in
admit orders
Interview Clinical Liaison Approximately 70% of charts audited are not updated properly
Interview Clinical Educator “The only evidence I have is the comments I hear from the 2
directors of the receiving floor that the orders are not cleaned
up” “We have had 1 incident where there was some confusion
with which orders where current and were not which almost
caused a deadly situation.”
Interviews (informal) 3 Medical Unit nurses
3 ICU nurses
On average, gave the following question a 2.17: On a scale of 1 to 5, 1 being not well at all and 5 being really well, how well do the nurses perform the transfer handoff process?
Step #5 Answered Harless’s first 3 questions
1. Do we have a problem?
Yes, nurses are not properly completing the orders management process in the
Electronic Health Record before transferring a patient to another unit.
2. Do we have a performance problem?
The nurses sometimes complete the orders management process, which indicates this
may not be a knowledge or skills problem, but instead a performance problem.
3. How will we know if the problem is solved?
We will know the problem is solved, when nurses are completing the orders
management process in the Electronic Health Record before transferring a patient from
the ER to an inpatient unit, more frequently than 30 % of the time (the estimated rate of
compliance), eventually reaching 100% compliance.
Step #6 Defined the need
Team Healthcare defined the need as:
The ER must increase the frequency at which the nurses complete the orders
management process in the Electronic Health Record before transferring a patient to an
inpatient unit.
Team Healthcare
14 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
The client agreed this was the appropriate need. When the team looked at the problem, the nurses
were discontinuing orders periodically, so the team determined they had skills and knowledge to do so.
However, something was causing them not to complete the process on a consistent basis. Based on this
assessment, the client gave Team Healthcare permission to investigate the causes of this performance
gap.
Graphic 1 The Gap
Analyzing the Cause of the Gap Watkins et al. (2012) notes that the second step in a needs assessment is to collect information
regarding the causal factors or root causes that could be causing the gap. Below are the methodical
steps Team Healthcare used in determining potential causes.
Step #1 Selected a systematic approach to determining the causes
After reviewing the need to increase the frequency at which the nurses complete the orders
management process in the Electronic Health Record before transferring a patient from the ER to an
inpatient unit, Team Healthcare identified this is an operational deficiency. As Watkins et al. (2012)
points out, the operational level includes “carrying out tasks to produce results” (p. 40). This
operational task or completing the orders takes place between nurses when a patient is transferred
from the Emergency Room to an inpatient unit.
Team Healthcare chose to use the Behavioral Engineering Model (BEM) and pieces of the Synchronized
Analysis Model (SAM) to guide us throughout the NA because they align well to smaller operational level
organizational needs. The SAM utilizes pieces of the Behavioral Engineering Model (BEM), a model that
focuses on the worker (see Appendix B for the SAM and BEM models). “At this level the SAM is identical
to the BEM’s original repertoire of personal behaviors. It includes knowledge, skills, capacity, and
motives” (Marker, 2007, p. 29). Because we suspected causes of the gap stemmed from factors in the
environment, or in the individuals carrying out the tasks in the organization, Team Healthcare believed
the BEM was the most appropriate model. The team also wanted to investigate the possibility of
internal organization communication issues, which is why the team also consulted the SAM.
Required State
Current State
Nurses complete the orders
management process (discontinuing and/or completing orders) in the Electronic Health Record, before they transfer a
patient to a new unit,
approximately 30% of the time.
Nurses do not complete the orders management process
at least 70% of the time.
Desired State
Nurses complete the orders
management process (discontinuing and/or completing orders) in the Electronic Health Record, before they transfer a
patient to a new unit, approximately 100% of the time.
The Gap
Team Healthcare
15 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
The team found using the BEM and SAM also helped us “to observe behavior in an orderly fashion and
ask the ‘obvious’ (the ones we so often forget to ask) toward the single end of improving human
competence” (Gilbert, 2007, p. 95).
Step #2 Collected data to establish the causes
As outlined in Table 5, a large portion of the data collected was qualitative data, obtained by one on one
interviews. Following the suggestions of Schensul and LeCompte (1999), the interviewer captured the
responses to the interview questions, verbatim. A third party conducted onsite observations of the
process, using a checklist provided by Team Healthcare. Due to concerns for patient confidentiality, the
team was unable to audit patient files, but did have onsite personnel conduct them and provide the
team with the results. Finally, the team reviewed training agendas provided by the Clinical Informatics
Department, the department responsible for training during the implementation and for ongoing
support for the EHR.
Table 5 Data Collection Methods Data Collection Methods
17 Individual Interviews
Unit Nursing Directors x 2 – Medical Unit and ICU
Physician Liaison- CID
Orderset Coordinator- CID
Clinical Liaison- CID
Clinical Educator- Clinical Education Department
Charge Nurse- ER
Staff Nurses x 2 – ER
Staff Nurses x 3 – Medical Unit
Staff Nurses x 3 - ICU
Clinical Informatics Specialists x 2 - CID
14 File Audits Due to concerns of violating HIPAA regulations, Team Healthcare was unable to review the charts personally. Hospital personnel audited the charts and relayed the information to Team Healthcare.
4 Observations
The team created an observation form, which the client and her colleague used to conduct observations of nurses completing the transfer process.
Review of Support Materials
The team reviewed training agendas used for implementation training to determine if the training covered orders management during transfer.
Step #3 Identified major and minor data trends
After collecting the data, the team used a deductive approach to coding the data (as outlined in table 6).
Using the advice from LeCompte and Schensul (1999), the team coded and organized the data into
categories related to the BEM/SAM framework. The team then color coded the data and then
quantified the data by counting each piece of data to identify trends.
Team Healthcare
16 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Table 6 Team Health Care Code Book
Step #4 Synthesized the data to find “evidence” of major trends Table 7 Overview of Major Data Trends (see Appendix D for complete synthesis)
Information Instrumentation Motivation
Data Resources Incentives
Environmental
Major Trends
The team found:
Conflicting information concerning task responsibility in 17 pieces of evidence extracted from 10 interviews
Lack of adequate feedback in 10 pieces of evidence extracted from 9 interviews
Lack of information about support in 7 pieces of evidence extracted from 7 interviews; this is also an information problem
Major Trends The team found: Lack of time to complete the
EHR in 7 pieces of evidence extracted from 7 interviews
Lack of adequate job aids in 6 pieces of evidence extracted from 6 interviews
Knowledge Capacity Motives*
Internal Major Trends
The team found:
A lack of confidence to manage the orders due to inadequate training in 5 pieces of evidence extracted from 4 interviews
Major Trends The team found: Inadequate motivation by nurses
to manage the orders in 8 pieces of evidence extracted from 7 interviews
Inadequate motivation to work with CID support team in 10 pieces of evidence from 3 sources
* These trends are a diffusion effect stemming from a lack of information
KNWLDG
Team Healthcare
17 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Step #5 Conceptualized the causes through graphic representation
Graphic 2 Environmental and Internal Causes of Gap
Conflicting information
concerning task responsibility
Lack of information about
support tools
Insufficient/confusing resources
Lack of feedback about current
performance
Lack of adequate confidence
(skill) to clean up orders
Need (Gap)
Nurses do not
complete the orders management process
at least 70% of the
time.
Environmental
Causes 65%
Internal Causes
35%
HMC transfer process and the process of HC Regional
Medical Center do not align
Lack of time to complete the
new orders management
process
Inadequate motivation by
nurses to clean up the
orders and accept new ideas
Diffusion effect
Lack of
Information
Lack of trust
Team Healthcare
18 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Step #6 Summarized major data trends for performance gap
As represented in Graphic 1, the preponderance of evidence pointed to environmental issues as the
most influential leverage points for the performance gap. For example, the Holy Medical Center
transfer process does not align with the process of HC Regional Medical Center, the largest hospital in
the local health system, and the first site to implement the EHR. The client site opted not to adopt the
process of HC Regional Medical Center, but their reasoning was unclear and the team was unable to
collect the necessary data to understand their decision.
The following are major trends noted during the data synthesis:
1. Conflicting information concerning task responsibility
Trainers taught the process to the nurses as if the doctors “owned” the process, but
nurses are the ones actually doing this process in real time. There is a lot of hesitation by
the nurses to complete the orders management process due to the confusion of roles
and responsibilities between the doctors and the nurses.
2. Lack of information about support tools
Staff does not know who to go to regarding support and/or clarification on the orders
management process, or for general process questions regarding the EHR.
3. Lack of adequate feedback
The nurses do not receive adequate feedback regarding their performance from
Directors, the Clinical Informatics Liaisons, and/or peers in receiving departments.
4. Insufficient/confusing resources
The staff believes job aids are either too long, too complicated, and/or have forgotten
where to find them.
While most of the issues stem from a lack of information and feedback, there appears also to be a
diffusion effect that leads to motivational issues. Because the nurses do not understand the
expectations of them in the orders management process and they are not receiving feedback regarding
the process, their comments suggest they are somehow not motivated to complete the process.
However, it is apparent that nurses’ motivation is not the cause of the problem, rather a symptom that
can likely be addressed by fixing problems of the sort noted above.
Step #7 Summarized strength of organization
During the investigation of causes of the gap, Team Healthcare identified key areas of strength for the organization.
1. Nurses’ dedication and care for patients
Evidence from interviews with nurses and Nursing Directors indicate a compassion and
commitment to the patients they serve. Although they may not understand the “why”
behind orders management and what it brings to them personally, they are dedicated to
trying to do what they need to, to care for their patients.
Team Healthcare
19 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
2. Verbal pass down between nurses
Observations of 4 nurses, showed all 4 nurses doing a verbal pass down and using the
Electronics Health Record while conversing. Although Team Healthcare originally
believed the verbal pass down process was an area of need for the hospital, there was
an improvement to this piece of the transfer process following their Skills Fair held in
October.
3. Computer access to complete orders
Observation of 4 nurses showed all nurses being able to access a computer to complete
the orders management process. Although the computers were not in the patient
rooms, nurses were able to find a computer in a nearby location.
Intervention Selection Watkins et al. (2012), notes that the third step in a needs assessment is to identify multiple solutions for
achieving the desired results. Team Healthcare presented our interventions and partnered with the
client to select the appropriate interventions that would eliminate the gap and meet the organization’s
constraints of limited budget and time.
The team systematically selected intervention options by aligning the underlying causes, as identified
through the analysis, with those solutions best equipped to address the issues while remaining within
the limitations of the client organization. There were two central steps in this process:
1. Selecting a range of interventions able to close the gaps according to the data
2. Narrowing the intervention list based on organizational restrictions
Methods in Intervention Selection
As per Van Tiem, Moseley, and Dessinger (2001), step one consisted of individual and team reviews of
the findings followed by proposals of possible solutions over the course of multiple brainstorming
sessions. Each member contributed approximately three solutions with accompanying rationale. The
team then used multi-voting to generate seven possible solutions viewed as to be appropriate in
combating the performance issue. This step also produced a working list of potential constraints to
discuss with the client.
As discussed previously, the analysis findings indicate impacts on performance in the categories of:
information, resources, and motivation, with the majority of the findings suggesting deficits in
information as the primary cause. Closely related was the lack of effective feedback on this task as well
as increased time to complete the task, caused by the requirement to complete the orders management
process in the EHR. Apparent motivational issues seem to stem from a hesitation by the nurses to clear
the old orders. Data from nurse interviews show that some believe that the orders management
process is not a nurse responsibility, but rather that of the doctor while others are unsure of their role.
While this may be, in part, contributing to the performance gap, the team believes that it is not the
primary cause but rather an effect of conflicting information about task responsibility. Consequently,
we believe that with clarified information and frequent, efficient feedback motivation may increase.
Team Healthcare
20 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
It was evident that a lack of skill or knowledge concerning how to perform the task was not the likely
cause as the nurses had previously managed orders prior to the implementation of the EHR and had
received mandatory training. Because of this, the team eliminated the training-based interventions
from the list of potential interventions.
From these findings, Team Healthcare recommended a course of action that would first and foremost
clarify and inform the nurses of their role and expectations in this task and secondly, improve
performance feedback. Without this information, the nurses cannot perform as expected as they do not
know:
1. that they are expected to complete this task, and
2. how this task it to be performed (expectations).
Furthermore, providing effective feedback and gauging performance on a task unknown to the
performance as required would be a challenge and potentially damaging to the nurse’s motivation if
they feel that they are being evaluated unfairly. Additionally, the team discovered that there is
currently no method in place to collect data on performance of this task. Because performance data is
crucial to feedback as well as a host of other initiatives, we also suggested the hospital consider
implementing a process to gathering this type of information.
Though the team did find evidence of a motivational issue, we felt that because motivation is an internal
construct, thus difficult to gauge, focusing on environmental solutions would be the best course of
action to address what we believe to be the root cause. Additionally as shown in many studies
conducted by Rothwell (2005), environmental factors constitute the majority of causes behind
performance problems. As such, interventions addressing environmental causes tend to be highly
effective while also cost-effective when compared to those targeted toward addressing internal factors.
Organizational Constraints in Selecting Interventions
This step involved Team Healthcare contacting the client in a series of discussions about organizational
constraints to consider while reviewing the options. The primary constraints identified by the client are
as follows:
A limited budget
Limited time, possible resistance
The necessary for ease of implementation
Team Healthcare and the client identified five of the original seven, based on these constraints, as
solutions well-suited to address the issues. Tables 8 and 9 found on pages 21 - 23 outline solutions
approved by the client contact for further consideration by additional stakeholders. Following the
tables, are the alternative interventions, those considered but not selected, as well as thoughts on
future interventions.
Team Healthcare
21 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Primary Interventions Presented
Table 8
Interventions Addressing BEM Information Factor
BEM Category: INFORMATION
Issue Evidence Recommended Solution Benefits Disadvantages Client’s Comments
Conflicting
information
concerning
task
responsibility
Trainers told nurses,
during training, that
orders management
during transfer is the
doctor’s responsibility.
However, the Nursing
Directors and Clinical
Informatics Liaisons
believe it is the
nursing staff’s
responsibility.
1. Provide clear and accurate information
regarding task responsibility
Managers will notify nurses during a team
huddle that they are expected to complete
and/or discontinue orders prior to
transferring a patient. The Clinical Education
Department will provide a brief (less one
page) job aid outlining how to clear the
order and how to retrieve old orders if a
mistake is made. The manager should allot
time for a Q and A session.
2. Provide clear communication of
performance expectations regarding
task
Management to provide clear performance
expectations to the nurses concerning the
orders management process during transfer.
The method of communication is at the
discretion of the manager and should align
with current communication processes.
Low cost
Simple implementation
Likely to be accepted by nursing staff
Time to create job aid
The meeting or huddle may run longer than expected
“This solution makes sense and we will be
able to easily communicate this at the daily
huddle. I can make a quick job aid reminding
the nurses of their role along with how to
retrieve ‘deleted’ orders. I will make sure
the charge nurses do this immediately
(communicate to the nurses that it is their
responsibility to complete orders
management and it is within their scope of
practice to do so). I will also create a short,
concise job aid to put at the workstations for
easy retrieval.”
Team Healthcare
22 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Perception of
a lack of
support from
the Clinical
Informatics
support team
Support tools are
available to the nurses
however, there has
been a lack of
communication about
them.
The majority of nurses
reported not knowing
whom to contact for
support.
Inform nurses of the help available
from the Clinical Informatics
Department
Provide all necessary information regarding
how to contact the Clinical Informatics
support line. This might be done a variety of
ways including:
Nursing “huddles”
Memos
Post informational flyers in the unit and break rooms
Official staff meetings
Low cost
Simple implementation
Likely little to no resistance
May need some nursing resources to help with the project
“This solution is doable. We will post the
number of the CID help line all over the
hospital – including at the workstations.”
BEM Category: RESOURCES
Issue
Evidence Recommended Solution Benefits Disadvantages Client Comments
Inefficient/confusing
resources
Nurses state
that the job
aids are
difficult to use
and difficult to
find.
Job aid redesign and relocation
Consult with members of the nursing staff and
Clinical Informatics Department to create new or
modify existing aids so that they are more user-
friendly and in an accessible location.
A redesigned job aid may be more effective
Easier to use
Easier to access
The redesign process may take a large amount of time to complete
“We would have to work with
the Clinical Informatics
Department on this, but I think
this is something we should look
at anyway. I think we will assess
which job aids the nurses need
and create a quick reference
guide that they can have in the
units and at the workstations.
Table 9
Interventions Addressing BEM Resources Factor
Team Healthcare
23 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Priority of selected interventions
Following the selection, the team and client contact prioritized the solutions in such a way to promote the provision of
needed information quickly while maintaining or improving motivation. As shown below, we are suggesting that the
interventions focused on providing clear and accurate task information and expectations come before the provision of
feedback. . We believe that providing feedback during the same timeframe as this information is being communicated
may negatively impact motivation if nurses feel they are being evaluated unfairly.
1. Provision of clear and accurate information regarding task responsibility and expectations
2. Provision of feedback
3. Job Aid Redesign and Relocation
Alternative Interventions Presented
Adoption of the Emergency Room Inpatient Holding (ERIH) Area
One intervention considered ideal by the team is the adoption of the Emergency Room Inpatient Holding area,
or ERIH. This holding area is an added component to the EHR in which all orders move from the active orders
page to an archive in a patient’s file, upon unit transfer. As a result, the patient enters the new unit with a
“clean slate”. In theory, the ERIH function effectively reduces the risk to patient safety, thereby eliminating the
risk of financial obligations (lawsuits and/or settlements) related to human errors in the orders management
process during transfer process. Currently, the ERIH has been effective in the ER unit of the sister location, as
noted by a number of Holy Medical Center staff members including a Clinical informatics Specialist, a Clinical
Educator, and a Clinical Informatics Liaison. They all stated that ERIH has created a smoother, faster transfer
process that to them appears to result in less confusion about current and expired orders.
Though Holy Medical Center has the same EHR system, they chose not to adopt the ERIH function because at
the time of the implementation, the hospital believed their process to be effective, thus, not viewing the added
function as a necessary cost. However, this may not be the case as audits and nurse and director interviews
indicated an increased amount of time spent using performing this task as well as several errors or instances of
nonperformance. These issues have the potential to be incredibly costly and damaging to the organization.
Because of the level of risk associated with the current problem as well as some evidence suggesting the
effectiveness of the ERIH, the team believes that the adoption of this function may provide an optimal solution
as it essentially eliminates the opportunity for error by bypassing the human component in this piece of the
orders management process. However, due to time and constraints from the client and the limitation of the
team’s 15-week school schedule, Team Healthcare was unable to gain enough data to fully support this
recommendation.
Status: Under consideration
Rationale: Acknowledging the limitations of our data, we are not recommending it as a solution, but
rather recommending that the client look for more data to support it as the optimum solution, and
subsequently evaluate the feasibility for their site.
Performance management
The Clinical Informatics Department would determine the cost of creating a computer-generated report that
tracks progress towards 100% ER compliance with orders management when moving / admitting a patient from
the ER to an inpatient unit. The purpose behind suggesting this intervention was to provide a way for the
hospital to gain definitive information about task performance, which they could use in a host of applications.
Team Healthcare
24 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Some of the uses might include feedback for staffing and compensation decisions, or larger scale decisions such
as training programs or process design.
Status: Not chosen by client
Rationale from Client: “Since the EHR is a corporate tool that supports 21 hospitals and an organization
with 85,000 employees, it is difficult to get any changes made. I have made note of this and will suggest
it, but it is not a solution that we could implement immediately and may never come to fruition.”
Adapt performance appraisal to include orders management during transfer
The team believed by adding orders management the performance appraisal, nurses would clearly see it as their
responsibility and would work to improve their performance.
Status: Not chosen by client
Rationale from Client: “This may be feasible in the future, but at this point, I think it makes more sense
to see if we can improve performance in other ways. We have an annual performance appraisal that
happens in July – the nursing goals have already been set for this year. If there has not been
improvement by this summer, I will recommend we look at this as an option.”
Future Recommendations Based on Findings
Recommended for future implementation are the following interventions:
1. Training modification, to include:
Update to information regarding orders management task responsibility
Nurses were misinformed about their role in the task during the mandatory EHR training conducted by the
Clinical Informatics Department. The team suggests updating the training program so that this task is clearly
presented as a nurse responsibility and is appropriately discussed so that the staff feels confident in
managing the orders prior to transfer.
2. Computer skills training
Some nurses reported having difficulty with the EHR system due to a lack of computer skills. The team believes
that offering optional computer skills training, to those interested, may increase the confidence of these nurses
in their ability to use the EHR.
3. More hands-on practice to increase confidence
A number of nurses reported the desire for more hands-on practice. Adjusting the training sessions to
include hands-on activities may increase the nurses’ confidence in their ability to use the system.
Team Healthcare
25 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Intervention Alignment with Organizational Goals
Once the team and the client agreed on suitable interventions to fill the performance gap, the team reviewed the
selected interventions to ensure they were in alignment with Holy Medical Center’s organizational goals. Table 10
outlines at which level (Watkins et.al, 2012) the intervention will impact the organization and how the intervention is in
alignment with the goals of the organization.
Table 10 Interventional Alignment with Organizational Goals
Intervention Level Link to Organizational Goal(s)
Provide clear and accurate information regarding task responsibility
Operational Goals:
Implement processes that are Lean
Provide excellence in the care experience In the past three years, there has been an organizational focus on Lean processes. The organization looks at all processes and asks, “Is the way we are doing this a ‘value add’ (adding value)?” The Lean philosophy challenges the associates in the organization to look at current processes and find ways of doing them more efficiently and effectively. Currently, the ER nurses are not completing the process or only partially completing the orders management process. This puts a burden on the receiving unit nurses – they spend productive time attempting to decipher the orders. Not only is this inefficient, it could compromise patient safety. By clarifying to the ER nurses what their responsibilities are related to the orders management process during transfer, they will more effectively complete the process, thereby, saving time for the receiving nurses. This helps the hospital meet their goal of being a “Lean” organization. The nurses completing the process correctly will mitigate confusion for the receiving nurses. They will no longer have to ‘guess’ which orders are current. An accurate and complete chart represents a safer environment for patients, which helps the hospital meet their goal of providing “excellence in the care experience” (see Appendix A).
Provide clear communication of performance expectations regarding task
Operational Goal:
Earn a 4 on the Joint Commission mock survey
A hospital goal for the year 2014 is to receive a score of 4 on their Joint Commission mock survey. A 4 indicates that the mock surveyors find 10 or fewer violations and/or opportunities for improvement. The Joint Commission requires that the nurses follow a standardized process during a handoff (Friesen, White, & Byers, 2008). By communicating to the nurses the expectations for performance related to the orders management process during transfer, nursing performance should improve both in the orders management process and the overall transfer process. If nurses improve their performance, the hospital will avoid a violation during the mock survey, which will help the hospital achieve their goal of a 4.
Team Healthcare
26 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Inform nurses of the help available from the Clinical Informatics Department
Tactical Goal:
Work together as a unified healthcare system to become the “most trusted healthcare partner for life” (see Appendix A).
Through the interview data gathered, the team was able to deduce that there is a lack of
communication between the main hospital in the health system and the ancillary
hospitals. Because this organizational communication issue was outside the scope of our
project (due to time constraints), the team did not make efforts to triangulate the data to
validate this. However, the team believes that one way to improve this organizational
communication problem is to inform the nurses that there is a Clinical Informatics
Department that is available to offer support. It was clear that the nurses did not know
the department is there to offer support regarding both technical and process questions.
The associates in the CID are experts and by offering their expertise to the nurses at Holy
Medical Center, they will bridge that communication/ relationship gap. As the two
hospitals work together more collaboratively, they will become more unified in their care.
This unification will allow them to provide excellent care across the entire system, and
patients will be able to trust that they will receive great care regardless of the facility in
which they receive care. Over time, patients will begin to trust the system and the care
they receive there and will “partner” with the system as a patient – for life.
Job aid redesign and relocation
Operational Goal:
“We are impatient to do better and hold ourselves accountable for continuous
improvement in the services we offer” (see Appendix A).
The job aids designed by the Clinical Informatics Department describe both the
functionality of the system and the processes that align with the hospital’s policies. While
the aids are complete and accurate, according to the data gathered by the team (see
Appendix C for complete data) they are difficult to locate and cumbersome to use. By
assessing which job aids are most relevant at the time, re-working them to make them
more user-friendly, and relocating them so they are easily accessible, the CID and the
Clinical Education department are contributing towards the hospital’s mission to
continuously improve in order to enhance the services offered.
Conclusion While the performance gap exists and the team believes the recommended interventions will fill that specific gap, the
team noted that orders management in general (not just at transfer) seems to be an area of confusion for nurses. The
team suggests that the organization align itself in its processes and communication to ensure that nurses understand
their roles in all EHR-related tasks and processes.
Team Healthcare
27 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
References
Anonymized. (2013). Mission, values, vision. Retrieved October 17, 2013 anonymized.
Chyung, S. Y. (2008). Foundations of instructional and performance technology. Amherst, MA: HRD
Press.
Friesen MA, White SV, Byers JF. Handoffs: Implications for Nurses. In: Hughes RG, editor. Patient Safety
and Quality: An Evidence-Based Handbook for Nurses.Rockville (MD): Agency for Healthcare
Research and Quality (US); 2008 Apr.Chapter 34.PubMed PMID: 21328750. Retrieved from:
http://www.ncbi.nlm.nih.gov/pubmed/21328750Gilbert, T. (2007). The behavior engineering
model. In Human competence: Engineering worthy performance (tribute edition) (pp. 73-107).
San Francisco: Pfeiffer.
Harless, J. H. (1978). An analysis of front-end analysis. Performance + Instruction, 26(2), pp 7-9.
Hospice Foundations of America. (2013). An Introduction to grieving. Retrieved from
http://www.hospicefoundation.org/pages/page.asp?page_id=171387
ISPI. (2002). Code of Ethics. Retrieved from http://www.ispi.org/pl/About/Code-of-Ethics.pdf
LeCompte, M., & Schensul, J. (1999). Analysis from the top down. In Analyzing & interpreting
ethnographic data, 5, pp. 45-66. Walnut Creek, CA: AltaMira Press.
Mager, R., & Pipe, P. (1997). Analyzing Performance Problems (3rd ed.). Atlanta, GA: Center for
Effective Performance.
Marker, A. (2007). Synchronized analysis model (SAM): Linking Gilbert's behavioral engineering model
with environmental analysis model. Performance Improvement, 46(1), 26-32.
Miles, M., & Huberman, A.M. (1984). Qualitative data analysis. Thousand Oaks, CA: Sage.
Rossett, A. (2009). First things fast: A handbook for performance analysis. San Francisco, CA: Pfeiffer.
Rothewell, W. J. (2005). Beyond training and development: The groundbreaking classic on human
performance enhancement (2nd ed.). Saranac Lake, NY: AMACOM Books.
Team Healthcare
28 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Schensul, J., & LeCompte, M. (2013). Essential ethnographic methods: A mixed methods approach (2nd
ed.). Lanham, MD: AltaMira Press.
Shreve, J., Van Den Bos, J., Gray, T., Halford, M., Rustagi, K., & Ziemkiewicz, E. (2010). The economic
measurement of medical errors. Retrieved from:
http://www.soa.org/Files/Research/Projects/research-econ-measurement.pdf.
United States Environment Protection Agency. (2011). Lean thinking and methods. Retrieved from:
http://www.epa.gov/lean/environment/methods/index.htm
Van Tiem, D. M., Moseley, J. L., & Dessinger, J. C. (2001). Performance improvement interventions:
Enhancing people processes and organizations through performance technology. Silver Spring,
MD: International Society for Performance Improvement
Watkins, R., West Meiers, M., & Visser, Y. (2012). A Guide to assessing needs: Essential tools for
collecting information, making decisions and achieving development results. Retrieved from
http://www.needsassessment.org
.
Team Healthcare
29 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Appendix A. Mission, Vision, Values of Holy Healthcare Mission We serve together in Holy Healthcare, in the spirit of the Gospel, to heal body, mind and spirit, to improve the health of our communities and to steward the resources entrusted to us.
Core Values
Respect - We value and esteem every human person because each and every one is created by God, in the image of God. Everyone, regardless of title or position, income, education or status, race, religion or ethnicity has a dignity that is sacred. We treasure and hold human life sacred from its simplest beginnings until its end. (Gen. 1:26; Luke 9:47-48; John 10:10)
Social Justice - In this age of globalization and instant communication, we more quickly recognize our common humanity. We recognize as well the great gaps in economy, health, education and development among the peoples of this earth. Social justice commits us to the common good so that all may have their basic needs met. We recognize health and access to healthcare as a basic human right and seek to provide and advocate for it. (Wisdom 9:2f; Isaiah 32:16-18.)
Compassion - People come to us when they are in need and in distress. In the spirit of Jesus, we recognize their need and seek to respond to it. We reach out to them in their pain and suffering and care for them in body, mind and spirit. The ability to feel and to respond to the suffering of others is an essential value in our ministry of healthcare, no matter where we serve. We recognize also that those we cannot cure we can still love, care for and be with in their suffering. (Mark 1:40f; Mark 10:51; Luke 4:40 Luke 10:30-37; Romans 12:15; I Cor. 13:4-7)
Care of the Poor and Underserved - God hears the cry of the poor and so, with respect and compassion, we seek out the poor and underserved as a special focus of our healthcare ministry. It is those without resources of their own who need us most. We seek to care not only for their immediate needs but also to change the structures that keep them in unhealthy environments and inhumane conditions. Through our ministry of health care and our persistent advocacy, we seek to serve the poor and underserved of our communities. We too hear the cry of the poor and underserved. (Acts 4:32-35; James 2:15)
Excellence - The scriptures look to the day when there will be a new heaven and a new earth, when creation will be made perfect. Our vision is no less. In all we do, we reach for more -greater respect, fuller justice, deeper compassion, better care, less poverty. We are impatient to do better and hold ourselves accountable for continuous improvement in the services we offer. (Matthew 25:14-23; II Corinthians. 9:6; Revelations 21:1)
Vision Inspired by our Catholic faith tradition, Holy Healthcare will be distinguished by an unrelenting focus on clinical and service outcomes as we seek to create excellence in the care experience. Holy Healthcare will become the most trusted health partner for life.
Team Healthcare
30 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Appendix B. Models
Synchronized Analysis Model (Marker, 2007, p. 28)
Behavioral Engineering Model (Gilbert, 1978, p. 88)
Team Healthcare
31 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Appendix C. Data Collection
Nursing Director Interviews
RN Director Interview Guide Date: Interviewer: Interviewee:
Question Response Please describe what Genesis is and what was the main purpose of implementing it at the hospital? Probe if needed: ● How was this message shared with the RNs?
● Do you believe the purpose has been reached?
How has the new online orders management process changed hospital processes? Where have you noticed the biggest impact? Probe if needed:
How has the transfer process been impacted?
How has the handoff process been impacted?
Do you have anyone who has truly been an exemplary performer since the implementation? Why have they embraced this change so well?
What type of support was originally given to the RN staff to learn the system and what support is given today? Probe if needed: ● How was training given? ● How is the information kept up?
● What is your role in this support?
What are some of the barriers that you believe might be stopping the RNs from doing the orders management process in the EHR? Probe if needed:
Do the RNs have the skills to use the system?
Do the RNs have the knowledge to use the system?
Are the nurses motivated to do the orders management process?
What are the consequences for not keeping up the EHR? Who is responsible for making sure these consequences don’t occur? Probe if needed: ● How would a nurse know when he/she isn’t meeting standards?
● Who monitors standards?
When I talk to the nurses, what do you believe they will say if I ask them what they need to be successful in complete the orders management process on a consistent basis?
What else would you like to share about this current situation with the EHR?
Team Healthcare
32 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
RN Director Interview Guide Date: 10/08/2013 Interviewer: Angie Wolthuis Interviewee: Director of ICU
Question Response Please describe what Genesis is and what was the main purpose of implementing it at the hospital? Probe if needed: ● How was this message shared with
the RNs?
● Do you believe the purpose has
been reached?
Purpose –INFO and MVTN it was a top down initiative, but the nurses do not care about the WHY. INFO and MTVTN I don’t know that they fully understand the purpose, however, it does not matter. They know those types of decisions cannot be changed, so they want to quickly incorporate it into their practice so they can continue to do good work.
How has the new online orders management process changed hospital processes? Where have you noticed the biggest impact? Probe if needed:
How has the transfer process been impacted?
How has the handoff process been impacted?
Do you have anyone who has truly been an exemplary performer since the implementation? Why have they embraced this change so well?
The nurses have done a good job adjusting to the new EHR. INFO The whole process is electronic now, which is a huge change. In the past the nurses charted in Meditech, but the orders were not in the chart. Meditech was different as well because we had the ability to make changes based on our needs. We could ask for a change and it would happen in a day (changes to help with workflow, etc.). The new system is a universal system for the entire corporation, so one change is huge because it impacts everyone. ORG We can no longer make changes based on our needs. ORG We can make requests, but there are not guarantees as to if the changes will be made or not. This change of not really having any say in the way the system works is huge. ORG The system used to work for us, now we work for the system. Cerner designed the system for Evidence-Based Practice (EBP). EBP is one size fits all. ORG The system is not bendable; the culture has to bend to fit the system. It has changed INFO the transfer process because it is a disruption in the workflow. There are interruptions by the system (EBP reminders, etc.) – interruptions disrupt the thought process and it takes time to get back on track. Nurses are used to getting focused and completing a task, now they are RSRC constantly interrupted by the system reminding them to do things. There is also SO much information on a chart. It is difficult to sort through all the information.
What type of support was originally given to the RN staff to learn the system and what support is given today?
KNWLDG The training was all done in a “fake” environment. It was all theoretical. We were not able to use a live system and practice on real charts. The real learning happened on the day of the go-live. It is impossible to teach everything in the training – you just cannot prepare people for every situation they will encounter in
Team Healthcare
33 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Probe if needed: ● How was training given?
● How is the information kept up?
● What is your role in this support?
the real world. A lot of people are tactile learners and will only learn by actually doing it. Another issue with the training was that ORG the people doing the training were not nurses. They were non-clinicians, so their brains process differently. It would be more effective if it were clinician to clinician training. The ones that were more effective that were not clinicians were educators (there was a gentleman that was a former teacher). He was effective because he understood how to teach. He had a training background. INFO and RSRC The directors disseminate all of the information to the nurses. Job aids come from somewhere… not sure where and we are expected to get them to the nurses. They are difficult to understand and way too long. INFO We don’t know who to contact to implement changes or who to talk to about training. (Angie: I followed up with, what about the Clinical Informatics Department - don’t you have a relationship with them?) I don’t know what the relationship between the hospital and the CID should be. INFO I don’t even know who to contact for training support or questions. I am a director, if I don’t know who to contact, how will the nurses know who to contact. ORG We don’t even know who our support from the CID should be. Shellie does something, but I am not sure what. We have no on sight support. The main location has a number they call and they get at the elbow, we have no at the elbow. We don’t even know who to call to ask for help.
What are some of the barriers that you believe might be stopping the RNs from doing the orders management process in the EHR? Probe if needed:
Do the RNs have the skills to use the system?
Do the RNs have the knowledge to use the system?
Are the nurses motivated to do the orders management process?
There are huge generational differences in the nurses. The younger generation is great with technology, but the older generation is experienced. For the experienced RNs, there has been a huge cultural shift. They were experts and now they are novices. KNWLDG I think some of the RNs do not feel confident in their computer skills; it has made them question their abilities as nurses.
What are the consequences for not keeping up the EHR? Who is responsible for making sure these consequences don’t occur? Probe if needed: ● How would a nurse know when
he/she isn’t meeting standards? ● Who monitors standards?
Patient safety is compromised. There are time delays. RN Consequences – the process implemented is not the RNs fault. We use Just Culture in the organization and we try to find out the root cause and understand why something is not working. MTVN We don’t punish people unless they are intentionally doing something that will harm patients.
Team Healthcare
34 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
When I talk to the nurses, what do you believe they will say if I ask them what they need to be successful in complete the orders management process on a consistent basis?
1:1 help –KNWLDG they need an expert with them. They never had a real expert at their elbow. They had had super users on the floor, but these were not necessarily experts in the process. If every nurse could have 1:1 help for a month – to ask about any situation that may arise (as not all situations arise every day), they could be consistent. They also need tools that are readily available. The job aids are on a website, but everything is there. RSRC If you are in a hurry, you do not have time to sort. I don’t have time to sort through them, I know a nurse does not have time to sort through them. There is too much information there. RSRC They need simple job aids with bullet points – do this, do this, do this. Not 15 page documents with step by step instructions, including the why they are doing it. It needs to be succinct. They need to be able to get to them quickly – one click, not fifteen clicks.
What else would you like to share about this current situation with the EHR?
RSRC The directors are responsible for auditing the charts, which adds even more to an already huge workload. RSRC We don’t know who to contact to implement changes.
Team Healthcare
35 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
RN Director Interview Guide Date: 10/08/2013 Interviewer: Angie Wolthuis Interviewee: Director of Medical Unit
Question Response
Please describe what Genesis is and
what was the main purpose of
implementing it at the hospital?
Probe if needed:
● How was this message shared with the RNs?
● Do you believe the purpose has been reached?
The purpose of implementation is to have a “flow” of information. In the past,
there was a paper chart and only one person could have access to that chart at a
time. With an electronic chart, there should be a more seamless flow of
information and anyone can have access to any chart at any time.
MTVN The RNs are mourning the loss of paper, particularly those that have been
in practice for a long time, but INFO the information is flowing and they
KNWLDGE understand the purpose. The purpose has been reached, but it does
not mean that they are not still adjusting to the huge change.
How has the new online orders management process changed hospital processes? Where have you noticed the biggest impact? Probe if needed:
How has the transfer process been impacted?
How has the handoff process been impacted?
Do you have anyone who has truly been an exemplary performer since the implementation? Why have they embraced this change so well?
INFO Everything has changed related to how the nurses do their work. There is
bedside medication administration – they scan a wristband on the patient when
they administer the meds. This slows things down for the nurses.
During the transfer process, the INFO information is easier to see. The nurses can
see everything that was done on the patient in the other unit.
I can’t think of an exemplary performer in this area because there are so many
variables. There are some that are comfortable with the documentation piece,
but the lack in other areas. The thing about this change is that the nurses that
were excellent nurses MTVN now feel like they are starting over. They are MTVN
unsure about themselves. They are still great nurses, but they INFO computer
has thrown them off and made them MTVN question themselves. Some of the
newer nurses are better at the computer piece of things, but they still need to
KNWLDG build their skills in other areas.
The biggest challenge is that MTVN experienced nurses now feel like novice
nurses. ORGIt has been a very difficult cultural change.
Team Healthcare
36 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
What type of support was originally
given to the RN staff to learn the system
and what support is given today?
Probe if needed:
● How was training given? ● How is the information kept up? ● What is your role in this support?
The nurses KNWLDG had training in a classroom and then “at the elbow” support
from Super Users.
There are ongoing changes in the system. INFO We (the Directors) get updates
on the system or processes from the corporate office about once a month (via
email) and then we are expected to disseminate the information to our staff
members. They send everything to us (the Directors) including job aids.
The only support we have now for training, other than the updates from
corporate, are the INFO Super Users that are on our floors. However, they have
gone back to their jobs, so they are not receiving ongoing training in the system.
Corporate will be back on site to do advanced training next month.
What are some of the barriers that you believe might be stopping the RNs from doing the orders management process in the EHR? Probe if needed:
Do the RNs have the skills to use the system?
Do the RNs have the knowledge to use the system?
Are the nurses motivated to do the orders management process?
RSRC Time can be a barrier – pressure to get the patient transferred out. This is
mostly related to the transfers that come to us from the Emergency Department.
MTVN Resistance – sometimes they don’t see the need to do the orders
management (when transferring). However, when we get the patient in our unit
and the orders haven’t been cleaned up, we don’t know what orders are current.
Discontinuing orders is a culture change. Previously, there was a paper chart.
MTVTN When a patient was transferred, you just turned the page and started
new orders and ignored the previous page. You could go back and refer to the
orders on the previous page, it didn’t feel as final as it does now, when you
discontinue in the system. There is something final about clicking delete in the
EHR. Even though we can still refer to the orders, it is a mindset change. INFO
Also, nurses don’t make changes to orders- that is supposed to be the role of the
doctor. I think some nurses feel like that by deleting the orders they are
practicing outside their scope – even though they aren’t. It is just such a cultural
change.
What are the consequences for not
keeping up the EHR? Who is responsible
for making sure these consequences
don’t occur?
Probe if needed:
● How would a nurse know when she isn’t meeting standards?
● Who monitors standards?
INFO Their peers will let them know – peer feedback. INFOThe receiving nurse
will call the transferring nurse and ask for clarification, which takes time. This
doesn’t happen all the time, but it is a consequence.
VOICE reports- if someone thinks something is unsafe they will report it in the
VOICE system (this is an anonymous system where employees can report safety,
ethics, and other violations).
INFO If there are real problems, the nurses will also communicate that to the
Directors when we round the floors.
Team Healthcare
37 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
When I talk to the nurses, what do you believe they will say if I ask them what they need to be successful in complete the orders management process on a consistent basis?
RSCRC More time – not to feel so much pressure about time.
KNWLDG Better knowledge of the system and the processes.
INFO We need to reiterate the message that it is okay (in reference to the earlier
comment about it being the doctor’s job) to clean up the orders – we need a
clean slate when the patients come to our floor. This helps everyone do their job
better.
What else would you like to share about
this current situation with the EHR?
Team Healthcare
38 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Emergency Room Nurse Interviews
Nursing Questions Unit Nurse is on: Emergency Department Any experience with EHR in past? Yes, but not Computerized Physician Order Entry
1. Why did the hospital move to the Electronic Health Record (EHR)?
2. How has the implementation of the EHR impacted the time it takes to transfer a patient (as an admit to the floor)?
3. When transferring a patient to the floor, whose responsibility is it to clean up the orders (e.g. discontinue, complete, etc.)?
How comfortable are you cleaning up orders?
Has the process been document or explained to you?
4. If you had trouble with the EHR system, not the computer itself, but questions on the process, do you know who to contact for support?
Would you contact someone for support if you knew who to contact?
5. Do you know how to locate the job aids that were made to help you with the EHR?
6. Help me understand how hard or easy it is to find information on the charts? Is it easy or hard to sort through information?
7. What is your comfort level using the computer?
8. If you don’t clean up the orders before you send the patient to the floor, what happens?
ADDITIONAL QUESTION (For RN Supervisor only) What kind of feedback do the nurses get if they do not complete the transfer process correctly?
What is the feedback from the supervisor or leader if they do not complete their orders before the transfer?
Team Healthcare
39 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
40 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Nursing Questions- Nurse #2 (Supervisor) Unit Nurse is on: Emergency Department Any experience with EHR in past? Yes, but not Computerized Physician Order Entry
1. Why did the hospital move to the Electronic Health Record (EHR)?
MTVN This was an initiative from the corporate office. To meet certain standards with the federal government, they had to go electronic. I don’t know if there are better systems out there, but everyone in health care will be in the same boat. I think for some things it is better (you don’t have to look for the patient’s chart), but for other things, it is not that great. INFO We will just have to figure it out.
2. How has the implementation of the EHR impacted the time it takes to transfer a patient (as an admit to the floor)?
It has definitely slowed things down. RSRC It takes more time to do pretty much everything and in the ER, that is rough.
3. When transferring a patient to the floor, whose responsibility is it to clean up the orders (e.g. discontinue, complete, etc.)?
How comfortable are you cleaning up orders?
Has the process been document or explained to you?
It depends; there is some drama around this. INFO The charge nurse is supposed to do the admit orders, so it can be a little unclear about who is supposed to do the other pieces. The nurse, in theory, should discontinue and complete orders, but they don’t always do it. MTVTN I would not say they are all comfortable with discontinuing orders. When we had training – KNWLDG it seems like they just “skipped over” this. I am INFO not sure if it is very clear as to what is okay and what is not okay.
4. If you had trouble with the EHR system, not the computer itself, but questions on the process, do you know who to contact for support?
Would you contact someone for support if you knew who to contact?
We would go to someone in our department – or Shellie (someone local that does not have the primary responsibility for training). INFO I don’t think that anyone knows how to get a hold of the people in the main location. I may call them if I needed help, but I don’t know how to get a hold of them.
5. Do you know how to locate the job aids that were made to help you with the EHR?
Hmm, job aids – I know what you mean, but I have RSRC no idea where they are. I don’t know if they would be helpful because things move so fast, I don’t really have time to refer to something that is that long – their job aids are long.
Team Healthcare
41 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
6. Help me understand how hard or easy it is to find information on the charts? Is it easy or hard to sort through information?
When we get the patient, it is easy as we are the first point of contact. INFO However, finding infor on the charts can be confusing. The other day we had an MD call us from the Health Plaza. He had a little boy that had been in the ER and wanted to know about the X-Ray we did on him.
7. What is your comfort level using the computer?
KNWLDG I am fine using a computer – this is just a totally different system than I am used to.
8. If you don’t clean up the orders before you send the patient to the floor, what happens?
MTVN It depends. Sometimes the floor calls us – other times they just figure it out.
ADDITIONAL QUESTION (This nurse is a supervisor in the ER)
9. What kind of feedback do the nurses get if they do not complete the transfer process correctly?
What is the feedback from the supervisor or leader if they do not complete their orders before the transfer?
INFO If we catch anything as supervisors or charge nurses, we will talk to them. Patient safety is the most important thing to any nurse, so we want to make sure the patients are taken care of and safe. INFO We don’t really have a system in place to check to see if the orders are taken care of before they are transferred. Again, I am not sure if anyone knows whose job it is. It is not the MD’s job to do it in the system – they wouldn’t have time. The nurses should be doing it, but if they aren’t, they may not know it is their responsibility.
Team Healthcare
42 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Nursing Questions- Nurse #3 Unit Nurse is on: Emergency Department Any experience with EHR in past? Quite a bit of experience, but not Computerized Physician Order Entry
1. Why did the hospital move to the Electronic Health Record (EHR)?
Every hospital and health care organization has to move to an computerized charting system. MTVN It is a part of Obamacare.
2. How has the implementation of the EHR impacted the time it takes to transfer a patient (as an admit to the floor)?
I have quite a bit of experience with electronic charting, so I don’t think it has really changed the time it takes me. RSRCS It may take a little more time, but not much.
3. When transferring a patient to the floor, whose responsibility is it to clean up the orders (e.g. discontinue, complete, etc.)?
How comfortable are you cleaning up orders?
Has the process been document or explained to you?
INFO The nurses are supposed to clean them up, I think, but it probably doesn’t always happen. It gets a little confusing with the holding orders, the admit orders, and the ER orders. It can be confusing as to who is supposed to do what. KNWLDG The process hasn’t really been explained; maybe it has, but I forgot. We had so much to learn, that is just one piece of it. I can’t really remember what they told us. (Future opportunity: Training)
4. If you had trouble with the EHR system, not the computer itself, but questions on the process, do you know who to contact for support?
Would you contact someone for support if you knew who to contact?
INFO Not like a formal help line or anything. I would maybe call someone if there was a number for support.
5. Do you know how to locate the job aids that were made to help you with the EHR?
RSRC I know they showed us where they are, but I don’t remember.
6. Help me understand how hard or easy it is to find information on the charts? Is it easy or hard to sort through information?
KNWLDG Finding information can be tricky, but I have a lot of experience, so I am fine.
Team Healthcare
43 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
7. What is your comfort level using the computer?
I am pretty experienced, so I am good with it.
8. If you don’t clean up the orders before you send the patient to the floor, what happens?
INFO It depends. Sometimes the floor calls us – other times they just figure it out.
Team Healthcare
44 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Training Department Interview Guide
Date: Interviewer: Interviewee:
Question Response
What experience do you have in using the EHR? Did you work with any of the nurses or managers to create the training? How was the course piloted before the roll out and to whom?
Who participated in the training? Probe if needed:
Was this mandatory training?
Did the managers have to participate? Do you know if you had 100%
attendance? Do you have records?
What were the course objectives for this training? (goals)
Probe if needed:
Who set the course objectives?
Were there any constraints such as limited of time, space, budgets to providing this training? Describe them.
Describe what training the nurses had on the orders management process during transfer. Probe if needed:
Describe how you showed or demonstrated to your participants how to use EHR.
Were they trained on workflow process
or only functionality?
What type of hands on practice did your audience get during training? Did they get on the computer and actually practice?
How did the nurses know if they were doing the orders management process correctly during training? (What type of feedback did they get?)
Describe to me how your participants were evaluated to determine competency of what they learned?
• During the training did you discuss the why of orders management?
• Do they understand why they need to
Clinical Informatics Department Trainer Interviews
Team Healthcare
45 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
complete and/or discontinue the orders?
• Was there any hesitation when it came to orders management on the part of the nurses?
• If so, why were the nurses hesitant?
What was the support plan for nurses after training? Probe if needed:
• Are there job aids available on the topic or orders management during transfer?
• How do they access the job aids?
• Who do they contact for support? Do
they have access to help from the CID team and how do they know who to contact?
• How often do you get calls from the
Other Location nurses?
• What would you say the majority of those calls are about?
• Do nurses have the option to take
advanced training? If so, is orders management during transfer part of this training?
What does that advanced training cover and how does it cover that material (hands-on training, scenario-based)?
Do you have any feedback from your participants about how confident they felt about completing an EHR after training?
Team Healthcare
46 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Training Department Interview Guide
Date: 11/8/2013 Interviewer: Angie Wolthuis Interviewee: Trainer D
Question Response
What experience do you have in using the EHR?
Did you work with any of the nurses or managers
to create the training?
How was the course piloted before the roll out
and to whom?
I worked on this project and previous ones with this same health system.
The content has been adjusted based on upgrades and feedback, but most of
the content comes from the parent company and was piloted long ago; if it
was piloted at all.
Who participated in the training?
Probe if needed:
● Was this mandatory training?
● Did the managers have to participate?
● Do you know if you had 100%
attendance? Do you have records?
KNWLDG All staff – RN, ancillary, billing, coding, registration, MDs, mid-
levels, etc.
The training was mandatory – everyone was paid for their time in training.
Yes, the managers participated in training as well.
If people did not attend, they were not allowed to work, so I think it is safe to
say we had 100% attendance.
What were the course objectives for this
training? (goals)
Probe if needed:
● Who set the course objectives?
● Were there any constraints such as
limited of time, space, budgets to
providing this training? Describe them.
We wanted the people going through training to be proficient in both
functionality and integrate it (the EHR) into their workflow process.
.
There are always time constraints – get everyone trained in a couple of
months, ready to work for the go-live. There is always a limited budget as
well in a non-profit healthcare organization. We had to work with what we
had and do the best job possible. (Future opportunity: Training)
Describe what training the nurses had on the orders management process during transfer.
Probe if needed:
● Describe how you showed or
INFO A lot of orders management falls on the Doc, so we don’t hit orders
management too hard in the classes. INFO They can do some orders
management over the phone with the doc, but it is really the job of the Doc,
so we don’t go over it too much. ORG The problem is, in the other location,
Team Healthcare
47 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
demonstrated to your participants how
to use EHR.
● Were they trained on workflow process or only functionality?
● What type of hands on practice did your
audience get during training? Did they
get on the computer and actually
practice?
● How did the nurses know if they were doing the orders management process correctly during training? (What type of feedback did they get?)
● Describe to me how your participants
were evaluated to determine
competency of what they learned?
• During the training did you discuss the
why of orders management?
• Do they understand why they need to
complete and/or discontinue the orders?
• Was there any hesitation when it came
to orders management on the part of the nurses?
• If so, why were the nurses hesitant?
they chose to do a different process than the main location on transfer from
the Emergency Department to an Inpatient department. The main location
has a “holding” area for patients and all their orders drop off after they leave
the ER. In the other location, their orders stay on the chart. I don’t know
why they chose to do that. It is a huge problem because the RSCR ER nurses
don’t have time to clean up the orders, yet it is supposed to be their job.
The nice thing is, at the main location they start with a clean slate. In the
other location, RSCR the chart is still full of orders and they are relying on
humans to clean the orders up – that human component is the wild card.
We did a presentation and then had the nurses do some hands-on practice.
KNWLDG They practiced both functionality and process. They had fake
patients that were prepped with orders so they could practice the scenarios.
I remind nurses of INFO he policy: use reason, logic, and good faith and you
will know if the decision you are making related to orders management is
correct. ( Team: Is this really the policy?)
The why we discussed was that when there is an order it generates a task for
the nurse. If you don’t clean up the orders, as a nurse, you are actually
creating more work for yourself by there being more tasks generated. MTVN
They understand the why, but I don’t know if they buy into it.
The hesitation is that it is not all their job. Part of it is the doctor’s job. It is
also a generational thing. KNWLDG Some RNs are more comfortable with
computers. INFO Some of the more tenured nurses also feel like it is outside
their scope of practice. I tell them not to think of the Orders tab as the
“orders” tab – they look at that as physician-driven. I tell them to think of it
as the engineering deck, not the orders component. This is where all other
tasks, etc. are “engineered” from.
What was the support plan for nurses after
training?
Probe if needed:
• Are there job aids available on the topic or orders management during transfer?
• How do they access the job aids?
We have been begging for advanced training forever and we are finally
getting it. KNWLDG They need more training on patient care, upgrades, etc.
There are RSCR job aids in Sharepoint. We do tell them in class how to access
the job aids. There is also a Guide Me function in Cerner now. This is really
high-level stuff from the parent company, but it is new and can be helpful.
Team Healthcare
48 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
• Who do they contact for support? Do
they have access to help from the CID team and how do they know who to contact?
• How often do you get calls from the Other Location nurses?
• What would you say the majority of those calls are about?
• Do nurses have the option to take advanced training? If so, is orders management during transfer part of this training?
● What does that advanced training cover
and how does it cover that material
(hands-on training, scenario-based)?
They contact 5448 for help, which is our team (CID team). We told them in
class to contact that number for support. That should be their first line of
defense. After that, they should contact their site liaisons, but they are so
busy with physician issues, they don’t have time to help. INFO I actually don’t
know which site liaison there would help with RN issues. I am not sure they
know who to contact there. I don’t think the liaisons understand their own
roles.
We rarely get calls from the other location. The majority of them are system
access requests (password help, etc.). ORG the other location is typically
dismissive and rude to the mian location people anyway. They are not open
to “help” from the main location – it is like they have something to prove.
They don’t listen when I try to help them.
MM is responsible for helping put together the advanced training; I am not
sure what they will cover, but I suppose orders management will be in there.
All the nurses will eventually take it.
Do you have any feedback from your participants
about how confident they felt about completing
an EHR after training?
INFO Not really, as I said – it is really the job of the Doctor.
Team Healthcare
49 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Training Department Interview Guide
Date: 11/18/2013 Interviewer: Angie Interviewee: Trainer P
Question Response
What experience do you have in using the EHR?
Did you work with any of the nurses or managers to create the training? How was the course piloted before the roll out and to whom?
I was hired to help with the implementation. The training was already created and I believe it was created by the Holy Healthcare, the parent company.
The hospitals have been using the same content for all of the go-lives in the system, so it was not piloted at our site.
Who participated in the training? Probe if needed:
Was this mandatory training?
Did the managers have to participate?
Do you know if you had 100% attendance? Do you have records?
KNWLDG There was training for everyone working in the hospital: Provider training, RN training, ancillary services training, billing training, registration, etc.;(team note: no orders management?) they were all split out by what they did. Training was separate for each “group” of people and all managers were required to participate.
Training was mandatory and people were not allowed to work until they completed the training.
There had to be 100% attendance as anyone that didn’t attend was not allowed to work.
What were the course objectives for this training? (goals)
Probe if needed:
Who set the course objectives?
Were there any constraints such as limited of time, space, budgets to providing this training? Describe them.
There are too many objectives to cover. We really wanted them to be able to use the system and be confident in their abilities to do so. The course objectives were set by the parent company. We had time constraints for sure. We had to get everyone trained by the go-live and had just a couple of months to do it. There are always budget constraints, but the nurses were paid for the training and if they needed additional training, they were paid for that as well. (Team: Note for future opportunities training improvement)
Describe what training the nurses had on the orders management process during transfer. Probe if needed:
Describe how you showed or demonstrated to your participants how to use EHR.
Were they trained on workflow process or only
INFO Orders management is the job of the provider (doctor). The nurses are there to facilitate it (they can take the orders to do it over the phone or verbally), but it is really the responsibility of the provider. If they are going from the Emergency Department to Inpatient the nurses call the physician they are admitting the patient to and let that MD know. The admitting MD goes through the meds. KNWLDG The nurses were trained on both process and functionality. The
Team Healthcare
50 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
functionality?
What type of hands on practice did your audience get during training? Did they get on the computer and actually practice?
How did the nurses know if they were doing the
orders management process correctly during training? (What type of feedback did they get?)
Describe to me how your participants were evaluated to determine competency of what they learned?
• During the training did you discuss the why of
orders management?
• Do they understand why they need to complete and/or discontinue the orders?
• Was there any hesitation when it came to
orders management on the part of the nurses?
• If so, why were the nurses hesitant?
practice was process, but of course we could not cover every situation in training. The KNWLDG RNs did get hands-on practice. They were all assigned to a workstation and they used them during the class. We had fake patients loaded with real data, provided to us by the parent company and created by real nurses. INFO I wouldn’t say we gave them feedback during training about that – we just really hammered things hard during our presentation. They were given quizzes at the end of each section. If they didn’t pass, they had to have more individualized training with a trainer. KNWLDG Eventually, everyone had to pass in order to go back to work. (team how do we know?)
The why was discussed – keeping the chart clean so that there are no mistakes with possible duplicate orders, etc. MTVTN However, nurses see orders management as “not my problem”. If they are the transferring nurse (particularly from the ED to inpatient), they see that the receiving nurse will be responsible for the care of the patient and will therefore be the scapegoat if there is a mistake. ED nurses say they do not have time to do the orders management process. RSRC The hesitation is time. It takes too much time and they INFO and MTVN feel some of it is out of their scope of practice.
What was the support plan for nurses after training? Probe if needed:
• Are there job aids available on the topic or orders management during transfer?
• How do they access the job aids? • Who do they contact for support? Do they have
access to help from the CID team and how do they know who to contact?
• How often do you get calls from the Other Location nurses?
• What would you say the majority of those calls are about?
• Do nurses have the option to take advanced training? If so, is orders management during transfer part of this training?
What does that advanced training cover and how does it cover that material (hands-on training, scenario-based)?
The nurses were given at the elbow support from super-users for approximately a month. After the support went away, RSCRS they were told to call ext. 5448, which is the CID team (support team). That number is RSRC available M-F 7am to 7pm and there is someone on call on the weekends as well. We do get calls from Other Location nurses. I remote in on their computer and sometimes look to see what the problem might be. This allows me to see their screen and what they are working on. The calls are mostly about charting. There are job aids on a Sharepoint site. RSCS I would say there are at least 100 job aids. There is probably too much info for them there and I have been told that they easily get sidetracked on the site and it is probably hard to find exactly what they need. They will be required to take advanced training in the next couple of months. They will be paid for their time in advanced training. I am not sure that everything is finalized on what the advanced training will cover, but I am sure it will be hands-on.
Clinical Informatics Department Liaison Interviews
Team Healthcare
51 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Question Data Source Answer
1. On a transfer who holds the primary responsibility for orders management during the transfer process?
(The physical process in the EHR, not responsible for determining which orders continue or not.)
2. Are the nurses doing it? If not, why do you think the nurses are not doing the orders management during the transfers?
3. Do you have any evidence that tells you they are not doing it?
4. What do you do with the
evidence that this the orders management is not being done?
Team Healthcare
52 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Question Data Source Answer
1. On a transfer who holds the primary responsibility for orders management during the transfer process?
(The physical
process in the EHR,
not responsible for
determining which
orders continue or
not.)
2. Are the nurses doing it? If not, why do you think the nurses are not doing the orders management during the transfers?
3. Do you have any evidence that tells you they are not doing it?
4. What do you do with the evidence that this the orders management is not being done?
Physician Liaison
Clinical Informatics
Department
1. INFO The nurses, they don’t do it, but it is their job.
2. INFO I think there has just been so much going on and no one is really clear on what their role is. Each hospital is different, so that can be challenging as well. ORG They have their own policies and sometimes our team is not even 100% clear on who is supposed to do what in each hospital.
3. INFO We audit the charts from time to time for our purposes, but in my audits, I have noted that the orders from the Emergency Department were not “cleaned up” as they should have been. I just did an audit last week and 3 out of the 4 charts were not properly updated.
4. Since most of our audits are informal, and more for the purpose of gathering information, I don’t do anything formalized with the information. My primary responsibility is to work with providers (physicians and mid-levels), and if there is something that is directly affecting them, it becomes my problem to solve. INFO At this point, I just noted the information and may bring it up to colleagues if it seems to be an issue.
Orderset Coordinator
Clinical Informatics
Department
1. INFO The nurses should be doing it – the actual in the computer piece. They obviously consult with the MDs on this, but they should be completing or discontinuing orders; cleaning them up.
Team Healthcare
53 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
2. INFO No, they are not consistent about it at all. I think they just aren’t sure about what their responsibilities are when it comes to this process.
3. Our department does chart audits from time to time and since I used to support the Emergency Department in the main location I look at ED stuff. Last week we all did some independent audits (your project has brought some awareness to this topic) and I found that 3 out of the 5 charts were clearly not updated properly before the transfer.
4. While my job is to review orders, it is merely for the purpose of making sure the orders that we built (the organization builds ‘sets’ of orders) are working correctly. I review the ordersets or the PowerPlans and note any issues with those. I noticed this in my audits because you asked us about it. INFO I merely gathered the information for you and we will probably note the information as a department and disseminate it to the directors at all sites.
Clinical Liaison
Clinical Informatics
Department
1. INFO The nurses should be doing the clean-up in the charts, I suppose. Really, I think the doctors should do it, but everyone says that since they won’t do it, so it falls on the nurses. That is typical – the nurses come behind the docs and clean up their messes! In this case, the nurses should be doing the clean-up after they have consulted the providers on what orders should and shouldn’t continue.
2. ORG I don’t think they are doing it in the other location like they should be. It is probably something that was lost in translation during the whole go-live process. We may need to go back and clarify things to them.
3. Yes, I am responsible for looking at clinical issues within the system. While I am not the clinical liaison for that site, I do look at issues in the system as a whole and it was recently brought to my attention, through some audits, that orders are not being properly managed during the transfer process (and in general). This is on my radar now. (Follow-up from interviewer: In the audit process, how many charts would you say are updated correctly during the transfer process?) I would say probably about 25-30% are correct. That is nowhere near where it needs to be. I would like to see 100%.
4. I did the audit at your request. INFO We did note the issue and will bring it up at IOT (a meeting that involves people from the corporate office as well as local leaders – they assign problems for resolution and report on those at this meeting). We have brought it up before, but the other location does not always have a representative at the meeting. They are always invited, but they are also in the middle of building a new “hospital” (heart care unit and maternity unit) and they have been working tirelessly to get this done. I will communicate the issue to them again and discuss it with their clinical liaison.
Clinical Educator
Clinical Education
Department
1. The nurses should do the orders management, but it can be a complex process. There are holding orders, admit orders, and current orders. It can be a little tricky to know who is supposed to do what.
2. From what I observed during the observations, they are “kind of” doing it. They are doing it to a point, which is why I checked “yes” on the observations, but they are not completely doing the entire process. Now that I think about it, I should have probably made more notes on the observations that talked about how they didn’t fully complete the process.
Team Healthcare
54 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
3. The only evidence I have is the comments I have from Directors or the receiving floors telling me that the orders are not cleaned up. We did have one incident where there was some confusion with which orders were current and which were not – it could have been a deadly situation, but it wasn’t. One near miss is enough.
4. It is not necessarily my job to fix this issue, but I do bring it up to the directors. I also was hoping you (Team Healthcare) could bring us some recommendations!
Charge Nurse
Emergency Department
1. I suppose it is the nurse’s job– well I am pretty sure it is, but it doesn’t get done like it should. It just doesn’t.
2. They are not doing it like they should. I think some of them know, but don’t have time and others are just not sure what is their job and what is the job of the MD.
3. When I charge (work as the charge nurse), I put in the admit orders and I see all the charts. From seeing these charts I know that they are not doing it. (Follow-up from interviewer: How many charts, out of every 5, would you estimate, do not have the orders cleaned-up like they should be?) Probably 3 out of every 5 do not have the orders cleaned up. Maybe even 4 out of 5. They are pretty consistently not cleaned up.
4. I do bring this up at huddle, but at this point I think I need the process and the responsibilities clarified myself.
Team Healthcare
55 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Observation Checklist and Data
Before conducting your observation please take a moment to familiarize yourself with this checklist so
that your observation will go smoothly.
Step 1: Complete the following demographics information
Observation Details
Name and title of person conducting
the observation
Date of observation
Time of observation began
Time checklist was completed
Titles of who is being observed
Step 2: Observe the subjects and complete this checklist during the observations.
**Note of important terms: EHR (Electronic Health Record) and Subject (the people under observation).
During your observation did the
nurses do the following:
Yes No Comments
Conduct a verbal hand off?
Meet with MD who told them it was ok
to discontinue orders?
If they did not update the orders or
meet with the MD ask follow up
questions:
a. Is it common or not common that you meet with the MD and get the ok to clean up orders?
b. How do you feel about not getting the ok from the MD to complete the orders?
c. How comfortable would you be cleaning up the orders without the MD’s approval?
Use the EHR to complete the verbal
pass down? Did you see them walk
to the computer and type in
information?
Use a computer in another area to
view the EHR? Please comment
where.
Where did they go use the computer?
Update the EHR after the hand off?
Show any concern through verbal or
nonverbal actions their frustration
with using the EHR?
Team Healthcare
56 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
57 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
58 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
59 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
60 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
61 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
62 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
63 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
64 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Informal Interview of Nurses Results
Below is the data the team collected from informal interviews conducted verbally with 6 Holy Medical Center nurses. Question Medical
Floor Nurse
Medical Floor Nurse
Medical Floor Nurse
ICU Nurse
ICU Nurse
ICU Nurse
Average
1.
On a scale of 1 to 5, 5 being really well and 1 being not well at all, how well does the transfer process, including the “computer” piece of it, work?
2 4 3 4 5 5 3.83
2.
On a scale of 1 to 5, 5 being really well and 1 being not well at all, when you receive a transfer patient to your unit, how well are the orders cleaned up in the chart (orders that are no longer relevant have been discontinued)?
3 2 1 3 2 2 2.17
3.
On a scale of 1 to 5, 5 being really well and 1 being not well at all, typically, how well does the verbal handoff process go between nurses (how thorough is it)?
4 3 4 5 4 3 3.83
4.
On a scale of 1 to 5, 5 being always and 1 being never, how often do you open the patient’s chart during the handoff process and review with the transferring nurse (either on the phone or in person)?
5 5 5 2 4 3 4
5.
On a scale of 1 to 5, 5 being I completely agree, and 1 being I do not agree at all, I understand my role in the orders management process (cleaning up orders) during a transfer.
2 4 4 3 3 1 2.83
Team Healthcare
65 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Results of Informal Poll with Nursing Directors and Clinical Educator
Below is the data collected from informal interviews conducted verbally with 2 nursing unit directors and the clinical educator at Holy Medical Center.
Question ICU Nursing Director
Medical Floor
Nursing Director
Clinical Educator
Average
1. On a scale of 1 to 5, 1 being not well at all and 5 being very well, how well does the transfer process work overall?
3 3 2 2.67
2. On a scale of 1 to 5, 1 being not well at all and 5 being really well, how well do the nurses perform the transfer handoff process?
3 3 3 3
3.
On a scale of 1 to 5, 1 being not well at all and 5 being really well, how well are the orders “cleaned up” in the patient chart when you receive a patient on your floor?
3 3 3 3
4. On a scale of 1 to 5, 1 being not well at all and 5 being really well, how well do the nurses complete the verbal portion of the handoff?
5 4 4 4.34
5. On a scale of 1 to 5, 1 being never and 5 being always, how often do the nurses open the electronic chart for review during the transfer process?
3 2 1.5 2.16
6. What percentage of the time do you think the nurses open the electronic patient chart during the transfer process (to review with the other nurse)?
50% 15% 15% 26.67%
Team Healthcare
66 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
ERIH Intervention Interviews
These interviews are not directly related to the performance gap, which is why they are not coded.
Follow-up Interviews (Intervention ERIH)
The purpose of these interviews is to help the team determine if the ERIH will eliminate the performance gap.
Questions Interviewee Response
Our team noted that there is an
ERIH area in the EHR at HC Regional
and the claims are that this holding
area eliminates the need for nurses
to do the orders management
process when they transfer a
patient from the ER to an inpatient
unit. Does the ERIH eliminate the
need for nurses to do orders
management during transfer?
Follow-up:
Based on what you have seen, does this eliminate any risk associated with possible human error during the transfer process (related to orders management)?
Do you know why Holy Medical Center chose not to include the ERIH in their system build?
Do you know if Holy Medical Center has the ability to add this functionality to their system?
Orderset Liaison Yes. I used to be the primary support person for the ER at HC Regional
and I used to work in the ER at HC Regional and having the ERIH makes
the orders just go away. The patient starts with new orders when they
transfer.
Yes – I am extremely familiar with the functionality and it literally dumps
the orders (they are obviously retrievable and a part of the permanent
record), but in the nurse’s mind, it is much more like the “old” paper
process. They are essentially turning the page.
No, I am not sure why they chose not to use it. My understanding is that
they had the option to use it, but chose not to.
I do not know at this point what it would take, we could research that and
find out.
Clinical Informatics
Specialist (assigned to
support the ER at HC
Regional Medical
Center)
Yes, that is accurate; utilizing the functionality of the ERIH allows the
orders to go away upon transfer or “admit” to an inpatient unit.
There is essentially no risk, as there are no orders to manage. The ER doc
talks to the inpatient doc and they coordinate – the inpatient doc puts in
new orders, but can still ‘view’ the ER orders, if necessary.
I don’t know for sure, but most things are political. I am thinking they
came up with a process they thought worked for them and went with
that.
I do not know – I am sure it is there in the background. I don’t know what
it would take to get it up and running.
Team Healthcare
67 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Clinical Educator
(involved in the ERIH
implementation at Holy
Medical Center)
From what I heard during the implementation, this is true. I have
observed at HC Regional and their process seems to go a lot more
smoothly than ours.
If it works the way I was told it would, the nurses would not have to do
orders management; thereby eliminating our problems. I completely
forgot about this.
I don’t remember – there were so many decisions to make and so many
variables, it is all a blur. I could probably find out though. It may take a
little digging!
I believe we do, but again, I would need to assess this and find out what
the associated costs are, etc.
Team Healthcare
68 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Appendix D. Data Synthesis
BEM Data Synthesis
Major trend is large data points that will be addressed by interventions
Minor trend is minor data points that should be noted or used for future
Information
Instrumentation
Motivation
Data 62 overall pieces of evidence that causes are
steaming from data issues in the environment. Data extracted from 11 interviews
Resources 24 overall pieces of evidence that causes are steaming
from resource issue in the environment. Data extracted from 11 interviews
Incentives 3 overall pieces of evidence that causes are
steaming from incentives issue in the environment. Data extracted from 11interviews
Environment
Major trends:
Inadequate information about roles and responsibilities of the process was found in 17 pieces of evidence extracted from 10 interviews
Evidence of trend found in interviews:
#of comments
Trainer D 2
Trainer P 2
RN Director 1 1
Nurse 2 3
Nurse 3 2
Physician Liaison 1
Orderset Coordinator
1
Clinical Educator 2
Charge nurse 1
Nurse 1 2
Trainers P comments: They can do some orders management over the phone with the doc, but it is really the job of the Doc, so we don’t go
Major trends: Lack of time to complete the EHR was found in 7 pieces of evidence extracted from 7 interviews
Evidence of trend found in interviews
#of comments
Trainer D 1
RN Director 1 1
RN Director 2 1
Nurse 2 1
Nurse 1 1
Nurse 3 1
Charge Nurse 1 RN Director 1 comment Time can be a barrier – pressure to get the patient transferred out. This is mostly related to the transfers that come to us from the Emergency Department. RN Director 2 Comment : What the nurses need is More time – not to feel so much pressure about time. Trainer Comments D: I don’t know why they chose to do that. It is a huge problem because the ER nurses don’t have time to clean up the orders, yet it is
Minor trend: No consequence for poor performance was found in 3 pieces of evidence extracted from 3 interviews
Evidence of trend found in interviews:
#of comments
Nurse 2 1
RN Director 1 1
RN Director 2 1 Nurse 2 comments: If I don’t clean up the orders then someone on the floor calls us, other times they just figure it out RN Director 2 comments: The process implemented is not the RN’s fault. We don’t punish people unless they are intentionally doing something that will harm the patient. RN Director 1 comments: If they don’t do the chart right, their peers will let them know The receiving nurse will call the transferring nurse and ask for clarification which takes time. If there is a real problem nurses will communicate to the
Team Healthcare
69 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
over it too much Trainer D comments: A lot of orders management falls on the Doc, so we don’t hit orders management too hard in the classes. RN Director 1 comments: Also, nurses don’t make changes to orders- that is supposed to be the role of the doctor. I think some nurses feel like that by deleting the orders they are practicing outside their scope Nurse 2 Comments: Drama around roles. Charge nurse is supposed to do the admit orders so it can be unclear about who is supposed to do the other pieces. Nurses, in theory should discontinue and complete orders but they don’t’ always do it. Nurse 2 comments: We don’t really have a system in place to check to see if the orders are taken a care of before they are transferred. Not sure if anyone knows whose job it is. It is not the MD’sjob in the system, they don’t have time. The nurses should be doing it but they aren’t, they may not know it is their responsibility. Nurse 3 comments: Nurses are supposed to clean up the orders. I don’t think it probably doesn’t always happen. It can be confusing as who is supposed to do what. Nurse 3 Comments. The process ( of who should clean up orders) hasn’t been explained; maybe it has but I have forgotten. Nurse 2 Comments: I would not say they are all comfortable with discontinuing orders. Physician Liaison comment: I think there is so
supposed to be their job. Nurse 2 Comments: The EHR has pretty much slowed everything down (note: nurse has little to no experience in computerized physician order) Nurse 3 Comments: I have quite a bit of experience with electronic chartings so I don’t think it has really changed the time it takes me. It may take a little more time but not much. Charge nurse comments: It is the nurses job, well I’m pretty sure, but it doesn’t get done like it should. I think some of them know, but don’t have the time and others are just not sure what is their job and what is the role of the doctor. Nurse 1 comments: It takes a lot more time – a ton more time! I hate it. It is not intuitive, like they said it would be Major trends: Lack of adequate job aids was found in 6pieces of evidence extracted from 6 interviews
Evidence of trend found in interviews
#of comments
RN Director 1 1
Trainer P 1
RN Director 2 1
Nurse 2 1
Nurse 3 1
Nurse 1 1 RN Director Comments: Job aids come from somewhere… not sure where and we are expected to get them to the nurses. They are difficult to understand
directors when we round the floors
Team Healthcare
70 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
much going on that no one is clear on what their role is. Orderset Coordinator Clinical Informatics comments: The nurses should be doing it, the actual in the computer piece. The obviously consult with the MD’s but they should be completing and discontinuing orders. Clinical Liaison comments: Nurses should be cleaning up the charts I suppose. Really I think the doctors should do it but everyone says that since they won’t it falls on the nurses. The nurses should clean up after they have consulted with providers what orders should and shouldn’t continue. Clinical Educator comments: Nurses should do the orders management. It can be tricky to know who should supposed to do what. Since I did the observations I realized although I said yes they are doing the orders management I should have said they are “kind of doing the process” Charge nurse comments: It is the nurses job, well I’m pretty sure, but it doesn’t get done like it should. I think some of them know, but don’t have the time and others are just not sure what is their job and what is the role of the doctor. Nurse 1 comments: I don’t discontinue stuff, I just know what he (the MD) wants me to do. I give the report of what has happened when I call the floor nurse; they can figure things out from there. Am I supposed to be cleaning up the orders? I suppose it might be my job, but I just worry about the ER orders and then I give info I have to the floor nurse.
and way too long. Trainer comments: There are job aids in Sharepoint. We do tell them in class how to access the job aids. There is also a Guide Me function in Cerner now. This is really high-level stuff from the parent company, but it is new and can be helpful. RN Director Comment: They need simple job aids with bullet points – do this, do this, do this. Not 15 page documents Trainer P comments: There are at least 100 job aids. Probably too much info for them there and I have been told hard to find exactly what they need. Nurse 2 Comments: I have no idea where the job aids are. I don’t really have time to refer to something that is that long. The job aids are long. Nurse 3 Comments: I know they showed us where they are but I don’t remember. Nurse 1 Comments: I have no idea where the job aids are. I supposed they told me, but I don’t know where to find them. I think they put them on a board or something. Not sure. Observations: 4 out of 4 nurses did not use job aids Minor trend: System not easy to use RN Director Comments: During the transfer process, the information is easier to see. The nurses can see everything that was done on the patient in the other unit.
Team Healthcare
71 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Trainer D Comments Nurses feel some of it is out of their scope of practice Trainer P Comment: Some of the more tenured nurses also feel like it is outside their scope of practice. I tell them not to think of the Orders tab as the “orders” tab – they look at that as physician-driven Major trends: Lack of adequate feedback was found in 10 pieces of evidence extracted from 9 interviews
Evidence of trend found in interviews
#of comments
RN Director 1
Observations 2
Trainer 1
Physician Liaison 1
Clinical Liaison 1
Charge Nurse 1
3 Nurses 3 Lack of feedback from Director to RN RN Director 2 comments We need to reiterate the message that it is okay (in reference to the earlier comment about it being the doctor’s job) to clean up the orders – we need a clean slate when the patients come to our floor. This helps everyone do their job better. Lack of feedback from Doctor to RN Observations 2 out 4 nurses did not talk to the provider during the orders management process
RN Director Comments: there is also SO much information on a chart. It is difficult to sort through all the information. Nurse 3 Comments: Finding information can be tricky but I have lots of experience so I am fine. Observations: Computers are not in the patients room must go somewhere else Nurse 2 comments: We don’t really have a system in place to check to see if the orders are taken a care of before they are transferred.
Team Healthcare
72 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Lack of communication between trainers and RN
Trainer Comment: I actually don’t know which site liaison there would help with RN issues. I am not sure they know who to contact there. I don’t think the liaisons understand their own roles. Lack of feedback to RN’s about charting audits
Physician Liaison Comments: Since most of our audits are informal, and more for the purpose of gathering information, I don’t do anything formalized with the information. My primary responsibility is to work with providers (physicians and mid-levels), and if there is something that is directly affecting them, it becomes my problem to solve. At this point, I just noted the information and may bring it up to colleagues if it seems to be an issue. Clinical Liaison Comments: I did the audit at your request. We did note the issue and will bring it up at IOT (a meeting that involves people from the corporate office as well as local leaders – they assign problems for resolution and report on those at this meeting). We have brought it up before, but the other location does not always have a representative at the meeting. Charge Nurse Comments: I do bring chart audit info up at huddle, but at this point I think I need the process and the responsibilities clarified myself. Nurse #1 Comment: Nurses If you don’t clean up the orders before you send the patient to the floor, what happens? Sometimes they call us to clarify, other times, nothing
Team Healthcare
73 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Nurse #2 Comment: If you don’t clean up the orders before you send the patient to the floor, what happens? It depends. Sometimes the floor calls us – other times they just figure it out Nurse #3 Comment: Nurses If you don’t clean up the orders before you send the patient to the floor, what happens? They seem to figure it out Major trends: Lack of information about support was found in 7 pieces of evidence extracted from 7 interviews This is also an information problem.
Evidence of trend found in interviews
#of comments
RN Director 1 2
Trainer P 1
RN Director 2 1
Nurse 2 1
Nurse 3 1
Nurse 1 1 RN Director Comment: We don’t know who to contact to implement changes or who to talk to about training. RN Director Comment: I don’t know what the relationship between the hospital and the CID
Team Healthcare
74 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
should be. I don’t even know who to contact for training support or questions Trainer Comment: I actually don’t know which site liaison there would help with RN issues. I am not sure they know who to contact there. I don’t think the liaisons understand their own roles. RN Director Comment: The only support we have now for training, other than the updates from corporate, are the Super Users that are on our floors. However, they have gone back to their jobs, so they are not receiving ongoing training in the system. Nurse 2 Comments: We would go to Shellie (someone local that does not have the primary responsibility for training). I may call them (Main Location) if I needed help but I don’t know how to get a hold of them. Nurse 3 Comments: I don’t think we have a formal help line or anything. I maybe call someone if there was a number for support. Nurse 1 Comments: If I had problem would I call the help desk? No, not really. The help desk? Not if it isn’t about the computer. Hmmm…. No, I would just ask the charge nurse for help. I don’t know if I really care.
Team Healthcare
75 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Knowledge 21 overall pieces of evidence that causes are
steaming from knowledge/skill issue from individual
Data extracted from 11 interviews
Capacity Motives 29 overall pieces of evidence that causes are steaming from motive issue from individual
Data extracted from 11 interviews
Individual
Major trends: Lack of adequate confidence to clean up orders .5 pieces of evidence extracted from 4 interviews
Evidence of trend found in interviews
#of comments
Trainer D 1
RN Director 1 1
Nurse 2 2
RN Director 2 1
RN Director 2 Comments: The training was all done in a “fake” environment. It was all theoretical. We were not able to use a live system and practice on real charts. Trainer D comments: There is just a lot of information and they were nervous about things. RN Director 1 Comments: A lot of people are tactile learners and will only learn by actually doing it they didn’t get this. Nurse 2 Comments: In training they skipped over this (the role of who does clean up) so not sure if it was very clear Nurse 2 Comments: I would not say they are all comfortable with discontinuing orders.
Major trends: ** This is a diffusion effect steam from a lack of information
Inadequate motivation by nurses to clean up the orders. 8 pieces of evidence extracted from 7 interviews
Evidence of trend found in interviews
#of comments
Trainer D 1
Trainer P 1
RN Director 1 1
RN Director 2 1
Nurse 1 1
Nurse 2 1
Nurse 3 1
Trainer D comments.. The why we discussed was that when there is an order it generates a task for the nurse. If you don’t clean up the orders, as a nurse, you are actually creating more work for yourself by there being more tasks generated. They understand the why, but I don’t know if they buy into it. Trainer P Comments: Nurses see Order Management as “not my problem”. The transferring nurse see that the receiving nurse will be responsible. RN Director 1 Comments: The RNs are mourning the loss of paper, particularly those that have been in practice for a long time, but the information is flowing and they understand the purpose. The purpose has been reached, but it does not mean that they are not still adjusting to the huge change.
Team Healthcare
76 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Minor trend: Lack of computer skills RN Director Comments: They are still great nurses, but they computer has thrown them off and made them question themselves. Some of the newer nurses are better at the computer piece of things, but they still need to build their skills in other areas. Trainer D Comments Some RNs are more comfortable with computers. Some of the more tenured nurses also feel like it is outside their scope of practice. RN Director 2 Comments: I think some of the RN’s do not feel confident in their computer skills Nurse 2 Comments: I am comfortable with the computer. But this is just a totally different system than I’m used to. Nurses Comments: I am a geriatric nurse. It was humiliating to be in class with these young kids just zipping around in the system. I felt like I was back in school. It was awful. I am figuring it out though and I don’t think that is the case for most nurses. I don’t really think it is that big of an issue for most of the nurses in my department. Mostly me! 4 Nursing Observations showed all nurses using the computer with proficiency
RN Director 2 Comments: it was a top down initiative, but the nurses do not care about the WHY. I don’t know that they fully understand the purpose RN Director Comments – sometimes they don’t see the need to do the orders management (when transferring). Nurse 2 Comments: We went to EHR because it was an initiative from the corporate office. To meet certain standards with the fed government they had to go electronic. Nurse 3 Comments: We went to EHR because it part of Obamacare. Nurse 1 Comments: It doesn’t really matter to me. They had to – we can’t go back. The government is telling them they have to. I understand why, and I know we can’t go back. They why doesn’t really matter, I just want to make sure my patients are taken care of. Major Trend: ORG ** This is a diffusion effect steaming from a lack of information and inadequate resource support from sister company. Distrust is coming from supervisor roles primarily Distrust between Main Location/Other Location or Between Departments (10 pieces of evidence)
Evidence of trend found in interviews
# of comments
Trainer D 2
RN Directors 7
Clinical Liaison 1
Team Healthcare
77 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Trainer D comment: We rarely get calls from the other location. The majority of them are system access requests (password help, etc.). the other location is typically dismissive and rude to the main location people anyway. They are not open to “help” from main location – it is like they have something to prove. They don’t listen when I try to help them RN Director comment: The system used to work for us, now we work for the system. Cerner designed the system for Evidence-Based Practice RN Director comment: I don’t know what the relationship between the hospital and the CID in he main location should be RN Director comment Another issue with the training was that the people doing the training were not nurses. They were non-clinicians, so their brains process differently. RN Director comment: It has been a very difficult
cultural change.
Clinical Liaison I don’t think they are doing it in the
other location like they should be. It is probably
something that was lost in translation during the
whole go-live process. We may need to go back and
clarify things to them.
RN Director comment: We can no longer make
changes based on our needs.
RN Director comment We can make requests, but
there are not guarantees as to if the changes will be
made or not
RN Director comment: The system is not bendable,
the culture has to bend to fit the system
Trainer comments: The problem is, in the other location, they chose to do a different process than main location on transfer from the Emergency
Team Healthcare
78 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Department to an Inpatient department
Team Healthcare
79 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Appendix E. Supporting Documents
Training Agenda
Team Healthcare
80 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
81 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
82 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
83 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
84 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
85 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
86 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
87 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Transfer Process Job Aid
Team Healthcare
88 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Team Healthcare
89 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Meeting OPWL Program’s Learning Outcomes
Conduct the HPT process in a way that is systematic
A systematic HPT process is a carefully planned, deliberate effort that uses appropriate methods of problem definition, data collection and data analysis to minimize the likelihood “errors and omissions”, that is, of not discovering vitally important information regarding a performance problem and/or misconstruing what evidence is gathered. Rummler and Brache (1995) state that organizations are systems but that many managers of these systems fail to recognize customers, product/service or workflow. As nascent HPT professionals, we do not want to make this same mistake!
Therefore, to insure our assessment will be as systematic as possible, the Team applied the principle of the “Seven Ps”, Proper Prior Planning Prevents Pitifully Poor Performance to our efforts. Specifically, before meeting with the client, the Team:
● selected a Needs Assessment model – the Behavior Engineering Model (Gilbert, 2007) with additional perspectives and concepts suggested by Marker (2007) and Langdon (2000);
● evaluated the worth of the project by consulting with both Chyung (2008) and Gilbert (2007);
● consulted with the client on the importance of the project to the hospital by employing the Lean principle of the 5 Whys;
● assessed if the problem was truly a performance problem by using the questions from Harless (1987);
● located prior research into problems associated with patient transfers with hospitals (aka “handoffs”), systemic medical errors, and medical provider communications;
● adopted a structured process of data collection as recommended by Schensul and LeCompte (2013) consisting of open-ended as well as focused questions using key terms gathered from the literature, such as “handoffs” between providers or departments. (Friesen, White & Bryers, 2008). By starting with open-ended questions we identified key issues and site-specific language which we incorporated into more narrowly focused interviews and observations during subsequent stages of data gathering.
● used deductive reasoning, as per Schensul and LeCompte (2013) to synthesize the data; and
● used cost and benefit analysis, as well as consulted with the client on intervention recommendations.
The team had to adjust the plan as the team noted specific trends in the data, but by having a plan in place the team was able to focus on the data and the assessment.
Team Healthcare
90 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Conduct the HPT process in a way that is systemic
Very early in the process, the team confirmed that the performance gap was not limited to a single person or shift of workers. For this reason, and consistent with HPT best practices, we quickly moved to study a larger context of peers, managers, processes and/or culture. Only by considering many inter-related factors in a systematic way can the root cause(s) of performance problems be found. As Tosti (n.d.) mentions, “[Determining] all the system factors that may impact the desired results [is the] KEY step in any improvement program”.
An overview of the transfer/admit process, as well as pre-existing conditions was also required since the client recently shifted from a very familiar paper-based medical chart to an Electronic Health Record (EHR). For these reasons, the team first explored the larger context of patient transfers, Electronic Health Records, and the directive that ER nurses manage the medical orders in the electronic system. The team uncovered numerous concerns related to information and incentives at the individual as well as environmental level. In other words, we uncovered evidence and possible causes of performance problems in four out of the six “cells” of the BEM.
Conduct the HPT process in a way that is consistent with established professional ethics
Team Healthcare conducted its on-site and preparatory efforts in a professional, ethical manner. We informed the client and all persons interviewed of our intentions and told all participants their involvement was voluntary. As suggested by the ISPI Code of Ethics (2002), the team masked individual identities to protect privacy as well as gather candid responses from hospital staff and managers. By doing this, the team was able to protect the client and keep their performance issues confidential.
The team requested to audit patient charts as well as observe nurses as they completed the transfer process. The audit would have allowed the team to understand what poor performance and exemplary performance would look like in the electronic record. Actually seeing the patient charts first-hand would have been optimal. Observations would have allowed the team to see nurses in action, actually performing the tasks related to transfer.
Due to HIPPA and privacy concerns voiced by the hospital staff, the team could neither collect nor see patient-specific information. Understanding that this request may put the client at risk, the team came up with alternate solutions that allowed the team to both keep the client’s trust and effectively complete the assessment. The ISPI Code of Ethics (2002) challenges professionals to show empathy for clients and their concerns; by formulating alternate solutions for our data collection, we honored this code.
Team Healthcare
91 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Conduct the HPT process in a way that is consistent with established professional standards
Team Healthcare’s effort has met all relevant standards established by the IPSI for a Needs Assessment effort. Specifically, we have:
not accepted the initial assertion that the client had a “training problem”;
“pushed back” on the above issue and created the necessary conditions to gather data from various actors and stakeholders; and
analyzed data to uncover possible causes of performance problems that would not be solved by additional training.
Because the team pushed back, the client was able to see alternative solutions to the performance problem. Through this assessment process, the team believes it added value to the client’s organization by offering solutions to help them move to where they needs to be in the future and focus on measured performance based on proven results (ISPI, 2002, p.1).
In support of the ISPI Code of Ethics (2002), Team Healthcare set clear expectations with the client about the systematic process we planned to follow, before proceeding with the investigation of the need. The team established the following boundaries with the client:
obtained official written permission from our client to conduct our study;
agreed to keep conceal the identity of the organization and interviewees;
agreed to keep the client informed of key findings before moving to each process; and
established a professional relationship with all parties involved when collecting data.
Furthermore, one team member that previously worked for the client removed herself from the data coding and synthesis process due to her fear that she may not be able to be objective in the process. Remaining objective is directly in line with the ISPI Code of Ethics (2002).
Align performance improvement solutions with strategic organizational goals
Our findings will help the client understand that solving the performance gap will require a multi-faceted solution well-beyond “more training”. The team aimed the solutions at filling the performance gap, which will help the hospital achieve their organizational goals. The primary concern of the hospital is patient safety and providing excellent patient care. The interventions selected by the team and the client align with these goals. The team believes if the hospital implements their solutions, they will be able to preserve patient safety, avoid expensive litigation due to medical errors, which will help them also fulfill their mission of “serving the poor and underserved” by preserving their resources for such.
Make recommendations that are designed to produce valued results
Our recommendations, summarized above, reflect a “sweet spot” between the risks, costs and benefits of doing nothing, contracting for more training, or calling for elaborate, costly organization development consulting. As per Gilbert (2007), it makes no sense to spend more money fixing a problem than is gained by even the most successful intervention.
Since the cost of a single legal settlement for malpractice, wrongful death or personal injury can easily top $1M dollars, the modest costs of the team’s recommendation are clearly cost-effective and consistent with the “no resources” available to close the performance gap.
Team Healthcare
92 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Collaborate effectively with others, in person and virtually
Team Healthcare members live in three different time zones and have collaborated across space and time by using tools such as Google hangout, Skype, conference calls, email and the Lotus Notes database.
The client and two members of the team are located in Boise Idaho. One team member is a former employee of the hospital and has acted as our single point of contact and tireless on-site representative.
The team determined that in order to verify a performance gap, it would be necessary to observe nurses in the process of a transfer. This proved logistically difficult since ER-to-Inpatient “transfers” do not happen that often. For this reason, the team prepared an observation form which the clinical educators used to make observations. This approach not only allowed the team to gather data, it built trust between the client and the team.
Communicate effectively in written, verbal, and visual forms
Team members have a diverse set of professional skills. We made deliberate efforts to couch our questions to clients in language consistent with a hospital setting, in general, and to this facility in particular.
Team members consistently interacted in ways intended to strengthen inter-team cooperation and communication.
Because only one of us was able to visit the hospital or interact with staff by telephone, the rest of us felt a duty to take on a much of the other work as possible. We kept in touch with each other via email almost daily and held conference calls at least once a week and often several times a week.
Use evidence-based practices
Team Healthcare collected mostly qualitative data from both unstructured and focused interviews and 3rd party observation. This raw data was coded and scored to enable some level of quantitative measurement of the problem space. For example, by counting key words and phrases used by respondents, we learned that the performance problem has deep roots in both the information and incentive columns of the BEM.
We employed data coding and analysis techniques recommended by Schensul and LeCompte (2013). Specifically, we collected data from interviews, 3rd party observations, and training documents, reviewed and color-coded them into BEM-based “cells”, then listed the exact words which justified each classification. We were them able to rank each of the BEM “cells” to identify those most represented tin the data gathered from various sources.
During the intervention selection, the team consulted several sources, both to guide the team in the selection process.
Team Healthcare
93 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
Contribute to the professional community of practice
If our client agrees, we will convert our project report into a “Tale from the Field” and submit it to PerformanceXpress with the hope that our efforts, however rudimentary, can help others facing similar problems.
In addition, we learned how valuable it is to gather data from multiple sources, job titles or organizational roles. Triangulation is a powerful tool that we will take our respective places of work.
We also learned how important it is to consider the cost of as intervention as well as its appropriateness. It makes no sense to spend $1 to fix a 10 cent problem!
Team Healthcare
94 © OPWL 2013. Do not distribute without written permission from the OPWL department at Boise State University.
References
Chyung, S. Y. (2008). Foundations of instructional and performance technology. Amherst, Mass: HRD
Press.
Friesen, M..A., White, S.V., and Byers, J.F. (2008) Handoffs: Implications for nurses. In Patient safety and
quality - An evidence-based handbook for nurses. Retrieved from:
http://www.ahrq.gov/professionals/cliniciansproviders/resources/nursing/resources/nurseshdb
k/index.html
Gilbert, T. (2007). The behavior engineering model. In Human competence: Engineering worthy
performance (tribute edition) (pp. 73-107). San Francisco, CA: Pfeiffer.
Harless, J. H. (1987). An analysis of front-end analysis. Performance + Instruction, 26 (2), pp. 7-9.
ISPI. (2002). Code of Ethics. Retrieved from http://www.ispi.org/pl/About/Code-of-Ethics.pdf
Langdon, D. (2000). Introduction to Performance Alignment in Business. In Aligning performance:
Improving people, systems, and organizations (pp. 1-15). San Francisco: Jossey-Bass/Pfeiffer.
LeCompte, M., & Schensul, J. (2013). Analysis from the top down. In Analyzing & interpreting
ethnographic data, 5, pp. 45-66. Walnut Creek, CA: AltaMira Press.
Marker, A. (2007). Synchronized analysis model (SAM): Linking Gilbert's behavioral engineering model
with environmental analysis model. Performance Improvement, 46(1), 26-32.
Rummler, G., & Brache, A. (1995). Viewing organizations as systems. In Improving performance: How to
manage the white space on the organization chart (pp. 5-14). San Francisco: John Wiley & Sons,
Inc.
Tosti, D. T. (n.d.). The performance engineering model. Retrieved from:
http://www.donaldtosti.com/The_performance_engineering_model.docx