BMJ OpenComplete List of Authors: Wesbrook, Johanna; Australian Institute of Health Innovation,...

For peer review only

A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication

management system to reduce medication errors, adverse drug events, and average length of stay at two paediatric

hospitals

Journal: BMJ Open

Manuscript ID bmjopen-2016-011811

Article Type: Protocol

Date Submitted by the Author: 07-Mar-2016

Complete List of Authors: Wesbrook, Johanna; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Li, Ling; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Raban, Magdalena; Macquarie University, Centre for Health Systems and

Safety Research, Australian Institute of Health Innovation Baysari, Melissa; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Mumford, Virginia; Macquarie University, Australian Institute of Health Innovation Prgomet, Mirela; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Lake, Rebecca; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University McCullagh, Cheryl; The Sydney Childrens Hospitals Network Dalla Pozza, Luciano; The Sydney Childrens Hospitals Network Karnon, Jon; The University of Adelaide

O'Brien, Tracey; The Sydney Childrens Hospitals Network Ambler, Geoff; The Children’s Hospital at Westmead, Institute of Endocrinology and Diabetes Day, Richard; St Vincents Hospital Sydney and UNSW, Clinical Pharmacology; University of New South Wales, Pharmacology Cowell, Christopher; University of Sydney, Sydney Medical School; Sydney Children's Hospitals Network, Institute of Endocrinology and Diabetes, Children’s Hospital at Westmead Gazarian, Madlen; University of New South Wales, School of Women's and Children's Health; Sydney Children's Hospital, Department of Immunology and Infectious Diseases Worthington, Rachael; The Sydney Childrens Hospitals Network

Lehmann, Christoph; Vanderbilt University White, Les; NSW Health , Office of Kids and Families Barbaric, Draga; The Sydney Childrens Hospitals Network Gardo, Alan; The Sydney Childrens Hospitals Network Kelly, Margaret; NSW Health , Office of Kids and Families Kennedy, Peter; New South Wales Ministry of Health, eHealth

<b>Primary Subject Heading</b>:

Health informatics

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on D

ecember 26, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2016-011811 on 21 O

ctober 2016. Dow

nloaded from

http://bmjopen.bmj.com/


Secondary Subject Heading: Paediatrics

Keywords: PAEDIATRICS, eMM Systems, Medication errors, Adverse drug events, Electronic Prescribing

Page 1 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem

ber 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2016-011811 on 21 October 2016. D

ownloaded from



Page 1

Version 2 – 28 March 2016

A stepped-wedge cluster randomised controlled trial to assess the effectiveness

of an electronic medication management system to reduce medication errors,

adverse drug events, and average length of stay at two paediatric hospitals

Westbrook JI1, Li L

1, Raban MZ

1, Baysari MT

1, Mumford V

1, Prgomet M

1, Georgiou A

1, Kim T

1,

Lake R1, McCullagh C

2, Dalla-Pozza L

2, Karnon J

4, O’Brien TA

2, Ambler G

3, Day R

5, Cowell CT

2,

Gazarian M5

,Worthington R2, Lehmann CU

6, White L

7, Barbaric D

2, Gardo A

2, Kelly M

7, Kennedy

P 8

1Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,

Macquarie University, Sydney, Australia, 2The Sydney Children’s Hospitals Network,

3The Sydney

Children’s Hospitals Network and The University of Sydney, 4University of Adelaide,

5School of

Medical Sciences, Faculty of Medicine, University of New South Wales, 6Vanderbilt University,

USA, 7Office of Kids and Families NSW Health,

8eHealth NSW Health Ministry

Corresponding Author: Professor Johanna Westbrook, Centre for Health Systems and Safety

Research, Australian Institute of Health Innovation, Macquarie University, level 6, 75 Talavera Rd,

Macquarie Park, 2109 [email protected] (+61 2) 9850 2402

Keywords: Paediatrics; eMM systems; medication errors; adverse drug events (ADEs); electronic

prescribing

Word count: 4225

Page 2 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 2

ABSTRACT

Introduction Medication errors are the most frequent cause of preventable harm in hospitals.

Medication management in paediatric patients is particularly complex and consequently potential

for harms are greater than in adults. Electronic medication management (eMM) systems are

heralded as a highly effective intervention to reduce adverse drug events (ADEs), yet internationally

evidence of their effectiveness in paediatric populations is limited. This study will assess the

effectiveness of an eMM system to reduce medication errors, ADEs, and length of stay (LOS). The

study will also investigate system impact on clinical work processes.

Methods and Analysis A stepped-wedge cluster randomised control trial (SWCRCT) will measure

changes pre and post eMM system implementation in prescribing and medication administration

error (MAE) rates, potential and actual ADEs, and average ward LOS. In stage 1, 8 wards within

the first paediatric hospital will be randomised to receive the eMM system one week apart. In stage

2, the second paediatric hospital will randomise implementation of a modified eMM and outcomes

will be assessed.

Prescribing errors will be identified through record reviews, and MAEs through direct observation

of nurses and record reviews. Actual and potential severity will be assigned. Outcomes will be

assessed at the patient-level using mixed models, taking into account correlation of admissions

within wards and multiple admissions for the same patient, with adjustment for potential

confounders. Interviews and direct observation of clinicians will investigate the effects of the

system on workflow. Data from site 1 will be used to develop improvements in the eMM and

implemented at site 2, where the SWCRCT design repeated (stage 2).

Ethics and Dissemination The research has been approved by the Human Research Ethics

Committee of the Sydney Children’s Hospitals Network and Macquarie University. Results will be

reported through academic journals and seminar and conference presentations.

Page 3 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 3

Strengths

• The Stepped-Wedge Cluster Randomised Controlled Trial (SWCRCT) study design is the

closest to the gold-standard of an RCT, and has rarely been applied to study the effects of

information technology in healthcare.

• Few previous studies have investigated the impact of electronic systems on medication

administration error rates. We present a novel approach to conduct direct observation of this

process using the Precise Observation System for Safe Use of Medicines (POSSUM).

POSSUM allows observers to quickly and accurately record drug information e.g. name,

strength, compliance with procedures f as well as the number and length of interruptions and

multi-tasking.

• We will assess potential harm from medication errors identified and importantly also

measure actual harm to children.

Limitations

• Direct, close observation lends itself to the “Hawthorne effect” whereby participants may

seek to ‘improve’ their performance. This may result in an underestimatione of the ‘true’

medication administration error rate.

Page 4 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 4

INTRODUCTION

Prescribing, administering, and monitoring medicines in children is highly complex. Compared

with adults, medication errors in children are three times more likely to result in harm.(1) Paediatric

patients are at particular risk of certain errors, such as tenfold dosing errors, facilitated by mistakes

in dose calculation, poor documentation of decimal points and confusion with the use of zeros.

Some errors have led to death.(2-4) The complex medication decision process in children often

requires information about age, weight, dosing ranges and off-label use of medicines.(2) Systematic

reviews show errors occur in 5-27% of all medication orders for children.(5-7) Prescribing errors

account for 3-37% of errors, dispensing 5-58%, administration 72-75%, and documentation 17-

21%, although accuracy of estimates is affected by the great variability in definitions and

measurement methods used.(6) There is much less research about the harms associated with

medicine use. One Australian study found 19.2% of paediatric inpatients experienced adverse drug

events (ADEs) and estimated 12.3% were potential ADEs, 7.0% were actual ADEs, and 3.6% were

preventable.(8)

The frequency and severity of medication administration errors (MAEs) in Australian adult

hospitals is a cause for concern.(9, 10) In 4,271 medications administered to 720 patients, 74.4%

were found to have at least one procedural failure (e.g. failure to check a patient’s identification).(9,

10) One in four had a clinical error (e.g. wrong dose). Of intravenous drug administrations, 70%

had one clinical error, of which 25.5% were judged to be serious and likely to cause permanent

harm.(9) MAEs among children are rarely studied.(6, 11) A major barrier to MAE research is the

methodological challenges. MAEs cannot be accurately detected from retrospective record reviews;

they require direct observation of nurses administering medications to patients.

Can information technology reduce medication errors?

Electronic medication management (eMM) systems are expected to reduce medication errors and

ADEs significantly. However, rigorous evidence demonstrating these effects is limited.(7) A

systematic review identified eight studies of eMM effectiveness among paediatric patients. Meta-

analysis showed a significant reduction in prescribing error risk (RR 0.08) but not in ADEs or

mortality.(12) There are no Australian studies of eMM system use in a paediatric setting. Previous

studies have often relied upon incident reports to measure error rates, which are generally

unreliable.(13) Only one study(14) of paediatric inpatients has used a control group to assess eMM

effectiveness. Internationally there is currently insufficient evidence to demonstrate clinical benefit

from eMM in paediatric patients. This view was confirmed by a policy review for the American

Academy of Pediatrics(15) which called for the demonstrable enhancement of eMM systems to

Page 5 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 5

better meet the specific needs of paediatrics to ensure their safety and effectiveness. Further,

existing eMM systems used in paediatrics have been found lacking in functionalities required for

safe prescribing, dispensing and administration of medications.(16, 17)

In 2005, Han et al(18) reported a significant increase in the mortality rate among critical care

children at a US paediatric hospital following introduction of a commercial eMM system. The rapid

implementation process and limited attention to the significant workflow re-design required were

considered major factors in this outcome. Subsequent studies(19) have shown no increase in

mortality rates. However, the Han et al(18) study caused considerable alarm and served to

demonstrate the substantial dangers of poor implementation and the importance of monitoring

outcomes following system implementation and responding to problems identified.(20)

eMM use in adult hospitals, while highly effective at reducing medication errors,(21) also

introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post eMM in

two adult Australian hospitals found 42.4% were facilitated by the system (78 per 100

admissions).(22) The most frequent mechanism was incorrect selection from a drop-down menu.

Results from that study were used to instigate changes to eMM software and to inform changes to

user training programs. The study was one of the first to quantify the rate of ‘system-related’ errors

and produced an innovative dual classification for categorising both the manifestations and

mechanisms of these system-related errors.(22) The nature and magnitude of such potential new

risks have rarely been studied in paediatrics.

Why evaluate eHealth systems

eHealth systems are having an increasing impact on the delivery of health care services, yet, despite

their widespread effects and vast cost, they are rarely subject to rigorous research.(23) This limited

evidence-base significantly hinders improvements and innovation in the design, implementation and

use of health information technology (IT) systems. Internationally the need for IT evaluation studies

to employ more robust designs and sophisticated analyses is well recognised.(24) Excessive

reliance has been placed upon uncontrolled before and after, and qualitative studies. Evidence of

eHealth system effectiveness and safety is crucial to facilitate policy-makers’ and health care

organisations’ informed decisions about investments and prioritisation of health IT systems.

This project presents an innovative and comprehensive program to assess eMM system

effectiveness in reducing medication errors, ADEs and length of stay (LOS) and rapidly deploying

new knowledge into practice for subsequent implementation.

Page 6 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 6

The project aims are:

Aim 1 To quantify the safety and effectiveness of an eMM system to reduce medication

errors, ADEs and average LOS among paediatric patients using a stepped-wedge

cluster randomised controlled trial (SWCRCT) design;

Aim 2 To assess the effects of the eMM on clinicians’ workflow and efficiency; and

Aim 3 To assess the extent to which feedback of study results and subsequent

modifications to the eMM design and associated work practices can improve eMM

effectiveness in reducing medication errors at a subsequent implementation tested

via a second SWCRCT.

METHODS AND ANALYSIS

Research Plan

Aim 1: Quantify the safety and effectiveness of an eMM system to reduce medication errors

(potential and actual ADEs), and average LOS among paediatric patients.

Study Design and Setting: We will conduct a SWCRCT to measure changes in prescribing and

medication administration errors (MAEs) which result in potential and actual ADEs, along with

changes in average ward LOS, pre and post eMM. Cluster randomised controlled trials (CRCT) are

ideally suited to test interventions where individual patient randomisation is not possible. CRCTs

commonly use a parallel group design, in which the clusters are randomised to either the

intervention or control arm of the study. It is often regarded as unethical to withhold an intervention

from a proportion of participants if it is believed that the intervention will do more good than harm.

The SWCRCT design, where the intervention is delivered sequentially to all trial clusters over a

number of time periods, is an alternative to the traditional parallel groups design. The order in

which the clusters (wards) receive the intervention is randomised, and by the end of the study all

clusters will have adopted the intervention.(25) The steps represent the predetermined periods

when data relating to each of the clusters will be collected. This design is the closest to the gold-

standard of an RCT, when such a design is not possible. The stepped-wedge design offers particular

strengths in allowing the modelling of the effect of time on the effectiveness of the intervention. In

stage 1 a SWCRCT will be conducted at site 1, and in stage 2 a SWCRCT will be conducted at site

2.

The study setting is the Sydney Children’s Hospitals Network which incorporates the two tertiary

paediatric hospitals in Sydney, The Children’s Hospital at Westmead (CHW; site 1) and Sydney

Page 7 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 7

Children’s Hospital, Randwick (SCH; site 2). The network provides care for 51,000 inpatient

admissions, 92,000 emergency department presentations and over 1,000,000 outpatient services

events each year. Both hospitals provide a complex and comprehensive range of services caring for

seriously ill and injured children and adolescents across the state of New South Wales and beyond.

Randomisation: In stage 1, 8 wards within site 1 will be randomised to receive the eMM system.

Randomisation will be conducted by a person blinded to ward identity. At baseline all 8 wards are

in the control period (Figure 1). At step 1, the eMM system will be implemented in the first ward.

The eMM system will then be implemented to a new ward in sequence weekly. By the end of step

8, all wards will have the eMM system at site 1.

<Insert Figure 1 here>

Medication error and ADE definitions: Medication errors are defined as any error in the

prescribing, supply, preparation, administration, or monitoring of a medication, regardless of

whether such errors lead to adverse consequences. In this study we will not measure dispensing

errors. ADEs are defined as harm or injury as a consequence of the use or non-use of medicines.(8)

Medication errors may result in actual ADEs or potential ADEs. For example, a medication error

may occur but is intercepted prior to administration thus preventing harm to the patient.

In this study we will be seeking to identify medication errors, and to determine those that resulted in

harm (actual ADEs) or potential harm (potential ADEs). Figure 2 illustrates the medication error

and ADE classification processes for this study.


Data Collection: Medication error data collection occurs at baseline and each stage, (i.e. in every

subsequent week as eMM implementation occurs and for two additional weeks after full

implementation). For the primary study objective of determining eMM effectiveness to reduce

medication errors and ADEs we will collect data at 11 points on all wards (baseline and at each step

including two weeks after full implementation, Figure 1). This will allow us to measure changes pre

and post eMM system introduction in: 1) Prescribing error rates per order and per admission by type

and severity (potential and actual ADEs); 2) MAEs per order and per admission by type and

severity (potential and actual ADEs). For the secondary outcome of changes in LOS we will obtain

data for a further 21 steps in the follow-up period to provide greater statistical power. As these are

routine administrative data, no additional data collection is required.

Page 8 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 8

Prescribing error and ADE detection: A review of medication charts at baseline and each step will

be conducted complying with a standard error protocol.(8-10, 21) This protocol will be extended to

develop methods for determining the rate at which errors are detected and intercepted by staff,

actions taken, and any harm experienced.

Medication administration error (MAE) and ADE detection: For the MAE study, data will be

collected using direct observation. Nurses will be observed preparing and administering

medications. In our previous studies using this approach in adult hospitals,(10) over 80% of nurses

consented to participate and we expect similar rates for this study. Direct observations will be

supported by an innovative data collection tool, the Precise Observation System for Safe Use of

Medicines (POSSUM, Figure 3).(10, 26) The POSSUM tool allows observers to quickly and

accurately record drug information e.g. name, strength, and dose. The POSSUM tool also allows

collection of the number and length of interruptions experienced and multi-tasking (e.g. answering a

question while also selecting medicines). Nurses’ compliance with core procedures, such as

checking a patient’s identification, will also be recorded. Comparing observational data with

patients’ medical records (via retrospective audit) will enable identification of the number, types

and severity of MAEs.


Observers will have a pre-allocated observation period to ensure coverage across the day and the

week.(9, 10) Observers will follow a “serious error” protocol i.e. they must intervene if they witness

an administration that is potentially dangerous to the patient. Observers will not have access to

patients’ medication charts and will record only what they observe. Thus, most MAEs will not be

identifiable until chart review. Past inter-rater reliability tests showed kappa scores from 0.94 to

0.96 following training in the use of POSSUM.(9, 10)

Direct, close observation lends itself to the “Hawthorne effect” whereby participants may seek to

‘improve’ their performance. If nurses change their practices, and are more careful when observed,

this will lead to an underestimation of the ‘true’ MAE rate. This bias would be present both pre and

post eMM. Our prior research suggests the likelihood of sustained change on busy wards is low.(27,

28)

Prescribing error, MAE and ADE classifications: Prescribing errors and MAEs will be classified

into: (i) procedural errors (for prescribing errors these include illegible order, illegal order,

Page 9 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 9

incomplete order; for MAEs these include failure to follow correct patient identification process,

compliance with double-checking requirements, etc.); and (ii) clinical errors (e.g. wrong dose,

wrong drug, wrong route, wrong strength) using previously applied classifications.(10, 21) .

Once an error has been identified, a rating of the potential severity of that error will be made.

Subsequently, records will be reviewed for evidence of error detection and interception, and for any

actual harm to the patient. Thus, medication errors which occur will receive both a ‘potential’ harm

rating and an ‘actual harm rating’ (Figure 2). As most previous medication error studies do not

assess actual harm, this double classification process will allow us to compare our findings with

previous studies, as well as allow an assessment of the accuracy of such approaches compared to

estimating the actual harm from medication errors.

Evidence of harm as a consequence of a medication error will be identified through a

comprehensive review of patients’ medical records. This clinical review process will be assisted by

the provision of specific harm identification guides for reviewers which will identify, for specific

drugs and error types, the types of evidence which would suggest harm had occurred following the

medication error. Figure 4 presents an example of one of the harm identification guides to be used.


Experienced clinicians will abstract data from medical records using a structured data collection

form and the harm identification guides. A multi-disciplinary clinical review panel will re-assess a

minimum 5% sample of the records and will also review any records which reviewers identify as

particularly complex. Panel members will be blind to the location, and whether data were generated

pre or post eMM. Panel members will not know the ward order of rollout and specific dates when

each ward became an intervention ward with the stepped wedge design and therefore blinding of

pre and post data will be possible. Actual and potential severity will be assigned using the National

Coordinating Council for Medication Error Reporting and Prevention (NCC-MERP) scale for

adverse event outcomes (29) and the 5-point Severity Assessment Code (SAC) Scale,(30) as used in

our past research.(9, 10, 21) This will allow comparison with a greater number of previous studies.

System-related errors: We will apply our two dimensional classification, modified to incorporate

recent recommendations in this area,(20) to assess whether medication errors post eMM were

facilitated by eMM design, i.e. are ‘system-related’. This process identifies the manifestation (e.g.

wrong dose) and mechanisms (e.g. incorrect menu selection). These results will be used to provide

Page 10 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 10

recommendations about IT design and user training.(22) Any changes to the eMM design features,

training or work processes during the study will be documented.

Sample sizes and analyses: Sample size calculations have taken into account the estimated

between-cluster variance, i.e. between wards variance, and the design effect associated with the

stepped-wedge design.(31) Calculations were based on our previous studies in adult hospitals and

hospital data from the paediatric sites. Each ward has on average 14 admissions per week with an

average LOS 3.78 days (SD=7.39) with seven medications per admission.

Prescribing errors: Based on our previous studies(21) the expected reduction in overall prescribing

error rate is 60%, from 4.06 errors per admission (SD=5.27) to 1.62 (SD=2.87) with an estimated

intraclass correlation coefficient (ICC) of 0.06 (Table 1). The number of wards required to detect a

60% change for two-sided tests (80% power; �<5%) is one, with 10 data collection steps after

baseline. For ADEs, the required number of wards is seven to detect a 60% reduction (Table 1). To

be conservative and provide greater power, we will collect data on all eight wards allowing

detection of a minimum change of 20% for overall errors and 42% for ADEs. At each step records

for 112 patient admissions will be reviewed, totalling 1,232 across the study.

MAEs: Based on our previous studies(9, 10) we expect the overall MAE rate per administration to

fall by 27%, from 0.37 (SD=0.65) to 0.27 (SD=0.52) with an estimated ICC of 0.03 (Table 1). The

required number of wards (two-sided test; 80% power; �<5%) is seven, with 10 steps after baseline.

For ADEs, the required number of wards is six. We will collect data using all eight wards which

will allow detection of a minimum 20% change overall and 48% for ADEs. At each step we will

observe at least 240 medication administrations, totalling 2,640 across the study.

LOS: There are very limited data on the impact of eMM systems on LOS at ward level. A study in

an ICU showed a 23% reduction in LOS post eMM system.(32) To detect a 23% reduction in LOS,

i.e. from 3.78 (SD=7.39) to 2.92 days, with eight wards, will require additional (routinely collected

LOS) data in a total of 31 steps for a two-sided test with 80% power �<5%.

Table 1 Prescribing error and MAE power calculations

ER Pre-

(SD)

ER Post

(SD)

% changes

from past

eMM

study

ICC

(7)

Avg. # of

Adm or

Admins/

study step

# of

steps

(k)

Min.

#

wards

req.

# of

wards

Min. %

change

detectable

Max.

power

Presc.

Errorsβ

4.1 (5.3) 1.6 (2.9) 60% 0.06 14 10 1 8 20% 100%

Page 11 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 11

Data analyses: Medication error rates per order, stratified by error type, study step, and ward will

be calculated. For each outcome of interest, data collected across all measurement periods and all

study steps will be used in the analyses comparing intervention status (pre versus post eMM).

Analyses will apply the intention-to-treat principle. Patient data will be analysed according to the

status of the wards (i.e. pre or post eMM) where patients were admitted. Outcomes will be assessed

at the patient-level using mixed models, taking into account correlation of patient admissions within

wards (clusters) and multiple admissions for the same patient, with adjustment for potential

confounding factors. For the MAE analyses we will adjust for contextual factors including

interruptions, multi-tasking, nurse age, gender, and adherence to policies. For LOS analysis we will

adjust for patient characteristics, such as major diagnoses, comorbidity, age, and gender. The mixed

models will incorporate fixed terms for ward intervention status, measurement time steps (including

baseline) and other confounders. The analyses will include multiple time points pre and post eMM

implementation. The study design will allow us to determine temporal changes in system

effectiveness, e.g. to determine if error rates continue to decline over time. We will apply the

‘system-related’ error classification (22) to identify system-related error rate and associated

mechanisms.

Aim 2: To assess the effects of the eMM on workflow and efficiency

Study design and sample: Observations and interviews will be held with medical, nursing, and

pharmacy staff at baseline to allow mapping of core work processes associated with medication

provision. At each step in the stepped-wedge design, a small number of interviews will be

conducted with nursing and medical staff to gain insights into clinical staff perceptions of the

impact of the system on workflow, efficiency and care delivery. Purposive sampling will continue

until theme saturation. Trustworthiness of the qualitative data will be achieved through triangulation

of data and investigators, engagement with the field with a documented audit trail and member

checking.(33)

The eMM is anticipated to have a significant impact on the work of hospital pharmacists. We will

conduct a direct observational study of approximately eight pharmacists at site 1. We will observe

them for 200 hours between 7:30 – 18:00 pre and post eMM system implementation to examine

ADEsβ 0.3 (0.7) 0.1 (0.4) 44% 0.005 14 10 7 8 42% 83%

MAEs¥ 0.4 (0.6) 0.3 (0.5) 27% 0.03 30 10 4 8 20% 97%

ADEs~ 4.2% 1.8% 57% 0.003 30 10 6 8 48% 93%

β Per admission; ER = error rate;

~ % all medication administrations;

# =number; Adm =

admissions; Admins=administrations; req= required; Min=minimum,¥

per admin.

Page 12 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 12

changes in i) task time distributions ii) location of work and iii) communication patterns. Using the

validated WOMBAT approach(26, 34) multiple dimensions of work will be captured (e.g. tasks

performed, with whom, with what, location, interruptions, and multi-tasking). On data entry, tasks

are automatically time stamped when entered in the WOMBAT data collection tool. Figure 5 is an

example of data collection within WOMBAT. An additional sample of 140 hours of observation

will be conducted to capture the work of oncology pharmacists whose work involves supporting the

delivery of complex drug regimens to children with cancer.


Data generated will allow changes in task time distributions and sequencing of work to be

determined. These data will be examined in relation to changes in outcome indicators generated

(from Aim 1, e.g. medication error rates, LOS) on the same wards, along with additional quality

process indicators (routinely collected) pre and post the eMM system. These will include time to

antibiotic treatment for children presenting with fever and neutropenia, a demonstrated indicator of

patient outcome,(35) and data on protocol compliance obtained from audits conducted by the

hospital. The research team have extensive experience in analysis of such work measurement

data.(28)

Aim 3: Assess the extent to which feedback (from Aims 1 and 2) and subsequent modifications to

an eMM system design can improve eMM system effectiveness in reducing medication errors

Evaluations of health IT serve multiple purposes, ranging from providing an objective assessment

of the success of the new technology in delivering anticipated benefits, to identification of deficits

in the system, their source, and the ways they can be addressed. This is critical to improving system

effectiveness, relevance, and responsiveness. For Aim 3, the findings of the SWCRCT at CHW (i.e.

stage 1) will be reported to the Project Evaluation Committee (PEC) made up of members of the

research team, and the Hospitals’ eMM system Project Steering Committee. The PEC will meet

every month to consider the implications of study findings across a number of domains including

the system’s technical features (e.g. compatibility with other hospital systems), effectiveness (e.g.

error reduction and system-related errors); professional attitudes (e.g. satisfaction) and

organisational features (e.g. work processes), as a means of formulating changes to eMM system

design features and user training.

This will form the key component of an action-oriented approach aimed at optimising system

Page 13 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 13

performance leading to an enhanced eMM system which will then be implemented across the

second site, SCH (stage 2). The SWCRCT design will be repeated at site 2 using the same methods

as above. Power calculations for stage 2 will be based on results from stage 1. We will conduct

separate analyses for all outcomes specified. Results for the two sites will be compared, using

multilevel and longitudinal analysis approaches to determine changes in error rates (taking baseline

data into account).

Expected outcomes and significance of the research project

This project will generate the first Australian data, in a paediatric setting, on the effectiveness of

eMM systems to reduce medication errors and ADEs, and provide an assessment of how systems

impact on the work of clinicians and the consequences for the delivery of care to children.

Importantly, the findings will be directly applied to enhance the eMM system design, and work

processes and then tested further through evaluation of the enhanced eMM system at a second

paediatric hospital. These results will be particularly valuable for other paediatric hospitals yet to

commence implementations. Exploiting the SWCRCT design within an action-research model is

highly innovative, and will deliver high quality data on system effectiveness. Such a model of

formally integrating health IT assessment results as a basis for active engagement with IT vendors

and clinicians to bring about system change has both national and international significance. The

study advances explicit methods for the systematic identification of harm associated with

medication errors. The data generated will also provide the basis for a robust cost-effectiveness

analysis, which will be the subject of a separate protocol.

ETHICS AND DISSEMINATION

The research has been approved by the Human Research Ethics Committee of the Sydney

Children’s Hospitals Network (HREC/15/SCHN/370). In the first instance, results from site 1 will

be reported to the PEC so that they can be used to inform eMM system and work process design

prior to implementation at site 2. Results will also be reported through academic journals and

conference presentations. The project is funded through a National Health and Medical Research

Council Partnership Grant. As such, the project team includes academic researchers, hospital

clinicians and experts involved in the implementation of the eMM system at the two hospital sites,

along with senior policy makers from agencies within the State Health Department involved in

eHealth system strategy and policy. This provides the project with access to a range of other

conduits through which to disseminate results to, for example, policy makers and system

implementers.

Page 14 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 14

Page 15 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 15

Authors Contributions: JW, LL, MB, AGe and CM initiated the project and led the development

of the NHMRC grant proposal. JW, LL, MB, AGe, RD, JK, LDP, CTC, GA, and TOB are chief

investigators on the project and all made contributions to the protocol in their specific areas of

expertise. CM, LW, NB, AGa, CUL, MG, PK, MK, AB and DB are associate investigators on the

NHMRC grant and provided input to the protocol, particularly in the areas of paediatric clinical

practice and broader eHealth strategy in relation to eMM systems. MZR, MP, VM, TK, RW, RL are

members of the project team and have made significant contributions to the protocol in terms of the

design of details regarding the collection and classification of medication errors and harm. JW

prepared the first draft of this manuscript based upon the grant proposal and all authors have

reviewed and provided input.

Funding Statement: The project is supported by a National Health and Medical Research Council

Partnership Grant (APP1094878) in partnership with: Sydney Children’s Hospitals Network;

eHealth New South Wales; Office of Kids and Families, New South Wales.

Competing Interests Statement: None to declare.

Page 16 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 16

References:

1. Kaushal R, Barker KN, Bates DW. How can information technology improve patient safety

and reduce medication errors in children's health care? Archives of Pediatrics & Adolescent

Medicine. 2001;155(9):1002-7.

2. Doherty C, Mc Donnell C. Tenfold Medication Errors: 5 Years’ Experience at a University-

Affiliated Pediatric Hospital. Pediatrics. 2012;129(5):916-24.

3. Gazarian M, Drew A, Bennett A. Medical mishap. Intravenous paracetamol in paediatrics:

cause for caution Australian Prescriber. 2014;37(1):24-5.

4. Kim GR, Chen AR, Arceci RJ, Mitchell SH, Kokoszka KM, Daniel D, et al. Error reduction

in pediatric chemotherapy: computerized order entry and failure modes and effects analysis.

Archives of pediatrics & adolescent medicine. 2006;160(5):495.

5. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, et al. Medication

errors and adverse drug events in pediatric inpatients. J Amer Med Assoc. 2001;285(16):2114-20.

6. Miller M, Robinson K, Lubomski L, Rinke M, Pronovost P. Medication errors in paediatric

care: a systematic review of epidemiology and an evaluation of evidence supporting reduction

strategy recommendations. Quality and Safety in Health Care. 2007;16(2):116-26.

7. Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to

Reduce Pediatric Medication Errors: A Systematic Review. Pediatrics. 2014.

8. Gazarian M, Graudins LV. Long-term Reduction in Adverse Drug Events: An Evidence-

Based Improvement Model. Pediatrics. 2012;129(5):e1334-e42.

9. Westbrook J, Rob M, Woods A, Parry D. Errors in the administration of intravenous

medications in hospital and the role of correct procedures and nurse experience. BMJ Quality and

Safety. 2011:doi: 10.1136/bmjqs-2011-000089.

10. Westbrook JI, Woods A, Rob MI, Dunsmuir WTM, Day RO. Association of interruptions

with an increased risk and severity of medication administration errors. Archives of Internal

Medicine. 2010;170(8):683-90.

11. Manias E, Kinney S, Cranswick N, Williams A, Borrott N. Interventions to Reduce

Medication Errors in Pediatric Intensive Care. Annals of Pharmacotherapy. 2014.

12. van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts ACG, Bollen CW. The

Effect of Computerized Physician Order Entry on Medication Prescription Errors and Clinical

Outcome in Pediatric and Intensive Care: A Systematic Review. Pediatrics. 2009;123(4):1184-90.

13. Westbrook J, Li L, Lehnbom E, M B, Braithwaite J, Burke R, et al. What are incident

reports telling us? A comparative study at two Australian hospitals of medication errors identified at

audit, detected by staff and reported to an incident system. International Journal of Quality in

Health Care. 2015;27(1):1-9.

14. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician

order entry on medication errors and adverse drug events in pediatric inpatients. Pediatrics.

2003;112(3 Pt 1):506-9.

15. Lehmann CU, Johnson KB, Council on Clinical Information Technology American

Academy of Pediatrics. Electronic Prescribing in Pediatrics: Toward Safer and More Effective

Medication Management. Pediatrics. 2013;131(4):e1350-e6.

16. Dufendach KR. Core functionality in pediatric electronic health records.

17. Slight SP, Berner ES, Galanter W, Huff S, Lambert BL, Lannon C, et al. Meaningful Use of

Electronic Health Records: Experiences From the Field and Future Opportunities. JMIR medical

informatics. 2015;3(3):e30.

18. Han Y, Carcillo J, Venkataraman S, Clark R, Watson R, Nguyen T, et al. Unexpected

increased mortality after implementation of a commercially sold computerized physician order

entry system. Pediatrics. 2005;116:1506 - 12.

19. Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS, et al. Decrease in

Hospital-wide Mortality Rate After Implementation of a Commercially Sold Computerized

Physician Order Entry System. Pediatrics. 2010;126(1):14-21.

Page 17 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 17

20. Brigham and Women's Hospital HMS, Partners HealthCare. . Computerized Prescriber

Order Entry Medication Safety (CPOEMS): Uncovering and learning from issues and errors Silver

Spring, MD: US Food and Drug Administration 2015.

21. Westbrook J, Reckmann M, Li L, Runciman W, Burke R, Lo C, et al. Effects of two

commercial electronic prescribing systems on prescribing error rates in hospital inpatients: a before

and after study. PLoS Medicine. 2012;9(1):e1001164. doi:10.1371/journal.pmed.

22. Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of

electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated

with two commercial systems in hospitals. Journal of the American Medical Informatics

Association. 2013;20(6):1159-67.

23. Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and

healthcare systems. Journal of the American Medical Informatics Association. 2015;22(2):465-71.

24. McKibbon K, Lokker C, Handler S, Dolovich LR, Holbrook A, O'Reilly D, et al. Enabling

medication management through health information technology. Rockville MD: Agency for

Healthcare Research and Quality, 2011.

25. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Medical

Research Methodology. 2006;6(1):54.

26. Westbrook J, Woods A. Development and testing of an observational method for detecting

medication administration errors using information technology. Studies in Health Technology &

Informatics. 146. Amsterdam: IOS Press; 2009. p. 429-33.

27. Dean B, Barber N. Validity and reliability of observational methods for studying medication

administration errors. American journal of health-system pharmacy : AJHP : official journal of the

American Society of Health-System Pharmacists. 2001;58(1):54-9.

28. Westbrook JI, Li L, Georgiou A, Paoloni R, Cullen J. Impact of an electronic medication

management system on hospital doctors’ and nurses’ work: a controlled pre–post, time and motion

study. Journal of the American Medical Informatics Association. 2013;20(6):1150-8.

29. Prevention NCCfMERa. NCC MERP Index for Categorizing Medication Errors 2001 [cited

2015]. Available from: http://www.nccmerp.org/types-medication-errors.

30. New South Wales Health Department. Severity Assessment Code (SAC) Matrix. Sydney:

NSW Health, 2005.

31. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials.

Contemporary Clinical Trials. 2007;28(2):182-91.

32. Lellouche F, Mancebo J, Jolliet P, Roeseler J, Schortgen F, Dojat M, et al. A Multicenter

Randomized Trial of Computer-driven Protocolized Weaning from Mechanical Ventilation.

American Journal of Respiratory and Critical Care Medicine. 2006;174(8):894-900.

33. Callen J, Paoloni R, Li J, Stewart M, Gibson K, Georgiou A, et al. Perceptions of the Effect

of Information and Communication Technology on the Quality of Care Delivered in Emergency

Departments: A Cross-Site Qualitative Study. Annals of Emergency Medicine. 2013;61(2):131-44.

34. Ballerman M, Shaw N, Mayes D, Gibney R, Westbrook J. Validation of the Work

Observational Method By Activity Timing (WOMBAT) method of conducting time-motion

observations in critical care settings: an observational study BMC Med Inform Decis.

2011;11:doi:10.1186/472-6947-11-32.

35. Gafter-Gvili A, Fraser A, Paul M, Leibovici L. Meta-Analysis: Antibiotic Prophylaxis

Reduces Mortality in Neutropenic Patients. Annals of Internal Medicine. 2005;142(12_Part_1):979-

95.

Page 18 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 18

FIGURES

Figure 1 Schematic of stepped-wedge cluster randomised controlled trial study design

Figure 2 Medication error, ADE and harm identification and classification process

Medication errors

Identified

Prescribing errors

identified via record

review

Administration errors via

direct observation &

record review

Classify procedural

&

Clinical error types

Error

intercepted

Error NOT

intercepted

Harm detected =

ADE

No harm detected

= Potential ADE

Classify types of harm

& resource use

Confirmed potential

ADE

Classify severity of

Actual ADE

Confirmed potential

ADENo harm occurred

=Potential ADE

Classify potential

error severity

Page 19 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 19

Figure 3 POSSUM tool for data collection during the direct observational study of medication

administration

Page 20 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 20

Figure 4 Example of harm assessment guide for paediatric opioid errors, to be used during medical

record review

Errors resulting in Identify harm indicators

Higher than

recommended dose or

concentration of the

drug

Symptoms: drowsiness, nausea

Signs: pinpoint pupils, lower level of consciousness, respiratory

depression Medications: naloxone, abrupt cessation of opioid

Tests: blood gasses – high CO2, O2 saturation <95%

Actions: any code or arrest

Care record: increased monitoring, increased level of care (e.g.

transfer to ICU), family notified, incident report filed, discharge

delayed Lower than

recommended dose or

concentration of the

drug

Symptoms: complaints of pain, agitation/restlessness

Signs: high pulse rate, high BP, pain scores noted and increasing

Medications: additional doses of analgesic/s

Page 21 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 21

Figure 5 Work Observation Method By Activity Timing (WOMBAT) for conducting observational

studies of health professionals work pre and post eMM system implementation

Page 22 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



The SPIRIT Checklist

Item 1: Descriptive title identifying the study design, population, interventions, and, if

applicable, trial acronym.

“A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an

electronic medication management system to reduce medication errors, adverse drug

events, and average length of stay at two paediatric hospitals.”

Can be found on title page, and page two of submitted protocol document.

Item 2a: Trial identifier and registry name. If not yet registered, name of intended registry.

Australian New Zealand Clinical Trials Registry (ANZCTR)

Item 2b: All items from the World Health Organization Trial Registration Data Set.

Nil.

Item 3: Date and version identifier.

7 March 2016 – Protocol Manuscript - Version One

Item 4: Sources and types of financial, material, and other support.

This project is funded by a National Health and Medical Research Council Partnership Grant

(APP1094878) in partnership with the Sydney Children’s Hospitals Network, eHeath New

South Wales, and the Office of Children and Families, New South Wales. [Page 16]

Item 5a: Names, affiliations, and roles of protocol contributors.

As listed on page 2 and 16

Item 5b: Name and contact information for the trial sponsor.

The principle investigator and sponsor of this study is Professor Johanna Westbrook, of the

Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,

Macquarie University, Sydney, NSW.

E-mail: [email protected]

Phone: +61 02 9850 2402

Item 5c: Role of study sponsor and funders, if any, in study design; collection,

management, analysis, and interpretation of data; writing of the report; and the decision

to submit the report for publication, including whether they will have ultimate authority

over any of these activities.

The trial has been principally designed by Professor Westbrook, and the research team at

the Macquarie University. The research team have full independence from the funding

bodies in terms of decisions regarding the interpretation and publication of findings.

Item 5d: Composition, roles, and responsibilities of the coordinating centre, steering

committee, endpoint adjudication committee, data management team, and other

individuals or groups overseeing the trial, if applicable (see Item 21a for Data Monitoring

Committee).

Page 23 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



The research team at CHSSR, headed by Professor Westbrook will have responsibility for

designing and managing the study, and coordinating the collection, management, analysis

and interpretation of data.

The steering committee will be convened at least every three months, although more

frequently at pertinent times during the study, and will be engaged to provide their

expertise and advice on data collection and study progress. The research at CHSSR will have

responsibility for data management and analysis.

Item 6a: Description of research question and justification for undertaking the trial,

including summary of relevant studies (published and unpublished) examining benefits

and harms for each intervention.

Introduction: Prescribing, administering, and monitoring medicines in children is highly

complex. Compared with adults, medication errors in children are three times more likely to

result in harm. Electronic medication management (eMM) systems are expected to reduce

medication errors and ADEs significantly. However, rigorous evidence demonstrating these

effects is limited. [Page 5]

Existing Knowledge: A systematic review identified eight studies of eMM effectiveness

among paediatric patients. Meta-analysis showed a significant reduction in prescribing error

risk (RR 0.08) but not in ADEs or mortality. There are no Australian studies of eMM system

use in a paediatric setting. Previous studies have often relied upon incident reports to

measure error rates, which are generally unreliable. Only one study of paediatric inpatients

has used a control group to assess eMM effectiveness. Internationally there is currently

insufficient evidence to demonstrate clinical benefit from eMM in paediatric patients.

In 2005, Han et al. reported a significant increase in the mortality rate among critical care

children at a US paediatric hospital following introduction of a commercial eMM system.

The rapid implementation process and limited attention to the significant workflow re-

design required were considered major factors in this outcome. Subsequent studies have

shown no increase in mortality rates. However, the Han et al. study caused considerable

alarm and served to demonstrate the substantial dangers of poor implementation and the

importance of monitoring outcomes following system implementation and responding to

problems identified.

eMM use in adult hospitals, while highly effective at reducing medication errors, also

introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post

eMM in two adult Australian hospitals found 42.4% were facilitated by the system (78 per

100 admissions). [Pages 5-6]

See further justification pages 5-6

Item 6b: Explanation for choice of comparators.

This is an pre-, post-, evaluation study, assessing changes between medication

administration errors, adverse drug events (ADEs) and effects on patient length of stay, pre

and post eMM roll-out. See pages 7-8 for further details

Item 7: Specific objectives or hypotheses.

Aim 1 - To quantify the safety and effectiveness of an eMM system to reduce medication

errors, ADEs and average LOS among paediatric patients using a stepped-wedge cluster

randomised controlled trial (SWCRCT) design;

Page 24 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Aim 2 - To assess the effects of the eMM on clinicians’ workflow and efficiency; and

Aim 3 - To assess the extent to which feedback of study results and subsequent



via a second SWCRCT. [Page 7]

Item 8: Description of trial design including type of trial (e.g., parallel group, crossover,

factorial, single group), allocation ratio, and framework (e.g., superiority, equivalence,

non-inferiority, exploratory).

The study is a stepped wedge cluster randomised controlled trial (SWCRCT) evaluating

differences pre- and post- eMM implementation.

Item 9: Description of study settings (e.g., community clinic, academic hospital) and list of

countries where data will be collected. Reference to where list of study sites can be

obtained.

The study will be conducted in Sydney, New South Wales and will occur at two large tertiary

pediatric hospitals from within the Sydney Children’s Hospitals Network. [Pages 7-8]

Item 10: Inclusion and exclusion criteria for participants. If applicable, eligibility criteria for

study centres and individuals who will perform the interventions (e.g., surgeons,

psychotherapists).

All patients on the study wards will be participants. [Pages 9-10]

Item 11a: Interventions for each group with sufficient detail to allow replication, including

how and when they will be administered.

Detailed in methods section.

Item 11b: Criteria for discontinuing or modifying allocated interventions for a given trial

participant (e.g., drug dose change in response to harms, participant request, or

improving/worsening disease).

See methods section

Item 11c: Strategies to improve adherence to intervention protocols, and any procedures

for monitoring adherence (e.g., drug tablet return; laboratory tests).

The intervention – the eMM will be mandatory for use in the study hospitals.

Item 11d: Relevant concomitant care and interventions that are permitted or prohibited

during the trial.

N/A

Item 12: Primary, secondary, and other outcomes, including the specific measurement

variable (e.g., systolic blood pressure), analysis metric (e.g., change from baseline, final

value, time to event), method of aggregation (e.g., median, proportion), and time point

for each outcome. Explanation of the clinical relevance of chosen efficacy and harm

outcomes is strongly recommended.

Outcome: Medication administration errors (MAEs), ADEs and LOS

See methods section

Page 25 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Item 13: Time schedule of enrolment, interventions (including any run-ins and washouts),

assessments, and visits for participants. A schematic diagram is highly recommended (see

Figure)

For Aim 1, the data collection at CHW will commence on 25th April, 2016, one week before

the eMM rollout will commence. Direct observations of medication administration will

continue until 10th

July, two weeks after the roll out is complete, while chart reviews will

continue into 2017, until they are completed.

Figure 1: Timeline for the SWCRCT, with the baseline commencing at 25th April

For Aim 2, data will be collected during the same schedule as the implementation.

Item 14: Estimated number of participants needed to achieve study objectives and how it

was determined, including clinical and statistical assumptions supporting any sample size

calculations

See methods section

Item 15: Strategies for achieving adequate participant enrolment to reach target sample

size

See methods section

Item 16a: Method of generating the allocation sequence (eg, computer-generated random

numbers), and list of any factors for stratification. To reduce predictability of a random

sequence, details of any planned restriction (eg, blocking) should be provided in a

separate document that is unavailable to those who enrol participants or assign

interventions

Randomisation of the 8 wards will be conducted by a person blinded to the ward identity.

[Page 8]

Item 16b: Mechanism of implementing the allocation sequence (eg, central telephone;

sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the

sequence until interventions are assigned

Page 26 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Randomisation will not be concealed. The study is a stepped wedge cluster randomised

controlled trial.

Item 16c: Who will generate the allocation sequence, who will enrol participants, and who

will assign participants to interventions?

The research team will determine the allocation sequence and communicate with the

hospital team who are responsible for the implementation of the eMM.

Item 17a: Who will be blinded after assignment to interventions (eg, trial participants,

care providers, outcome assessors, data analysts), and how

No one will be blinded to the assignment of the intervention.

Item 17b: If blinded, circumstances under which unblinding is permissible, and procedure

for revealing a participant’s allocated intervention during the trial

Methods: Data collection, management, and analysis

Item 18a: Plans for assessment and collection of outcome, baseline, and other trial data,

including any related processes to promote data quality (eg, duplicate measurements,

training of assessors) and a description of study instruments (eg, questionnaires,

laboratory tests) along with their reliability and validity, if known. Reference to where

data collection forms can be found, if not in the protocol

See methods

Item 18b: Plans to promote participant retention and complete follow-up, including list of

any outcome data to be collected for participants who discontinue or deviate from

intervention protocols

See methods. Retrospective review hospital medical records is the main source of data.

Item 19: Plans for data entry, coding, security, and storage, including any related

processes to promote data quality (eg, double data entry; range checks for data values).

Reference to where details of data management procedures can be found, if not in the

protocol

The methods section outlines the main data management processes. Information in the

individual ethics applications specified further details.

Item 20a: Statistical methods for analysing primary and secondary outcomes. Reference to

where other details of the statistical analysis plan can be found, if not in the protocol

Data analyses are specified in the methods section/

Page 27 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Item 20b: Methods for any additional analyses (eg, subgroup and adjusted analyses)

See methods section

Item 20c: Definition of analysis population relating to protocol non-adherence (eg, as

randomised analysis), and any statistical methods to handle missing data (eg, multiple

imputation)

See methods section. Analyses will be performed as per intention to treat.

Methods: Monitoring

Item 21a: Composition of data monitoring committee (DMC); summary of its role and

reporting structure; statement of whether it is independent from the sponsor and

competing interests; and reference to where further details about its charter can be

found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed

A formal DMC has not been convened, however members of the research team will have

roles in monitoring the collection and quality of the data. Funding bodies will not have

access to the raw study data.

Item 21b: Description of any interim analyses and stopping guidelines, including who will

have access to these interim results and make the final decision to terminate the trial.

N/A

Item 22: Plans for collecting, assessing, reporting, and managing solicited and

spontaneously reported adverse events and other unintended effects of trial

interventions or trial conduct

The project manager (MZR) and the principle investigator (JIW), as well as the site

coordinators (CM and TOB) will monitor any adverse events which occur and wil report to

the Steering Committee.

Item 23: Frequency and procedures for auditing trial conduct, if any, and whether the

process will be independent from investigators and the sponsor

The project will be undertaken in line with specific human ethics approval requirements.

Item 24: Plans for seeking research ethics committee/institutional review board (REC/IRB)

approval

This study has been approved by the SCHN HREC (EC00130), reference code

HREC/15/SCHN/370.

Item 25: Plans for communicating important protocol modifications (eg, changes to

eligibility criteria, outcomes, analyses) to relevant parties (eg, investigators, REC/IRBs,

trial participants, trial registries, journals, regulators)

Any proposed modifications to the study will be discussed with the project steering

committee, and where required modification from the SCHN HREC will be sought.

Page 28 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Item 26a: Who will obtain informed consent or assent from potential trial participants or

authorised surrogates, and how (see Item 32)

See methods

Item 26b: Additional consent provisions for collection and use of participant data and

biological specimens in ancillary studies, if applicable

Not applicable.

Item 27: How personal information about potential and enrolled participants will be

collected, shared, and maintained in order to protect confidentiality before, during, and

after the trial

See methods

Item 28: Financial and other competing interests for principal investigators for the overall

trial and each study site

The research team declare no competing interests.

Items 29: Statement of who will have access to the final trial dataset, and disclosure of

contractual agreements that limit such access for investigators

Members of the research team involved in the data analysis and the writing up of the study

results will have access to the data set. Access will also be granted to selected members of

the steering committee and the site coordinators, where appropriate. The main funding

bodies will not have access to the raw data.

Item 30: Provisions, if any, for ancillary and post-trial care, and for compensation to those

who suffer harm from trial participation

The intervention implementation is the sole responsibility of the hospital sites and was

intended to proceed regardless of the evaluation study outlined here.

Item 31a: Plans for investigators and sponsor to communicate trial results to participants,

healthcare professionals, the public, and other relevant groups (eg, via publication,

reporting in results databases, or other data sharing arrangements), including any

publication restrictions

Study results will be communicated to the staff of the two study hospitals in meetings and

briefing sessions, to be arranged in coordination with senior staff. The results will be further

communicated with healthcare professionals and the public, through publication of journal

articles, conference presentations and other appropriate means.

Item 31b: Authorship eligibility guidelines and any intended use of professional writers

Page 29 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Standard academic and ethical guidelines for authorship and publication of results will

apply.

Item 31c: Plans, if any, for granting public access to the full protocol, participant-level

dataset, and statistical code

The study protocol will be published and thus available to the public. There are no plans to

provide public access to the full dataset or statistical code at this time.

Page 30 of 29


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication

management system to reduce medication errors, adverse drug events, and average length of stay at two paediatric

hospitals: a study protocol

Journal: BMJ Open

Manuscript ID bmjopen-2016-011811.R1

Article Type: Protocol

Date Submitted by the Author: 18-Jul-2016

Complete List of Authors: Westbrook, Johanna; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Li, Ling; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Raban, Magdalena; Australian Institute of Healht Innovation, Faculty of

Medicine and Health Sciences, Macquarie University Baysari, Melissa; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Mumford, Virginia; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Prgomet, Mirela; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Georgiou, Andrew; Macquarie University, Australian Institute of Health Innovation Kim, Tara; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Lake, Rebecca; Australian Institute of Health Innovation, Faculty of

Medicine and Health Sciences, Macquarie University McCullagh, Cheryl; The Sydney Childrens Hospitals Network Dalla Pozza, Luciano; The Sydney Childrens Hospitals Network Karnon, Jon; The University of Adelaide O'Brien, Tracey; The Sydney Childrens Hospitals Network Ambler, Geoff; The Children’s Hospital at Westmead, Institute of Endocrinology and Diabetes Day, Richard; St Vincents Hospital Sydney and UNSW, Clinical Pharmacology; University of New South Wales, Pharmacology Cowell, Christopher; University of Sydney, Sydney Medical School; Sydney Children's Hospitals Network, Institute of Endocrinology and Diabetes, Children’s Hospital at Westmead

Gazarian, Madlen; University of New South Wales, School of Women's and Children's Health; Sydney Children's Hospital, Department of Immunology and Infectious Diseases Worthington, Rachael; The Sydney Childrens Hospitals Network Lehmann, Christoph; Vanderbilt University White, Les; NSW Health , Office of Kids and Families Barbaric, Draga; The Sydney Childrens Hospitals Network Gardo, Alan; The Sydney Childrens Hospitals Network


BMJ Open on D

ecember 26, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2016-011811 on 21 O

ctober 2016. Dow

nloaded from



Kelly, Margaret; NSW Health , Office of Kids and Families Kennedy, Peter; New South Wales Ministry of Health, eHealth

<b>Primary Subject Heading</b>:

Health informatics

Secondary Subject Heading: Paediatrics

Keywords: PAEDIATRICS, Medication errors, Adverse drug events, Electronic Prescribing, medical order entry systems, hospital medication systems

Page 1 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 1

A stepped-wedge cluster randomised controlled trial to assess the effectiveness

of an electronic medication management system to reduce medication errors,

adverse drug events, and average length of stay at two paediatric hospitals: a

study protocol

Westbrook JI1, Li L

1, Raban MZ

1, Baysari MT

1, Mumford V

1, Prgomet M

1, Georgiou A

1, Kim T

1,

Lake R1, McCullagh C

2, Dalla-Pozza L

2, Karnon J

4, O’Brien TA

2, Ambler G

3, Day R

5, Cowell CT

2,

Gazarian M5 ,Worthington R

2, Lehmann CU

6, White L

7, Barbaric D

2, Gardo A

2, Kelly M

7, Kennedy

P 8

1Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,

Macquarie University, Sydney, Australia, 2The Sydney Children’s Hospitals Network,

3The Sydney

Children’s Hospitals Network and The University of Sydney, 4University of Adelaide,

5School of

Medical Sciences, Faculty of Medicine, University of New South Wales, 6Vanderbilt University,

USA, 7Office of Kids and Families NSW Health,

8eHealth NSW Health Ministry

Corresponding Author: Professor Johanna Westbrook, Centre for Health Systems and Safety

Research, Australian Institute of Health Innovation, Macquarie University, level 6, 75 Talavera Rd,

Macquarie Park, 2109 [email protected] (+61 2) 9850 2402

Keywords: Paediatrics; medication errors; adverse drug events (ADEs); electronic prescribing,

medical order entry systems, hospital medication systems, electronic medical records

Word count: 4515

Page 2 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 2

ABSTRACT

Introduction Medication errors are the most frequent cause of preventable harm in hospitals.

Medication management in paediatric patients is particularly complex and consequently potential

for harms are greater than in adults. Electronic medication management (eMM) systems are

heralded as a highly effective intervention to reduce adverse drug events (ADEs), yet internationally

evidence of their effectiveness in paediatric populations is limited. This study will assess the

effectiveness of an eMM system to reduce medication errors, ADEs, and length of stay (LOS). The

study will also investigate system impact on clinical work processes.

Methods and Analysis A stepped-wedge cluster randomised controlled trial (SWCRCT) will

measure changes pre and post eMM system implementation in prescribing and medication

administration error (MAE) rates, potential and actual ADEs, and average LOS. In stage 1, 8 wards

within the first paediatric hospital will be randomised to receive the eMM system one week apart.

In stage 2, the second paediatric hospital will randomise implementation of a modified eMM and

outcomes will be assessed.

Prescribing errors will be identified through record reviews, and MAEs through direct observation

of nurses and record reviews. Actual and potential severity will be assigned. Outcomes will be

assessed at the patient-level using mixed models, taking into account correlation of admissions

within wards and multiple admissions for the same patient, with adjustment for potential

confounders. Interviews and direct observation of clinicians will investigate the effects of the

system on workflow. Data from site 1 will be used to develop improvements in the eMM and

implemented at site 2, where the SWCRCT design will be repeated (stage 2).

Ethics and Dissemination The research has been approved by the Human Research Ethics

Committee of the Sydney Children’s Hospitals Network and Macquarie University. Results will be

reported through academic journals and seminar and conference presentations.

Page 3 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 3

Strengths

• The Stepped-Wedge Cluster Randomised Controlled Trial (SWCRCT) study design is the

closest to the gold-standard of an RCT, and has rarely been applied to study the effects of

information technology in healthcare.

• Few previous studies have investigated the impact of electronic systems on medication

administration error rates. We present a novel approach to conduct direct observation of this

process using the Precise Observation System for Safe Use of Medicines (POSSUM).

POSSUM allows observers to quickly and accurately record drug information e.g. name,

strength, compliance with procedures, as well as the number and length of interruptions and

multi-tasking.

• We will assess potential harm from medication errors identified and importantly also

measure actual harm to children.

Limitations

• Direct, close observation lends itself to the “Hawthorne effect” whereby participants may

seek to ‘improve’ their performance. This may result in an underestimation of the ‘true’

medication administration error rate.

• This study will not evaluate adverse drug events occurring post discharge.

Page 4 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 4

INTRODUCTION

Prescribing, administering, and monitoring medicines in children is highly complex. Compared

with adults, medication errors in children are three times more likely to result in harm.(1) Paediatric

patients are at particular risk of certain errors, such as tenfold dosing errors, facilitated by mistakes

in dose calculation, poor documentation of decimal points and confusion with the use of zeros.

Some errors have led to death.(2-4) The complex medication decision process in children often

requires information about age, weight, dosing ranges and off-label use of medicines.(2) Systematic

reviews show errors occur in 5-27% of all medication orders for children.(5-7) Prescribing errors

account for 3-37% of errors, dispensing 5-58%, administration 72-75%, and documentation 17-

21%, although accuracy of estimates is affected by the great variability in definitions and

measurement methods used.(6) There is much less research about the harms associated with

medicine use. One Australian study found 19.2% of paediatric inpatients experienced adverse drug

events (ADEs) and estimated 12.3% were potential ADEs, 7.0% were actual ADEs, and 3.6% were

preventable.(8)

The frequency and severity of medication administration errors (MAEs) in Australian adult

hospitals is a cause for concern.(9, 10) In 4,271 medications administered to 720 patients, 74.4%

were found to have at least one procedural failure (e.g. failure to check a patient’s identification).(9,

10) One in four had a clinical error (e.g. wrong dose). Of intravenous drug administrations, 70%

had one clinical error, of which 25.5% were judged to be serious and likely to cause permanent

harm.(9) MAEs among children are rarely studied.(6, 11) A major barrier to MAE research is the

methodological challenges. MAEs cannot be accurately detected from retrospective record reviews;

they require direct observation of nurses administering medications to patients.

Can information technology reduce medication errors?

Electronic medication management (eMM) systems incorporate software which provides users with

the ability to prescribe, monitor and administer medications to patients. These systems also provide

the capacity to incorporate decision support tools such as alerts for drug-drug interactions(12).

eMM are usually integrated into a hospital’s clinical information system (Computerised provider

order entry system). These systems are expected to reduce medication errors and adverse drug

events significantly as a result of improved legibility of medication orders, complete and legally

compliant documentation, and through both the active and passive decision support tools embedded

in them. However, rigorous evidence demonstrating these effects are limited.(7) A systematic

review identified eight studies of eMM effectiveness among paediatric patients. Meta-analysis

showed a significant reduction in prescribing error risk (RR 0.08; 95%CI 0.01-0.77) across the three

Page 5 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 5

included studies, but not in ADEs or mortality.(13) There are no Australian studies of eMM system

use in a paediatric setting. Previous studies have often relied upon incident reports to measure error

rates, which are generally unreliable.(14) Only one study(15) of paediatric inpatients has used a

control group to assess eMM effectiveness. Internationally there is currently insufficient evidence to

demonstrate clinical benefit from eMM in paediatric patients. This view was confirmed by a policy

review for the American Academy of Pediatrics(16) which called for the demonstrable

enhancement of eMM systems to better meet the specific needs of paediatrics to ensure their safety

and effectiveness. Further, existing eMM systems used in paediatrics have been found lacking in

functionalities required for safe prescribing, dispensing and administration of medications.(17, 18)

In 2005, Han et al(19) reported a significant increase in the mortality rate among critical care

children at a US paediatric hospital following introduction of a commercial eMM system. The rapid

implementation process and limited attention to the significant workflow re-design required were

considered major factors in this outcome. Subsequent studies(20) have shown no increase in

mortality rates. However, the Han et al(19) study caused considerable alarm and served to

demonstrate the substantial dangers of poor implementation and the importance of monitoring

outcomes following system implementation and responding to problems identified.(21)

eMM use in adult hospitals, while highly effective at reducing medication errors,(22) also

introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post eMM in

two adult Australian hospitals found 42.4% were facilitated by the system (78 per 100

admissions).(23) The most frequent mechanism was incorrect selection from a drop-down menu.

Results from that study were used to instigate changes to eMM software and to inform changes to

user training programs. The study was one of the first to quantify the rate of ‘system-related’ errors

and produced an innovative dual classification for categorising both the manifestations and

mechanisms of these system-related errors.(23) The nature and magnitude of such potential new

risks have rarely been studied in paediatrics.

Why evaluate eHealth systems

eHealth systems are having an increasing impact on the delivery of health care services, yet, despite

their widespread effects and vast cost, they are rarely subject to rigorous research.(24) This limited

evidence-base significantly hinders improvements and innovation in the design, implementation and

use of health information technology (IT) systems. Internationally the need for IT evaluation studies

to employ more robust designs and sophisticated analyses is well recognised.(25) Excessive

reliance has been placed upon uncontrolled before and after, and qualitative studies. Evidence of

Page 6 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 6

eHealth system effectiveness and safety is crucial to facilitate policy-makers’ and health care

organisations’ informed decisions about investments and prioritisation of health IT systems.

This project presents an innovative and comprehensive program to assess eMM system

effectiveness in reducing medication errors, ADEs and length of stay (LOS) and rapidly deploying

new knowledge into practice for subsequent implementation.

The project aims are:

Aim 1 To quantify the safety and effectiveness of an eMM system to reduce medication

errors, ADEs and average LOS among paediatric patients using a stepped-wedge

cluster randomised controlled trial (SWCRCT) design;

Aim 2 To assess the effects of the eMM on clinicians’ workflow and efficiency; and

Aim 3 To assess the extent to which feedback of study results and subsequent



via a second SWCRCT.

METHODS AND ANALYSIS

Research Plan

Aim 1: Quantify the safety and effectiveness of an eMM system to reduce medication errors

(potential and actual ADEs), and average LOS among paediatric patients.

Study Design and Setting: We will conduct a SWCRCT to measure changes in prescribing and

medication administration errors (MAEs) which result in potential and actual ADEs, along with

changes in average LOS, pre and post eMM. Cluster randomised controlled trials (CRCT) are

ideally suited to test interventions where individual patient randomisation is not possible. CRCTs

commonly use a parallel group design, in which the clusters are randomised to either the

intervention or control arm of the study. It is often regarded as unethical to withhold an intervention

from a proportion of participants if it is believed that the intervention will do more good than harm.

The SWCRCT design, where the intervention is delivered sequentially to all trial clusters over a

number of time periods, is an alternative to the traditional parallel groups design. The order in

which the clusters (wards) receive the intervention is randomised, and by the end of the study all

clusters will have adopted the intervention.(26) The steps represent the predetermined periods

when data relating to each of the clusters will be collected. This design is the closest to the gold-

standard of an RCT, when such a design is not possible. The stepped-wedge design offers particular

Page 7 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 7

strengths in allowing the modelling of the effect of time on the effectiveness of the intervention. In

stage 1 a SWCRCT will be conducted at site 1, and in stage 2 a SWCRCT will be conducted at site

2.

The study setting is the Sydney Children’s Hospitals Network which incorporates the two major

urban tertiary referral paediatric hospitals in Sydney, Australia, The Children’s Hospital at

Westmead (CHW; site 1) and Sydney Children’s Hospital, Randwick (SCH; site 2). The network

provides care for 51,000 inpatient admissions, 92,000 emergency department presentations and over

1,000,000 outpatient services events each year. Both sites are acute paediatric tertiary hospitals with

emergency departments, out-patient and home services. During the study period the eMM will not

be available in the ICUs, theatres or outpatients. Site 1 accommodates 310 beds and site 2 has 180

beds. Both hospitals provide a complex and comprehensive range of services caring for seriously ill

and injured children and adolescents across the state of New South Wales and beyond.

The eMM Intervention: At baseline medication orders are written on paper medication charts and

details of medications administered are written on the same charts. The eMM clinical module will

be an addition to the hospitals’ existing commercial electronic clinical information system (Cerner

Corporation). Both hospital sites use the same commercial clinical system, however the software

can be customised to meet each hospital’s individual requirements. Based on results from our first

site and applying our action research methodology, we will provide advice to the second

implementation site on optimal customisation of the eMM (and associated processes) for their site.

The eMM allows electronic prescribing, recording of drug dispensing, drug administration and

medication reconciliation and monitoring processes. (See supplementary file for screenshots from

the eMM). The system allows for the ordering and administration of all oral, and IV medications

and fluids, but excludes anaesthesia medications. The eMM contains both passive and active

decision support in the form of links to guidelines, policies, protocols, order-sets, order sentences,

safety alerts (e.g. drug-drug interactions, dose range checks), and dosage calculators. During the

course of the study the eMM system will be accessible via any computer in the hospital allocated

for inpatient clinical care, but will not be available for patients in the intensive care units, theatres or

outpatients. The system will be predominantly accessed on hospital wards and in the hospital

pharmacy. Both fixed and mobile computing devices are available to staff using the system.

Medication reconciliation on admission and at discharge will be performed using the eMM system

when implemented. On admission, medication histories are taken and converted to inpatient orders.

While the patient is in hospital any new medication orders will be created within the eMM system.

Page 8 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 8

On discharge a discharge medication reconciliation occurs and orders are converted to paper

prescriptions for the patient. Patients then have their prescriptions filled at community pharmacies.

Randomisation: In stage 1, 8 wards within site 1 will be randomised to receive the eMM system.

Randomisation will be conducted by a person blinded to ward identity. At baseline all 8 wards are

in the control period (Figure 1). At step 1, the eMM system will be implemented in the first ward.

The eMM system will then be implemented to a new ward in sequence weekly. By the end of step

8, all wards will have the eMM system at site 1.


Medication error and ADE definitions: Medication errors are defined as any error in the

prescribing, supply, preparation, administration, or monitoring of a medication, regardless of

whether such errors lead to adverse consequences. In this study we will not measure dispensing

errors. ADEs are defined as harm or injury as a consequence of the use or non-use of medicines.(8)

Medication errors may result in actual ADEs or potential ADEs. For example, a medication error

may occur but is intercepted prior to administration thus preventing harm to the patient.

In this study we will be seeking to identify medication errors, and to determine those that resulted in

harm (actual ADEs) or potential harm (potential ADEs). Figure 2 illustrates the medication error

and ADE classification processes for this study.


Data Collection: Medication error data collection occurs at baseline (a one week period) and each

stage, (i.e. in every subsequent week as eMM implementation occurs and for two additional weeks

after full implementation). For the primary study objective of determining eMM effectiveness to

reduce medication errors and ADEs we will collect data at 11 points on all wards (baseline and at

each step including two weeks after full implementation, Figure 1). This will allow us to measure

changes pre and post eMM system introduction in: 1) Prescribing error rates per order and per

admission by type and severity (potential and actual ADEs); 2) MAEs per order and per admission

by type and severity (potential and actual ADEs). For the secondary outcome of changes in LOS we

will obtain data for a further 21 steps in the follow-up period to provide greater statistical power. As

these are routine administrative data, no additional data collection is required. Data on adverse drug

events occurring post discharge is not within the scope of this study.

Prescribing error and ADE detection: A review of medication charts at baseline and each step will

Page 9 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 9

be conducted complying with a standard error protocol.(8-10, 22) This protocol will be extended to

develop methods for determining the rate at which errors are detected and intercepted by staff,

actions taken, and any harm experienced.

Medication administration error (MAE) and ADE detection: For the MAE study, data will be

collected using direct observation. Nurses will be observed preparing and administering

medications. In our previous studies using this approach in adult hospitals,(10) over 80% of nurses

consented to participate and we expect similar rates for this study. Direct observations will be

supported by an innovative data collection tool, the Precise Observation System for Safe Use of

Medicines (POSSUM, Figure 3).(10, 27) The POSSUM tool allows observers to quickly and

accurately record drug information e.g. name, strength, and dose. The POSSUM tool also allows

collection of the number and length of interruptions experienced and multi-tasking (e.g. answering a

question while also selecting medicines). Nurses’ compliance with core procedures, such as

checking a patient’s identification, will also be recorded. Comparing observational data with

patients’ medical records (via retrospective audit) will enable identification of the number, types

and severity of MAEs.


Observers will have a pre-allocated observation period to ensure coverage across the day and the

week.(9, 10) Observers will follow a “serious error” protocol i.e. they must intervene if they witness

an administration that is potentially dangerous to the patient. Observers will not have access to

patients’ medication charts and will record only what they observe. Thus, most MAEs will not be

identifiable until chart review. Past inter-rater reliability tests showed kappa scores from 0.94 to

0.96 following training in the use of POSSUM.(9, 10)

Direct, close observation lends itself to the “Hawthorne effect” whereby participants may seek to

‘improve’ their performance. If nurses change their practices, and are more careful when observed,

this will lead to an underestimation of the ‘true’ MAE rate. This bias would be present both pre and

post eMM. Our prior research suggests the likelihood of sustained change on busy wards is low.(28,

29)

Prescribing error, MAE and ADE classifications: Prescribing errors and MAEs will be classified

into: (i) procedural errors and (ii) clinical errors using previously applied classifications.(10, 22)

Procedural errors include, for prescribing orders, illegible orders, illegal orders (in which an aspect

Page 10 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 10

of the prescription does not comply with hospital policy, with law, or with the State Department of

Health policies) and incomplete orders. Medication administration procedural errors include, for

example, failure to follow the correct patient identification process prior to drug administration, and

correct conduct of medication double-checking requirements (many drugs within paediatric

hospitals require two nurses to independently check aspects of the drug preparation and

administration process). Clinical errors include wrong dose, wrong drug, wrong route, and wrong

strength errors.

Once an error has been identified, a rating of the potential severity of that error will be made, based

upon the NCC-MERP rating scale for adverse event outcomes.(30) Subsequently, records will be

reviewed for evidence of error detection and interception, and for any actual harm to the patient.

Thus, medication errors which occur will receive both a ‘potential’ harm rating and an ‘actual harm

rating’ (Figure 2). As most previous medication error studies do not assess actual harm, this double

classification process will allow us to compare our findings with previous studies, as well as allow

an assessment of the accuracy of such approaches compared to estimating the actual harm from

medication errors.

Evidence of harm as a consequence of a medication error will be identified through a

comprehensive review of patients’ medical records. This clinical review process will be assisted by

the provision of specific harm identification guides for reviewers which will identify, for specific

drugs and error types, the types of evidence which would suggest harm had occurred following the

medication error. Figure 4 presents an example of one of the harm identification guides to be used.


Page 11 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 11

Experienced clinicians will abstract data from medical records using a structured data collection

form and the harm identification guides. A multi-disciplinary clinical review panel will re-assess a

minimum 5% sample of the records and will also review any records which reviewers identify as

particularly complex. Panel members will be blind to the location, and whether data were generated

pre or post eMM. Panel members will not know the ward order of rollout and specific dates when

each ward became an intervention ward with the stepped wedge design and therefore blinding of

pre and post data will be possible. Actual and potential severity will be assigned using the National

Coordinating Council for Medication Error Reporting and Prevention (NCC-MERP) scale for

adverse event outcomes (30) and the 5-point Severity Assessment Code (SAC) Scale,(31) as used in

our past research.(9, 10, 22) This will allow comparison with a greater number of previous studies.

System-related errors: We will apply our two dimensional classification, modified to incorporate

recent recommendations in this area,(21) to assess whether medication errors post eMM were

facilitated by eMM design, i.e. are ‘system-related’. This process identifies the manifestation (e.g.

wrong dose) and mechanisms (e.g. incorrect menu selection). These results will be used to provide

recommendations about IT design and user training.(23) Any changes to the eMM design features,

training or work processes during the study will be documented.

Sample sizes and analyses: Sample size calculations have taken into account the estimated

between-cluster variance, i.e. between wards variance, and the design effect associated with the

stepped-wedge design.(32) Calculations were based on our previous studies in adult hospitals and

hospital data from the paediatric sites. Each ward has on average 14 admissions per week with an

average LOS 3.78 days (SD=7.39) with seven medications per admission.

Prescribing errors: Based on our previous studies(22) the expected reduction in overall prescribing

error rate is 60%, from 4.06 errors per admission (SD=5.27) to 1.62 (SD=2.87) with an estimated

intraclass correlation coefficient (ICC) of 0.06 (Table 1). The number of wards required to detect a

60% change for two-sided tests (80% power; �<5%) is one, with 10 data collection steps after

baseline. For ADEs, the required number of wards is seven to detect a 60% reduction (Table 1). To

be conservative and provide greater power, we will collect data on all eight wards allowing

detection of a minimum change of 20% for overall errors and 42% for ADEs. At each step records

for 112 patient admissions will be reviewed, totalling 1,232 across the study.

MAEs: Based on our previous studies(9, 10) we expect the overall MAE rate per administration to

fall by 27%, from 0.37 (SD=0.65) to 0.27 (SD=0.52) with an estimated ICC of 0.03 (Table 1). The

required number of wards (two-sided test; 80% power; �<5%) is seven, with 10 steps after baseline.

Page 12 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 12

For ADEs, the required number of wards is six. We will collect data using all eight wards which

will allow detection of a minimum 20% change overall and 48% for ADEs. At each step we will

observe at least 240 medication administrations, totalling 2,640 across the study.

LOS: There are very limited data on the impact of eMM systems on LOS at ward level. A study in

an ICU showed a 23% reduction in LOS post eMM system.(33) To detect a 23% reduction in LOS,

i.e. from 3.78 (SD=7.39) to 2.92 days, with eight wards, will require additional (routinely collected

LOS) data in a total of 31 steps for a two-sided test with 80% power �<5%.

Data analyses: Medication error rates per order, stratified by error type, study step, and ward will

be calculated. For each outcome of interest, data collected across all measurement periods and all

study steps will be used in the analyses comparing intervention status (pre versus post eMM).

Analyses will apply the intention-to-treat principle. Patient data will be analysed according to the

status of the wards (i.e. pre or post eMM) where patients were admitted. Outcomes will be assessed

at the patient-level using mixed models, taking into account correlation of patient admissions within

wards (clusters) and multiple admissions for the same patient, with adjustment for potential

confounding factors. For the MAE analyses we will adjust for contextual factors including

interruptions, multi-tasking, nurse age, gender, and adherence to policies. For LOS analysis we will

adjust for patient characteristics, such as major diagnoses, comorbidity, age, and gender. The mixed

models will incorporate fixed terms for ward intervention status, measurement time steps (including

baseline) and other confounders. The analyses will include multiple time points pre and post eMM

implementation. The study design will allow us to determine temporal changes in system

effectiveness, e.g. to determine if error rates continue to decline over time. We will apply the

‘system-related’ error classification(23) to identify system-related error rate and associated

mechanisms.

Page 13 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 13

No. = Number

Table 1 Prescribing error and medication administration error power calculations

Error Rate

Pre-

(SD)

Error Rate

Post-

(SD)

% changes

from past

eMM study

ICC (7) Mean No. of

admissions or

administrations/

study step

No. of

steps (k)

Min. No.

wards

required

No. of

wards

Min. % change

detectable

Max.

power

Prescribing

errors

(per admission)

4.1 (5.3) 1.6 (2.9) 60% 0.06 14 10 1 8 20% 100%

ADEs

(per admission)

0.3 (0.7) 0.1 (0.4) 44% 0.005 14 10 7 8 42% 83%

MAEs

(per

administration)

0.4 (0.6) 0.3 (0.5) 27% 0.03 30 10 4 8 20% 97%

ADEs (% all

medication

administrations)

4.2% 1.8% 57% 0.003 30 10 6 8 48% 93%

Page 14 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on December 26, 2020 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2016-011811 on 21 October 2016. Downloaded from



Page 14

Aim 2: To assess the effects of the eMM on workflow and efficiency

Study design and sample: Observations and interviews will be held with medical, nursing, and

pharmacy staff at baseline to allow mapping of core work processes associated with medication

provision. At each step in the stepped-wedge design, a small number of interviews (approximately

four on each ward) will be conducted with nursing and medical staff to gain insights into clinical

staff perceptions of the impact of the system on workflow, efficiency and care delivery. These

interviews will be held one week, three weeks, six weeks and six months post eMM implementation

on each study ward. Members of the research team will directly approach hospital staff and invite

them to participate in interviews which will take approximately 10 minutes each. We anticipate

approximately 80-90 interviews will be conducted across the course of the study. Trustworthiness

of the qualitative data will be achieved through triangulation of data and investigators, engagement

with the field with a documented audit trail and member checking.(34)

The eMM is anticipated to have a significant impact on the work of hospital pharmacists. We will

conduct a direct observational study of approximately eight pharmacists at site 1. We will observe

them for 200 hours between 7:30 – 18:00 pre and post eMM system implementation to examine

changes in i) task time distributions ii) location of work and iii) communication patterns. Using the

validated WOMBAT approach(35, 36) multiple dimensions of work will be captured (e.g. tasks

performed, with whom, with what, location, interruptions, and multi-tasking). On data entry, tasks

are automatically time stamped when entered in the WOMBAT data collection tool. Figure 5 is an

example of data collection within WOMBAT. An additional sample of 140 hours of observation

will be conducted to capture the work of oncology pharmacists whose work involves supporting the

delivery of complex drug regimens to children with cancer.


Data generated will allow changes in task time distributions and sequencing of work to be

determined. These data will be examined in relation to changes in outcome indicators generated

(from Aim 1, e.g. medication error rates, LOS) on the same wards.

Aim 3: Assess the extent to which feedback (from Aims 1 and 2) and subsequent modifications to

an eMM system design can improve eMM system effectiveness in reducing medication errors

Evaluations of health IT serve multiple purposes, ranging from providing an objective assessment

Page 15 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 15

of the success of the new technology in delivering anticipated benefits, to identification of deficits

in the system, their source, and the ways they can be addressed. This is critical to improving system

effectiveness, relevance, and responsiveness. For Aim 3, the findings of the SWCRCT at CHW (i.e.

stage 1) will be reported to the Project Evaluation Committee (PEC) made up of members of the

research team, and the Hospitals’ eMM system Project Steering Committee. The PEC will meet

every month to consider the implications of study findings across a number of domains including

the system’s technical features (e.g. compatibility with other hospital systems), effectiveness (e.g.

error reduction and system-related errors); professional attitudes (e.g. satisfaction) and

organisational features (e.g. work processes), as a means of formulating changes to eMM system

design features and user training.

This will form the key component of an action-oriented approach aimed at optimising system

performance leading to an enhanced eMM system which will then be implemented across the

second site, SCH (stage 2). The SWCRCT design will be repeated at site 2 using the same methods

as above. Power calculations for stage 2 will be based on results from stage 1. We will conduct

separate analyses for all outcomes specified. Results for the two sites will be compared, using

multilevel and longitudinal analysis approaches to determine changes in error rates (taking baseline

data into account).

Expected outcomes and significance of the research project

This project will generate the first Australian data, in a paediatric setting, on the effectiveness of

eMM systems to reduce medication errors and ADEs, and provide an assessment of how systems

impact on the work of clinicians and the consequences for the delivery of care to children.

Importantly, the findings will be directly applied to enhance the eMM system design, and work

processes and then tested further through evaluation of the enhanced eMM system at a second

paediatric hospital. These results will be particularly valuable for other paediatric hospitals yet to

commence implementations. Exploiting the SWCRCT design within an action-research model is

highly innovative, and will deliver high quality data on system effectiveness. Such a model of

formally integrating health IT assessment results as a basis for active engagement with IT vendors

and clinicians to bring about system change has both national and international significance. The

study advances explicit methods for the systematic identification of harm associated with

medication errors. The data generated will also provide the basis for a robust cost-effectiveness

analysis, which will be the subject of a separate protocol.

Page 16 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 16

ETHICS AND DISSEMINATION

The research has been approved by the Human Research Ethics Committee of the Sydney

Children’s Hospitals Network (HREC/15/SCHN/370). In the first instance, results from site 1 will

be reported to the PEC so that they can be used to inform eMM system and work process design

prior to implementation at site 2. Results will also be reported through academic journals and

conference presentations. The project is funded through a National Health and Medical Research

Council Partnership Grant. As such, the project team includes academic researchers, hospital

clinicians and experts involved in the implementation of the eMM system at the two hospital sites,

along with senior policy makers from agencies within the State Health Department involved in

eHealth system strategy and policy. This provides the project with access to a range of other

conduits through which to disseminate results to, for example, policy makers and system

implementers.

Page 17 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 17

Authors Contributions: JW, LL, MB, AGe and CM initiated the project and led the development

of the NHMRC grant proposal. JW, LL, MB, AGe, RD, JK, LDP, CTC, GA, and TOB are chief

investigators on the project and all made contributions to the protocol in their specific areas of

expertise. CM, LW, NB, AGa, CUL, MG, PK, MK, AB and DB are associate investigators on the

NHMRC grant and provided input to the protocol, particularly in the areas of paediatric clinical

practice and broader eHealth strategy in relation to eMM systems. MZR, MP, VM, TK, RW, RL are

members of the project team and have made significant contributions to the protocol in terms of the

design of details regarding the collection and classification of medication errors and harm. JW

prepared the first draft of this manuscript based upon the grant proposal and all authors have

reviewed and provided input.

Funding Statement: The project is supported by a National Health and Medical Research Council

Partnership Grant (APP1094878) in partnership with: Sydney Children’s Hospitals Network;

eHealth New South Wales; Office of Kids and Families, New South Wales.

Competing Interests Statement: None to declare.

Page 18 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 18

References:

1. Kaushal R, Barker KN, Bates DW. How can information technology improve patient safety

and reduce medication errors in children's health care? Arch Pediat Adol Med.

2001;155(9):1002-7.

2. Doherty C, McDonnell C. Tenfold medication errors: 5 years’ experience at a university-

affiliated pediatric hospital. Pediatrics. 2012;129(5):916-24.

3. Gazarian M, Drew A, Bennett A. Medicinal mishap. Intravenous paracetamol in

paediatrics: cause for caution. Aust Prescr. 2014;37(1):24-5.

4. Kim GR, Chen AR, Arceci RJ, Mitchell SH, Kokoszka KM, Daniel D, et al. Error reduction

in pediatric chemotherapy: computerized order entry and failure modes and effects analysis.

Arch Pediat Adol Med. 2006;160(5):495.

5. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, et al.

Medication errors and adverse drug events in pediatric inpatients. J Amer Med Assoc.

2001;285(16):2114-20.

6. Miller MR, Robinson KA, Lubomski LH, Rinke ML, Pronovost PJ. Medication errors in

paediatric care: a systematic review of epidemiology and an evaluation of evidence

supporting reduction strategy recommendations. Qual Saf Health Care. 2007;16(2):116-26.

7. Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to

reduce pediatric medication errors: a systematic review. Pediatrics. 2014.

8. Gazarian M, Graudins LV. Long-term reduction in adverse drug events: an evidence-based

improvement model. Pediatrics. 2012;129(5):e1334-e42.

9. Westbrook JI, Rob MI, Woods A, Parry D. Errors in the administration of intravenous

medications in hospital and the role of correct procedures and nurse experience. BMJ Qual

Saf. 2011:doi: 10.1136/bmjqs-2011-000089.

10. Westbrook JI, Woods A, Rob MI, Dunsmuir WTM, Day RO. Association of interruptions

with an increased risk and severity of medication administration errors. Arch Intern Med.

2010;170(8):683-90.

11. Manius E, Kinney S, Cranswick N, Williams A, Borrot N. Interventions to reduce

medication errors in pediatric intensive care. Ann Pharmacother. 2014;48(10):1313-31.

12. Australian Commission on Safety and Quality in Health Care. Electronic medication

management (EMM): a guide for healthcare providers 2015.

13. van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts ACG, Bollen CW. The

effect of computerized physician order entry on medication prescription errors and clinical

outcome in pediatric and intensive care: a systematic review. Pediatrics. 2009;123(4):1184-

90.

14. Westbrook JI, Li L, Lehnbom EC, M Baysari, Braithwaite J, Burke R, et al. What are

incident reports telling us? A comparative study at two Australian hospitals of medication

errors identified at audit, detected by staff and reported to an incident system. Int J Qual

Health Care. 2015;27(1):1-9.

15. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician

order entry on medication errors and adverse drug events in pediatric inpatients. Pediatrics.

2003;112(3 Pt 1):506-9.

Page 19 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 19

16. Lehmann CU, Johnson KB, Technology Council on Clinical Information, American

Academy of Pediatrics. Electronic prescribing in pediatrics: toward safer and more effective

medication management. Pediatrics. 2013;131(4):e1350-e6.

17. Dufendach KR, Eichenberger JA, McPheeters ML, al. et. Core functionality in pediatric

electronic health records: Agency for Healthcare Research and Quality; 2015.

18. Slight SP, Berner ES, Galanter W, Huff S, Lambert BL, Lannon C, et al. Meaningful use of

electronic health records: experiences from the field and future opportunities. JMIR Med

Inform. 2015;3(3):e30.

19. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, et al.

Unexpected increased mortality after implementation of a commercially sold computerized

physician order entry system. Pediatrics. 2005;116:1506 - 12.

20. Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS, et al. Decrease in

hospital-wide mortality rate after implementation of a commercially sold computerized

physician order entry system. Pediatrics. 2010;126(1):14-21.

21. Brigham and Women's Hospital, Partners HealthCare, Harvard Medical School.

Computerized Prescriber Order Entry Medication Safety (CPOEMS): Uncovering and

learning from issues and errors. 2015.

22. Westbrook JI, Reckmann M, Li L, Runciman WB, Burke R, Lo C, et al. Effects of two

commercial electronic prescribing systems on prescribing error rates in hospital inpatients:

a before and after study. PLoS Med. 2012;9(1):e1001164. doi:10.1371/journal.pmed.

23. Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of

electronic prescribing: manifestations, mechanisms, and rates of system-related errors

associated with two commercial systems in hospitals. J American Med Inform Assn.

2013;20(6):1159-67.

24. Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and

healthcare systems. J Amer Med Inform Assoc. 2015;22(2):465-71.

25. McKibbon KA, Lokker C, Handler SM, Dolovich LR, Holbrook AM, O'Reilly D, et al.

Enabling medication management through health information technology. Rockville MD:

Agency for Healthcare Research and Quality, 2011.

26. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res

Methodol 2006;6(1):54.

27. Westbrook JI, Woods A. Development and testing of an observational method for detecting

medication administration errors using information technology. St Heal T. 2009;146:429-

33.

28. Dean B, Barber N. Validity and reliability of observational methods for studying

medication administration errors. Am J Health Syst Pharm. 2001;58(1):54-9.

29. Westbrook JI, Li L, Georgiou A, Paoloni R, Cullen J. Impact of an electronic medication

management system on hospital doctors’ and nurses’ work: a controlled pre–post, time and

motion study. J Am Med Inform Assn. 2013;20(6):1150-8.

30. National Coordinating Council for Medication Error Reporting and Prevention. NCC

MERP Index for categorizing medication errors 2001 [cited 2015]. Available from:

http://www.nccmerp.org/types-medication-errors.

31. Government NSW, Department Health. Incident Management Policy. 2014.

32. Hussey MA, Hughes JPs. Design and analysis of stepped wedge cluster randomized trials.

Contemp Clin Trials. 2007;28(2):182-91.

Page 20 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 20

33. Lellouche F, Mancebo J, Jolliet P, Roeseler J, Schortgen F, Dojat M, et al. A multicenter

randomized trial of computer-driven protocolized weaning from mechanical ventilation. Am

J Resp Crit Care. 2006;174(8):894-900.

34. Callen J, Paoloni R, Li J, Stewart M, Gibson K, Georgiou A, et al. Perceptions of the effect

of information and communication technology on the quality of care delivered in

emergency departments: a cross-site qualitative study. Ann Emerg Med. 2013;61(2):131-44.

35. Ballerman MA, Shaw NT, Mayes DC, Gibney RTN, Westbrook JI. Validation of the Work

Observational Method By Activity Timing (WOMBAT) method of conducting time-motion

observations in critical care settings: an observational study BMC Med Inform Decis.

2011;11:doi:10.1186/472-6947-11-32.

36. Westbrook JI, Ampt A. Design, application and testing of the Work Observation Method

by Activity Timing (WOMBAT) to measure clinicians’ patterns of work and

communication. Int J Med Inform. 2009;78S:S25-S33.

Page 21 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Page 21

FIGURES

Figure 1 Schematic of stepped-wedge cluster randomised controlled trial study design

Figure 2 Medication error, ADE and harm identification and classification process

Figure 3 POSSUM tool for data collection during the direct observational study of medication

administration

Figure 4 Example of harm assessment guide for paediatric opioid errors, to be used during medical

record review following identification of an opioid prescribing error

Figure 5 Work Observation Method By Activity Timing (WOMBAT) for conducting observational

studies of health professionals work pre and post eMM system implementation

Page 22 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Figure 1: Schematic of stepped-wedge cluster randomised controlled trial study design

124x30mm (300 x 300 DPI)

Page 23 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Figure 2: Medication error ADE and harm identification and classification process

163x97mm (300 x 300 DPI)

Page 24 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Figure 3: POSSUM tool for data collection during the direct observational study of medication administration

270x433mm (300 x 300 DPI)

Page 25 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Figure 4: Example of harm assessment guide

184x173mm (300 x 300 DPI)

Page 26 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Figure 5: WOMBAT tool for conducting observational studies of health professionals work pre and post eMM system implementation

318x527mm (300 x 300 DPI)

Page 27 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Example of a medication administration screen

Page 28 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Example of a medication order screen in the eMM system

Page 29 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



The SPIRIT Checklist

Item 1: Descriptive title identifying the study design, population, interventions, and, if

applicable, trial acronym.

“A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an

electronic medication management system to reduce medication errors, adverse drug

events, and average length of stay at two paediatric hospitals.”

Can be found on title page, and page two of submitted protocol document.

Item 2a: Trial identifier and registry name. If not yet registered, name of intended registry.

Australian New Zealand Clinical Trials Registry (ANZCTR)

Item 2b: All items from the World Health Organization Trial Registration Data Set.

Nil.

Item 3: Date and version identifier.

7 March 2016 – Protocol Manuscript - Version One

Item 4: Sources and types of financial, material, and other support.

This project is funded by a National Health and Medical Research Council Partnership Grant

(APP1094878) in partnership with the Sydney Children’s Hospitals Network, eHeath New

South Wales, and the Office of Children and Families, New South Wales. [Page 16]

Item 5a: Names, affiliations, and roles of protocol contributors.

As listed on page 2 and 16

Item 5b: Name and contact information for the trial sponsor.

The principle investigator and sponsor of this study is Professor Johanna Westbrook, of the

Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,

Macquarie University, Sydney, NSW.

E-mail: [email protected]

Phone: +61 02 9850 2402

Item 5c: Role of study sponsor and funders, if any, in study design; collection,

management, analysis, and interpretation of data; writing of the report; and the decision

to submit the report for publication, including whether they will have ultimate authority

over any of these activities.

The trial has been principally designed by Professor Westbrook, and the research team at

the Macquarie University. The research team have full independence from the funding

bodies in terms of decisions regarding the interpretation and publication of findings.

Item 5d: Composition, roles, and responsibilities of the coordinating centre, steering

committee, endpoint adjudication committee, data management team, and other

individuals or groups overseeing the trial, if applicable (see Item 21a for Data Monitoring

Committee).

Page 30 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



The research team at CHSSR, headed by Professor Westbrook will have responsibility for

designing and managing the study, and coordinating the collection, management, analysis

and interpretation of data.

The steering committee will be convened at least every three months, although more

frequently at pertinent times during the study, and will be engaged to provide their

expertise and advice on data collection and study progress. The research at CHSSR will have

responsibility for data management and analysis.

Item 6a: Description of research question and justification for undertaking the trial,

including summary of relevant studies (published and unpublished) examining benefits

and harms for each intervention.

Introduction: Prescribing, administering, and monitoring medicines in children is highly

complex. Compared with adults, medication errors in children are three times more likely to

result in harm. Electronic medication management (eMM) systems are expected to reduce

medication errors and ADEs significantly. However, rigorous evidence demonstrating these

effects is limited. [Page 5]

Existing Knowledge: A systematic review identified eight studies of eMM effectiveness

among paediatric patients. Meta-analysis showed a significant reduction in prescribing error

risk (RR 0.08) but not in ADEs or mortality. There are no Australian studies of eMM system

use in a paediatric setting. Previous studies have often relied upon incident reports to

measure error rates, which are generally unreliable. Only one study of paediatric inpatients

has used a control group to assess eMM effectiveness. Internationally there is currently

insufficient evidence to demonstrate clinical benefit from eMM in paediatric patients.

In 2005, Han et al. reported a significant increase in the mortality rate among critical care

children at a US paediatric hospital following introduction of a commercial eMM system.

The rapid implementation process and limited attention to the significant workflow re-

design required were considered major factors in this outcome. Subsequent studies have

shown no increase in mortality rates. However, the Han et al. study caused considerable

alarm and served to demonstrate the substantial dangers of poor implementation and the

importance of monitoring outcomes following system implementation and responding to

problems identified.

eMM use in adult hospitals, while highly effective at reducing medication errors, also

introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post

eMM in two adult Australian hospitals found 42.4% were facilitated by the system (78 per

100 admissions). [Pages 5-6]

See further justification pages 5-6

Item 6b: Explanation for choice of comparators.

This is an pre-, post-, evaluation study, assessing changes between medication

administration errors, adverse drug events (ADEs) and effects on patient length of stay, pre

and post eMM roll-out. See pages 7-8 for further details

Item 7: Specific objectives or hypotheses.

Aim 1 - To quantify the safety and effectiveness of an eMM system to reduce medication

errors, ADEs and average LOS among paediatric patients using a stepped-wedge cluster

randomised controlled trial (SWCRCT) design;

Page 31 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Aim 2 - To assess the effects of the eMM on clinicians’ workflow and efficiency; and

Aim 3 - To assess the extent to which feedback of study results and subsequent



via a second SWCRCT. [Page 7]

Item 8: Description of trial design including type of trial (e.g., parallel group, crossover,

factorial, single group), allocation ratio, and framework (e.g., superiority, equivalence,

non-inferiority, exploratory).

The study is a stepped wedge cluster randomised controlled trial (SWCRCT) evaluating

differences pre- and post- eMM implementation.

Item 9: Description of study settings (e.g., community clinic, academic hospital) and list of

countries where data will be collected. Reference to where list of study sites can be

obtained.

The study will be conducted in Sydney, New South Wales and will occur at two large tertiary

pediatric hospitals from within the Sydney Children’s Hospitals Network. [Pages 7-8]

Item 10: Inclusion and exclusion criteria for participants. If applicable, eligibility criteria for

study centres and individuals who will perform the interventions (e.g., surgeons,

psychotherapists).

All patients on the study wards will be participants. [Pages 9-10]

Item 11a: Interventions for each group with sufficient detail to allow replication, including

how and when they will be administered.

Detailed in methods section.

Item 11b: Criteria for discontinuing or modifying allocated interventions for a given trial

participant (e.g., drug dose change in response to harms, participant request, or

improving/worsening disease).

See methods section

Item 11c: Strategies to improve adherence to intervention protocols, and any procedures

for monitoring adherence (e.g., drug tablet return; laboratory tests).

The intervention – the eMM will be mandatory for use in the study hospitals.

Item 11d: Relevant concomitant care and interventions that are permitted or prohibited

during the trial.

N/A

Item 12: Primary, secondary, and other outcomes, including the specific measurement

variable (e.g., systolic blood pressure), analysis metric (e.g., change from baseline, final

value, time to event), method of aggregation (e.g., median, proportion), and time point

for each outcome. Explanation of the clinical relevance of chosen efficacy and harm

outcomes is strongly recommended.

Outcome: Medication administration errors (MAEs), ADEs and LOS

See methods section

Page 32 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Item 13: Time schedule of enrolment, interventions (including any run-ins and washouts),

assessments, and visits for participants. A schematic diagram is highly recommended (see

Figure)

For Aim 1, the data collection at CHW will commence on 25th April, 2016, one week before

the eMM rollout will commence. Direct observations of medication administration will

continue until 10th

July, two weeks after the roll out is complete, while chart reviews will

continue into 2017, until they are completed.

Figure 1: Timeline for the SWCRCT, with the baseline commencing at 25th April

For Aim 2, data will be collected during the same schedule as the implementation.

Item 14: Estimated number of participants needed to achieve study objectives and how it

was determined, including clinical and statistical assumptions supporting any sample size

calculations

See methods section

Item 15: Strategies for achieving adequate participant enrolment to reach target sample

size

See methods section

Item 16a: Method of generating the allocation sequence (eg, computer-generated random

numbers), and list of any factors for stratification. To reduce predictability of a random

sequence, details of any planned restriction (eg, blocking) should be provided in a

separate document that is unavailable to those who enrol participants or assign

interventions

Randomisation of the 8 wards will be conducted by a person blinded to the ward identity.

[Page 8]

Item 16b: Mechanism of implementing the allocation sequence (eg, central telephone;

sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the

sequence until interventions are assigned

Page 33 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Randomisation will not be concealed. The study is a stepped wedge cluster randomised

controlled trial.

Item 16c: Who will generate the allocation sequence, who will enrol participants, and who

will assign participants to interventions?

The research team will determine the allocation sequence and communicate with the

hospital team who are responsible for the implementation of the eMM.

Item 17a: Who will be blinded after assignment to interventions (eg, trial participants,

care providers, outcome assessors, data analysts), and how

No one will be blinded to the assignment of the intervention.

Item 17b: If blinded, circumstances under which unblinding is permissible, and procedure

for revealing a participant’s allocated intervention during the trial

Methods: Data collection, management, and analysis

Item 18a: Plans for assessment and collection of outcome, baseline, and other trial data,

including any related processes to promote data quality (eg, duplicate measurements,

training of assessors) and a description of study instruments (eg, questionnaires,

laboratory tests) along with their reliability and validity, if known. Reference to where

data collection forms can be found, if not in the protocol

See methods

Item 18b: Plans to promote participant retention and complete follow-up, including list of

any outcome data to be collected for participants who discontinue or deviate from

intervention protocols

See methods. Retrospective review hospital medical records is the main source of data.

Item 19: Plans for data entry, coding, security, and storage, including any related

processes to promote data quality (eg, double data entry; range checks for data values).

Reference to where details of data management procedures can be found, if not in the

protocol

The methods section outlines the main data management processes. Information in the

individual ethics applications specified further details.

Item 20a: Statistical methods for analysing primary and secondary outcomes. Reference to

where other details of the statistical analysis plan can be found, if not in the protocol

Data analyses are specified in the methods section/

Page 34 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Item 20b: Methods for any additional analyses (eg, subgroup and adjusted analyses)

See methods section

Item 20c: Definition of analysis population relating to protocol non-adherence (eg, as

randomised analysis), and any statistical methods to handle missing data (eg, multiple

imputation)

See methods section. Analyses will be performed as per intention to treat.

Methods: Monitoring

Item 21a: Composition of data monitoring committee (DMC); summary of its role and

reporting structure; statement of whether it is independent from the sponsor and

competing interests; and reference to where further details about its charter can be

found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed

A formal DMC has not been convened, however members of the research team will have

roles in monitoring the collection and quality of the data. Funding bodies will not have

access to the raw study data.

Item 21b: Description of any interim analyses and stopping guidelines, including who will

have access to these interim results and make the final decision to terminate the trial.

N/A

Item 22: Plans for collecting, assessing, reporting, and managing solicited and

spontaneously reported adverse events and other unintended effects of trial

interventions or trial conduct

The project manager (MZR) and the principle investigator (JIW), as well as the site

coordinators (CM and TOB) will monitor any adverse events which occur and wil report to

the Steering Committee.

Item 23: Frequency and procedures for auditing trial conduct, if any, and whether the

process will be independent from investigators and the sponsor

The project will be undertaken in line with specific human ethics approval requirements.

Item 24: Plans for seeking research ethics committee/institutional review board (REC/IRB)

approval

This study has been approved by the SCHN HREC (EC00130), reference code

HREC/15/SCHN/370.

Item 25: Plans for communicating important protocol modifications (eg, changes to

eligibility criteria, outcomes, analyses) to relevant parties (eg, investigators, REC/IRBs,

trial participants, trial registries, journals, regulators)

Any proposed modifications to the study will be discussed with the project steering

committee, and where required modification from the SCHN HREC will be sought.

Page 35 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Item 26a: Who will obtain informed consent or assent from potential trial participants or

authorised surrogates, and how (see Item 32)

See methods

Item 26b: Additional consent provisions for collection and use of participant data and

biological specimens in ancillary studies, if applicable

Not applicable.

Item 27: How personal information about potential and enrolled participants will be

collected, shared, and maintained in order to protect confidentiality before, during, and

after the trial

See methods

Item 28: Financial and other competing interests for principal investigators for the overall

trial and each study site

The research team declare no competing interests.

Items 29: Statement of who will have access to the final trial dataset, and disclosure of

contractual agreements that limit such access for investigators

Members of the research team involved in the data analysis and the writing up of the study

results will have access to the data set. Access will also be granted to selected members of

the steering committee and the site coordinators, where appropriate. The main funding

bodies will not have access to the raw data.

Item 30: Provisions, if any, for ancillary and post-trial care, and for compensation to those

who suffer harm from trial participation

The intervention implementation is the sole responsibility of the hospital sites and was

intended to proceed regardless of the evaluation study outlined here.

Item 31a: Plans for investigators and sponsor to communicate trial results to participants,

healthcare professionals, the public, and other relevant groups (eg, via publication,

reporting in results databases, or other data sharing arrangements), including any

publication restrictions

Study results will be communicated to the staff of the two study hospitals in meetings and

briefing sessions, to be arranged in coordination with senior staff. The results will be further

communicated with healthcare professionals and the public, through publication of journal

articles, conference presentations and other appropriate means.

Item 31b: Authorship eligibility guidelines and any intended use of professional writers

Page 36 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from



Standard academic and ethical guidelines for authorship and publication of results will

apply.

Item 31c: Plans, if any, for granting public access to the full protocol, participant-level

dataset, and statistical code

The study protocol will be published and thus available to the public. There are no plans to

provide public access to the full dataset or statistical code at this time.

Page 37 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Decem


http://bmjopen.bm

j.com/

BM



ownloaded from


Date post:	05-Sep-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

BMJ OpenComplete List of Authors: Wesbrook, Johanna; Australian Institute of Health Innovation,...

Documents