Safety Culture and Uncertainty Over Device Behaviour:

The Limitations and Hazards of Telemedicine

Chris Johnson

University of Glasgow, Scotland.http://www.dcs.gla.ac.uk/~johnson

ISSC: 4th August 2003

• Medical errors in 45,000-100,000 deaths (US). • RTA=43,000, Aids=16,000. • Additional care cost $15 billion:

– 45% experience some medical mismanagement.– 17% suffer events that prolong hospital stay.

• Retrospective studies of patient records in NY State and Utah.

  Leape et al, 1984* Weighted proportion

of adverse events

Thomas et al, 1992 * Percentage of

adverse events

Operative 47.7 44.9

Drug related 19.4 19.3

Anesthesia related

1.1 1.3

Courtesy: Univ. of Virginia, Office of Telemedicine

•SE Virginia medical centres:

1 nurse monitors system; 49 remote patients; 5 ICUs at 3 centres.

• Staff 50-80% of ICU budget.

Courtesy: NASA Telemedicine Instrumentation Pack project

Motivation

“The benefits and costs of this blend of medicine and digital technologies must be better demonstrated before today's cautious decision-makers invest significant funds in its development”.

Telemedicine: A Guide to Assessing Telecommunications for Health Care Marilyn J. Field, Editor; Committee on Evaluating Clinical Applications of Telemedicine, Institute of Medicine

Haemofiltration Device

“What’s it saying?I don’t know it’s stopped saying it…”

“I’ve no idea what was happening…”

“What did you do?I don’t know, I just took it apart…”

“That must be a record. I think that’s worth minuting…I think the thing is to reemphasise that we did have it working for 30 odd hours…”Sister leading the Renal Core Group

“You know we said that their kidney’s not working as well as it should? Well this machine is going to do the job of their kidneys…”

A: MDR Report Identifier

B: Event Information

E: Professional information

F: Distributor Information

G: Manufacturer Information

H: Device Information

MDR Report Key MDR Event Key Report Number

Source Code Number of devices

Date receivedNumber of patients

Master Event Data File, Section A: MDR Report Identifier

MDR Report Key Manufacturer’sName

Master Event Data File, Section G: Manufacturer Information

Manufacturer’sAddress

Source Type

Date Manufacturer received report

MDR Report Key Made when?

Master Event Data File, Section H: Device Information

Single use device?

Remedial Action

Use code Correction number

Event type

Master Event Data File Format Identifier

MDR Report Key Device Event Key

Device Data File

Device Seq. Number

Device available for examination?

Brand Name

Generic Name

Age? …

MDR Report Key Patient Seq. Number

Patient Data File

Date report received

Sequence and treatment

Patient Outcome

MDR Report Key Text key

Text Data File

Text type

Patient Seq. number

Report date

Text

• Telemedicine changes relationships:– clinicians and technicians but also– device suppliers and device manufacturers.

• No common ‘safety culture’;– many incidents stem from poor

communication.

Findings from MAUDE: Safety Culture

Findings from MAUDE: System Uncertainty

• Introduction of telemedicine implies:– less clinical staff more technical staff;– technical staff don’t understand devices/procedures?

• Increasing reliance on vendor’s guidance:– vendors in turn rely on manufacturers;– communication often breaks down or is too slow.

• So technicians often resort to coping strategies.

Safety-Culture in Healthcare

• Strong ‘safety-culture’ implies shared concern for safety.

SINCE PLACEBO TREATMENT IS STILL ACTIVE IN REVISION 9 OF SOFTWARE, IT IS POSSIBLE TO UNINTENTIONALLY DELIVER PLACEBO TREATMENT.

IT APPEARS COINCIDENTAL THAT reporter USED THIS PARTICULAR PASSWORD…

X-RAY TECH MAY SOMEHOW MISTOOK DEFAULT PHYSICS PASSWORD "9999" FOR "4444", WHICH MEANS OPERATOR WOULD HAVE ALSO CONFUSED TREATMENT PASSWORD WITH PHYSICS’

(MDR TEXT KEY: 1490034)

• Complex nature of safety in healthcare applications:– Operator error? Physics Department Password? Developer

neglect?

Uncertain DiagnosisPATIENT HEART RATE OF 71 WITH CONSISTENT QRS (QRS waves monitor ventricular

contraction on ECG).

HOWEVER, BEDSIDE AND CENTRAL MONITORS INDICATED A Ventricular-Tachycardia ALARM (lower chambers of heart beating unusually fast).

ALARM EXTINGUISHED WHEN BEDSIDE MONITOR DISCONNECTED FROM CENTRAL, INDICATING ERROR GENERATED BY CENTRAL MONITOR.

CYCLING POWER ON CENTRAL STATION MONITOR RESTORED PROPER OPERATION.

BIOMEDICAL TECHNICIAN SAYS MANUFACTURER ACKNOWLEDGED THIS CAN OCCUR AFTER PERIOD OF CONSTANT OPERATION WITHOUT ANY POWER DOWN.

REPORTER UNDERSTANDS THIS TYPE OF DEVICE FAILURE HAPPENED IN 2 ICUs AT DIFFERENT TIMES.

(MDR TEXT KEY: 1505404)

Uncertain Diagnosis

• Support staff use diagnostic skills to identify causes:– selective disconnect finds problem in central monitor;– uncertainty shown by previous unreported failures in 2 ICUs.

• Further analysis finds:– problem if V-Tach alarm suspended at bedside; – alarm is then set at central monitor and cannot be cleared.

• Problem known to manufacturer but not in the manual.

• Subsequently corrected by software upgrade.

Uncertainty over New Technology

• EASITM software provides 12-lead ECG data on 5-leads to patient.

TELEMETRY TECH NOTED EASI 12-LEAD DISPLAY ON CENTRAL STATION FROM TRANSMITTER THAT WASNT EASI CAPABLE.

CUSTOMER ENGINEER REPLACED TRANSMITTER AND RELOADED CENTRAL STATION SOFTWARE, CONFIRMED ALL SIGNALS WERE CORRECTLY TRANSMITTED AND LABELED.

CUSTOMER DID NOT UNDERSTAND DIFFERENCE BETWEEN STANDARD ECG AND EASI.

CUSTOMER WAS RETRAINED TO FURTHER THEIR UNDERSTANDING OF DIFFERENCE. (MDR TEXT KEY: 1379795)

• Less electrodes reduce work for nurses, improves patient comfort.

• Social implications: clinicians and support rely on suppliers’ explanations.

• Symptomatic of system safety problems:– manufacturers gain insights that should be caught earlier in development.

• Retraining is proposed, no idea of systemic causes of human ‘error’?

DURING INVESTIGATION, ENGINEERS CONFIGURED A SYSTEM IN SAME SETUP AS CUSTOMER. FOUND MAINFRAME RECEIVERS CAN RECEIVE INCORRECT BIT TO MISIDENTIFY TRANSMITTER AS EASI

CAPABLE…

• Report doesn’t state how to prevent mis-configuration.

Uncertainty over New Technology

Uncertainty and Reliability

• End-user frustrated by device unreliability and manufacturers’ response:

SEVERAL UNITS RETURNED FOR REPAIR HAD FAN UPGRADES TO ALLEVIATE TEMP PROBLEMS. HOWEVER, THEY FAILED IN USE AGAIN AND WERE RETURNED FOR REPAIR…

AGAIN SALESMAN STATED ITS NOT A THERMAL PROBLEM ITS A PROBLEM WITH X’s Circuit Board.

X ENGINEER STATED Device HAS ALWAYS BEEN HOT INSIDE, RUNNING AT 68⁰C AND THEIR product ONLY RATED AT 70⁰C….

ANOTHER TRANSPONDER STARTED TO BURN…SENT FOR REPAIR. SHORTLY AFTER MONITOR BEGAN RESETTING FOR NO REASON… (MDR TEXT KEY: 1370547)

• Manufacturers felt reports not safety-related: – “reports relate to end-user frustration regarding product reliability (not safety)”.

• Telemedicine applications developed by groups of suppliers:– flexibility and cost savings during development, manufacture, marketing; – problems if incidents stem from sub-components not manufactured by

suppliers; – incident reports must be propagated back along the supply chain.

• Manufacturer states problems stem from subcontractors circuit board: – more problems after faulty board replaced, customer returns unit again; – connectors to PCB not properly seated but still passes acceptance test? – connector not seated completely during initial repair and gradually loosens

over time?

Uncertainty and Reliability

• “Fly-fix-fly” approach undermines attempts to improve patient safety.

• Confused dialogue between clinician, vendor, manufacturer…– End-user may see technical issues as form of excuse (eg PCB

connectors)…

• Device repairs not only rectify problems, they introduce new ones:– compounds end-user uncertainty and distrust of device reliability;– communication fails and shared safety culture erodes over time.

Uncertainty and Reliability

Uncertainty and Complexity

IN SOFTWARE RELEASE VF2, IF PATIENT IN "AUTOADMIT" MODE, PARAMETER DATA AUTOMATICALLY COLLECTED AND STORED IN THE SYSTEMS DATABASE,

IF THE PATIENT LATER REMOVED (BUT NOT DISCHARGED) FROM ORIGINAL BED/NETWORK LOCATION, DATA COLLECTION TEMPORARILY DEACTIVATED (EG DURING MOVE FOR TREATMENT).

PROBLEM PRESENTS WHEN NEW PATIENT ADMITTED TO SAME BED/NETWORK LOCATION, BUT ORIGINAL PATIENT WAS NEVER DISCHARGED WHILE CONNECTED TO THAT LOCATION.

NEW PATIENT ADMISSION BEGINS STORING DATA IN DATABASE APPROPRIATELY. HOWEVER, IN PARALLEL, DATABASE INCORRECTLY APPENDS NEW PATIENT DATA ON TOP OF OLD PATIENT'S DATA RECORD…

(MDR TEXT KEY: 1340560)

Uncertainty and Complexity

• ‘Wicked’ problem; difficult to recreate conditions for failure to occur.

• No problem occurred if:– patient data entered in AUTOADMIT not MANUAL. – patient returned to monitor after treatment if no new patient used it in meantime. – original patient reconnected to another monitor then no problem.

• Even once company found context of incident, hard to trace causes:– In previous versions, no more data collected until patient re-connects to new

monitor; – new distributed monitoring function creates relatively complex hazard.

• Company recodes software for all deployed monitoring systems.

Conclusions• Clinicians increasingly rely on technicians who are often:

– Extremely uncertain about device behaviour;– Poorly trained and equipped to identify causes of adverse events.

• On the supply side, manufacturers and distributors often:– Cannot recreate the failures that are reported;– Rely upon incident reporting to debug aspects of their systems.

• ‘Fly-fix-fly’ and basic unreliability of devices:– undermines communication and shared ‘safety culture’;– may ultimately erode clinical confidence in many aspects of telemedicine.

Conclusions• No ‘quick fixes’ but:

– Regulators need to focus on dialogue between manufacturers and users;

– Consider detailed training requirements for telemedicine before approval;

– Especially look at end-user maintenance and configuration issues;– Introduce training in safety and risk management for support staff?

• Things are only going to get worse…

Da Vinci, 1st robotic aid approved by the FDA: New York Presbyterian Hospital uses it on atrial septal defects. ‘Fly-by-wire’ technology, enhanced 3D ‘virtual’ display of site. Ave. hospital stay 7-10 days (traditional) now 3 days (N=17).

Roboscope:

Removing deep-seated brain tumours.

Robot holds endoscope and allows constrains surgeons motions to a specific region.

Involves MRI processing, ultrasound guidance, robotics and visualisation. (http://www.me.ic.ac.uk/case/mim/)

Questions?Thanks are due in particular to:

David Wright,Intensive Care Unit,Edinburgh Western General Hospital.

Barbara Holland,Paediatric Intensive Care Unit,Yorkhill Hospital, Glasgow.

Clinical Negligence and Other Risks Scheme,Scottish Executive.

http://www.dcs.gla.ac.uk/~johnson

0 2 4 6 8 10

Could the incident have been

anticipated by risk managers?

Could the incident have been

anticipated by participants?

How severe was the incident?

How much is such an incident

f eared by staff ?

How confi dent are you in

avoiding such incidents?

How risky was the incident?

How easy is it to control the

outcome of such incidents?

How visible was the incident?

How much eff ort is necessary

to avoid f uture incidents?

I nteraction

Human Factors

TechnologicalI ncidents badgood

0

1

2

3

4

5

6

7

8

9

10

Could the

incident have

been anticipated

by risk

managers?

Could the

incident have

been anticipated

by participants?

How severe was

the incident?

How much is

such an incident

f eared by staff ?

How confi dent

are you in

avoiding such

incidents?

How risky was

the incident?

How easy is it to

control the

outcome of such

incidents?

How visible was

the incident?

How much eff ort

is necessary to

avoid f uture

incidents?

bad

good

Scenario 1: Weight Dose Error (Human-Factors)

0

1

2

3

4

5

6

7

8

9

10

Could the incident

have been

anticipated by risk

managers?

Could the incident

have been

anticipated by

participants?

How severe was

the incident?

How much is such

an incident f eared

by staff ?

How confi dent are

you in avoiding

such incidents?

How risky was the

incident?

How easy is it to

control the

outcome of such

incidents?

How visible was

the incident?

How much eff ort

is necessary to

avoid f uture

incidents?

bad

good

Scenario 2: Infusion (Technological)

0

1

2

3

4

5

6

7

8

9

10

Could the incident

have been

anticipated by risk

managers?

Could the incident

have been

anticipated by

participants?

How severe was

the incident?

How much is such

an incident f eared

by staff ?

How confi dent are

you in avoiding

such incidents?

How risky was the

incident?

How easy is it to

control the

outcome of such

incidents?

How visible was

the incident?

How much eff ort

is necessary to

avoid f uture

incidents?

bad

good

Scenario 5: Doppler (FHR)

Low-Tech Reports…

A TECHNICIAN FOUND THAT TABLE WAS WORKING ACCORDING TO SPECIFICATIONS, NO COMPONENTS WERE BROKEN…

CAUSE APPEARS TO BE USER ERROR IN IMPROPERLY TIGHTENING HEAD SECTION THUMB SCREWS.

TECHNICIAN DISCUSSED IMPORTANCE OF PROPERLY TIGHTENING THUMB SCREWS WITH HOSPITAL BIOMEDICAL DEPARTMENT.

(MDR TEXT KEY: 1506987)

Higher-Tech Reports

DOSE CALCULATOR FUNCTION FOR A NITROGLYCERIN DRIP PROGRAM CHANGES FROM Micrograms (MCG) ON ONE SCREEN TO milligrams (MG) ON THE NEXT.

DECIMAL POINT IS NOT EASY TO READ AND DOSING ERROR MAY NOT BE NOTICED.

ALLOWS 10% OF THE DESIRED DOSE TO BE DELIVERED.

SOFTWARE CHANGES RELEASED THIS SUMMER. (MDR TEXT KEY: 1128586)