Continuing Quality ImprovementRadiation Oncology

Jeff Michalski, MD, MBA, FACRProfessor and Vice-Chair

Department of Radiation OncologyWashington University School of

Medicine

Modern RTRecent sophistication – large fraction

of modern treatment technology and practices developed in the past ten years

High technical complexityMultiple systems (software and

hardware)Limited to non-existent

guidance/standardsHigh pressureIncreased potential for catastrophic

failures

Errors in Radiation OncologyStaff and public

exposures Suboptimal treatmentsMisadministrations

Underdose Overdose Anatomical misses

Magnitude From few percent to

lethal doses From couple of

millimeters to complete misses

Regulatory Nuclear Regulatory

Commission Errors that do not

necessarily affect patients but have regulatory/legal consequences

Sources Staff Software Hardware

Random Affect one to few

patientsSystematic

Affect hundreds of patients

Potentially in a short period

BackgroundGlobal Problem

“…it calls into question the integrity of hospital systems and their ability to pick up errors and the capability to make sustainable changes.” Sir Liam Donaldson, Chief Medical Officer, Department of Health

Towards Safer Radiotherapy. London: The Royal College of Radiologists,

2008.

Radiotherapy Risk Profile, Geneva:

World Health Organization, 2009.

Error ReportingWe are not airline industry nor nuclear

powerPerfection in complex systems across

hundreds of diverse clinics is impossibleReporting systems for sake of reporting

alone are a great way to squander resources and demoralize staff

Error reporting as a part of broader process improvement efforts can be very valuable

Event ReportingMandatory (statutory)

Reporting required by lawNRC in U.S.State requirementsMainly concentrated on well defined

treatment delivery errorsGuidelines for near-miss reporting

typically not providedVoluntary

Mainly at institutional levelSome states in the U.S. have voluntary

reporting systems – utility for rad onc not clear

Errors and near misses tracked

Voluntary ReportingDependent on Many FactorsCultureReporting guidelinesReporting system Competence to interpret reported

data Willingness to implement, when

necessary, significant changes based on collected data and subsequent analyses

Ability to share the collected data and provide feedback

Reporting CultureIndemnity against disciplinarily

proceedings and retributionConfidentialityTo the extent practical, separation of

those collecting the event data from those with the authority to impose disciplinary actions

An efficient method for event submission

A rapid, intelligent, and broadly available method for feedback to the reporting community

Taxonomy and Event Classification

Event reporting should enable process improvement

This requires efficient processing and analysis of data

Submitted events must be classified and organized

Enables efficient processing, analysis, and communication of data and trends

Web-Based Reporting

System AcceptancePaper

System AcceptanceVoluntary Web-based

Mutic et al, Submitted for publication, Med Phys, May 2010

System AcceptanceVoluntary Web-based

Mutic et al, Submitted for publication, Med Phys, May 2010

Therapy

-2

0

2

4

6

8

10

12

14

16

18

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Jan-08

Feb-08

Mar-08

Apr-08

May-08

Jun-08

Jul-08

Aug-08

Sep-08

Oct-08

Nov-08

Dec-08

Jan-09

Feb-09

Mar-09

Time

Fre

qu

en

cy

IMPAC Prob

Port-film Prob

Delivery Prob

Customer/Patient Satisfaction

Dosimetry

-5

0

5

10

15

20

25

30

35

40

45

Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Jul-08 Aug-08 Sep-08 Oct-08 Nov-08 Dec-08 Jan-09 Feb-09 Mar-09

Time

Fre

qu

ency

IMPAC Prob

Tx plan Incorrect

Initial Chart to Phy late

Customer/Patient Satisfaction

Calc Prob

Dosimetry Board

Physics

-5

0

5

10

15

20

25

30

35

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Jan-08

Feb-08

Mar-08

Apr-08

May-08

Jun-08

Jul-08

Aug-08

Sep-08

Oct-08

Nov-08

Dec-08

Jan-09

Feb-09

Mar-09

Time

Fre

qu

en

cy

IMPAC Prob

Calc Prob

QA incorrect

Customer/Patient Satisfaction

Simulation

-5

0

5

10

15

20

25

30

35

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Jan-08

Feb-08

Mar-08

Apr-08

May-08

Jun-08

Jul-08

Aug-08

Sep-08

Oct-08

Nov-08

Dec-08

Jan-09

Feb-09

Mar-09

Time

Fre

qu

en

cy

IMPAC Prob

Scheduling Prob

Sim-film Prob

Customer/Patient Satisfaction

Therapy Nursing

Physics Simulator

Dosimetry

What are we doing?

Organizational learning Error and near miss reporting Use reported events to improve processes and clinical tools

StandardizationQAISQC – Based on reported errors and risk analysis

Electronic Automated Intelligent

Workflow tools - Based on reported errors and risk analysis

Electronic Chart Check Work Flow

RT TPS Database

OQA Database

R&V Database

Query Script

sdcm

rtog

ascii

txt

EcCk

TPS & RV

• Prescription

• Field Parameters

• Consistency Reports etc

Warning and Alerts

• Pending work

• Delivery problem indicators

• Independent checks

Protocol Compliance

• Data

• Approvals, etc

Statistics

• Benchmarking

• Future system improvements

Sample EcCk Report

Precheck scripts - workflow

Precheck: sample report

Plan Quality Benchmarking

Moore et al, Submitted for publication, IJROBP, May 2010

Dynalog-based IMRT QA ReportSummary Delta Fluence MLC Graph Error

Historical Performance of MLC

MLC Leaf Error Fequency of Failed DynalogQA

0

1

2

3

4

5

6

A1

A5

A9

A13

A17

A21

A25

A29

A33

A37

A41

A45

A49

A53

A57 B1

B5

B9

B13

B17

B21

B25

B29

B33

B37

B41

B45

B49

B53

B57

MLC Leaf Number

# o

f o

ccu

ran

ce L

eaf

erro

r

MLC Leaf Error Fequency of Failed DynalogQA

0

1

2

3

4

5

6

A1

A5

A9

A13

A17

A21

A25

A29

A33

A37

A41

A45

A49

A53

A57 B1

B5

B9

B13

B17

B21

B25

B29

B33

B37

B41

B45

B49

B53

B57

MLC Leaf Number

# o

f o

ccu

ran

ce L

eaf

erro

r

Tomotherapy: Sinogram based QA

The downside to the electronic worldAs implemented today

Record and Verify (R&V) system was originally designed to operate as an independent system (Big Brother)

Today these systems are integral part of the delivery process and the independent verification process is missing

If data in the R&V system is wrong there is much less opportunity and chance that the error may be discovered

Possible Solution

Electronic QA system (EQS)Independent system which compares

TPS data with the data in the R&V system

Greatly improves ability to compare initial data transfer and consistency of data in the R&V

Distributed Data Collection

Hospital Hospital Hospital HospitalHospital HospitalHospital

Each clinic with its own independent database

Centralized Database

Manufacturers

Regulatory Agencies

Professional Societies

ConclusionError reporting in technologically advanced

healthcare presents an opportunity to improve patient safety

Independent oversight of established systems is desirable to monitor safety systems and implement immediate intervention

Centralized reporting of errors and similar events (near misses, latent errors) is highly encouraged

Use reported errors to improve commercial systems