Cumulative Sum (CUSUM) charts for medical student peripheral venous cannulation;

development of a difficulty-adjusted CUSUM

Dr Harry MurgatroydSpR Anaesthesia

Leeds Teaching Hospitals Trust

Sumaiyah KolaMedical Student

Leeds University Medical School

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Runcie CJ. Assessing the performance of a consultant anaesthetist by control chart methodology. Anaesthesia. 2009; 64(3): 293-296

CUSUM Chart

Graphical presentation over timeUsed to determine

competency

Monitors performance

Learning Curves

Developed initially to look at industrial processes

Audit of quality clinical practice

Theory

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Set:definition of success / failureacceptable failure rateunacceptable failure rateerror

Collect:binary data

Algorithm:Score falls with successScore increases with failureGraphBoundary Lines

Failure seen as a rise in the graph

Success is seen as a fall in the graph

Each point represents a Each point represents a single cannulation attemptsingle cannulation attempt

Sequential cannulation attempts

Score derived using the CUSUM formula

Problems

•Patient variability

•Standard CUSUM

•Constant failure and success rates

•Risk adjusted CUSUM

•Complicated

•Not intuitive

•Failure rates•Set by user

•Can affect results considerably

• Aims– Proficiency at

intravenous cannulation

– Plot individual CUSUM charts

– Develop a ‘difficulty-adjusted’ CUSUM technique

• Time Scale: – 5 weeks

Medical student project

How to insert an intravenous cannula 101

Methodology

• Setting– Teaching hospital

– Elective surgical lists

• Procedures– Verbal consent

– Peripheral venous cannulation • Standard technique• Size of cannula appropriate to surgical

procedure

– Data collection• Success or failure• Appearance of vein• Size of cannula• Patient awake or anaesthetised

Conventional CUSUM

• Definition of ‘success and failure’

• Acceptable and unacceptable failure rates– Consultant consensus– Literature– 0.2 and 0.4 respectively

• Calculation– Published formulae– Error rates = 0.1– MS Excel

Williams et al. BMJ 1992;304:1359-61.

de Oliveira. Anesth Analg 2002;95:411-6.

Failure rates

Upper and lower boundries

0= failure,1= success

Data is plotted sequentially

Example if “IF” formula in Excel

Running total, CUSUM

Conventional CUSUM

Difficulty Adjusted CUSUM

• Difficulty score– Appearance of vein– Cannula size– Awake or anaesthetised

• Different failure rates– Two stages

• Vein adjusted• All three variables

– Intervention line• Average of all prior lines

Difficulty Adjusted CUSUMVein adjusted

Vein appearance

Acceptable failure rate

Unacceptable failure rate

Visible, palpable

0.15 0.3

Just visible 0.3 0.6

Table shows the scoring of each of the recorded variables. These are then added up to give the total score for the cannulation attempt

Shows the standard CUSUM formula, whilst incorporating different failure rates and scores dependent on the difficulty of the variables recorded.

Spreadsheet showing the final added up scores of the different variables. Hence including the vein, consciousness and cannula size. Using “IF” formulas the correct value of S is selected from the table above and the CUSUM then plotted in the same way as before.

Three variable methodology

Difficulty Adjusted CUSUM

• Successful difficult cannulation– Large fall in score

• Failed difficult cannulation– Small rise in score

• Successful easy cannulation– Small fall in score

• Failed easy cannulation– Large rise in score

Conventional CUSUM vs Vein DA-CUSUMAn example CUSUM and difficulty-adjusted CUSUM chart for student A

-5

-4

-3

-2

-1

0

1

2

3

0 10 20 30 40 50 60

Patient number

CU

SU

M s

core

Conventional CUSUM

DA-CUSUM

DA-CUSUM intervention line

Conventional CUSUM intervention line

Conventional CUSUM vs Three Variable DA-CUSUM

-5

-4

-3

-2

-1

0

1

2

3

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55

Standard

Including Difficulty

Three variable DA-CUSUM for two students

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55

Series

CU

SU

M Sumi

Dave

Monday Morning

POSITIVES of CUSUM

•Objective

•Simple Calculations

•Shows improvement in learners

•Early detection of poor performance

•Allows comparisons between students

LIMITATIONS of CUSUM

•Only technical skills

•Must have binary outcome

•Relies on logbooks and honesty of user

•Time consuming

•Open to manipulation

•Does not show improvements that do not change binary outcomeBolson S, Colon M. Int J Health Care Qual Assur

2000;12:433-438.

Kestin IG. BJA 1995;75:805-809.

Difficulty adjusted CUSUM

• POSITIVES

• Potentially corrects for patient variability

• Easier and more intuitive than other methods of adjustment

• LIMITATIONS

• Failure rates set by the user

• The more variables ‘corrected’ for, the more layers of estimation and inaccuracy

• Loss of statistical element of conventional CUSUM

Summary

• Easy technique– Handheld devices– Electronic logbooks

• Objective• Can be adjusted for patient

variability• Allows

– Charting of ‘learning curve’ – Comparison between practitioners– Identification of poor performance

?