04/21/23 Dr Andy Brooks 1

Lecture 3

London Ambulance System (LAS) 26 & 27 October and November 4 1992

FOR0383 Software Quality Assurance

04/21/23 Dr Andy Brooks 2

Facts about LAS ~ 1992

• LAS was the largest in the world, handling 1,300-1,600 ‘999 calls’ daily.

• £69.7 million budget• 2,700 staff• 305 A&E ambulances (Accident and Emergency)

• 445 PTS ambulances (Patient Transport Service)

• 1 helicopter• 0.5 million A&E journeys per year

04/21/23 Dr Andy Brooks 3

Deficiencies of manual system• Identification of exact location

– “About half-way along London Road…”

• Physical movement of paper• Maintaining proper vehicle status• Time consuming use of voice

– “Can you repeat that address please…”

• Identifying duplicated calls– “This sounds like a report of the same accident…”

• Dealing with call backs– “It has been 25 minutes, the ambulance is not here yet…”

• Identifying special incidents– “Could this be a tanker chemical spill?...”

04/21/23 Dr Andy Brooks 4

With Computer Aided Dispatch (CAD)

• a gazetteer provided telephone box identification

• no more paper

• timely updates of resource availability

• intelligent help to identify duplicates

• direct mobilisation of an ambulance on completion of an emergency call

04/21/23 Dr Andy Brooks 5

Some Inquiry Conclusions• software was not complete and not fully

tested and the backup was not tested• there were outstanding data transmission

problems to/from MDTs (Mobile Data Terminals)

• there was scepticism over AVLS accuracy (Automatic Vehicle Location System)

• the ambulance crews had no confidence in the system and had not been fully trained

“Train train and train again”Ian Tighe, 999 rescue, The Computer Bulletin October 1997

04/21/23 Dr Andy Brooks 6

• CAC (Central Ambulance Control) staff were placed in unfamiliar positions in the control room and could not easily work together to solve problems

• the system relied on near perfect information of vehicle location and crew/vehicle status

• no attempt was made to deal with the effects of inaccurate or incomplete data which led to an increase in the number of exception messages

Some Inquiry Conclusions

04/21/23 Dr Andy Brooks 7

Poor interface between crews, MDTs, and the system.

• the system failed to capture all of the data• ambulance crews sometimes failed to press

the correct status button due to the pressure of dealing with certain incidents

• there were radio black spots• ambulance crews sometimes failed to press

status buttons because of frustration• there were radio communication bottlenecks

during busy periods

04/21/23 Dr Andy Brooks 8

• missing or swapped callsigns• there were faults in handshaking routines:

MDTs and the system showed different status!• sometimes ambulance crews intentionally did

not press the correct status button or pressed buttons in an incorrect order (frustration)

• ambulance crews sometimes took a different vehicle to the one they had logged onto

• incorrect or missing vehicle locations• too few call takers

Poor interface between crews, MDTs, and the system.

04/21/23 Dr Andy Brooks 9

Unreliability, slowness and operator interface• the system fell over and screens locked

– staff were advise to re-boot

• there was a failure to identify all duplicate calls• exception messages were not prioritised• messages scrolled off the top of screens!• the software made resource allocation errors• there were slow response times for certain

screen based activities

04/21/23 Dr Andy Brooks 10

26 and 27 October 1992

• the system went live

• there were no paper records

• there was no manual back up

• the launch was across all of London

• resource allocators were separated from radio operators and exception rectifiers

• system proposed resource allocations were used

04/21/23 Dr Andy Brooks 11

• it was extremely difficult for staff to intervene and correct problems

• the system rapidly knew the correct location and status of fewer and fewer vehicles

• multiple vehicles were sent to the same incident or not the closest vehicle was sent

• the number of exception messages rapidly increased so that staff could not clear the queue

• each effect quickly reinforced the others– see the Cause/Effect diagram in the Inquiry Report

26 and 27 October 1992

04/21/23 Dr Andy Brooks 12

Ambulance crew frustration• crew frustration may have led crews not

to press the status buttons in the correct order

• crew frustration may have led crews taking a different vehicle to the one allocated to them

• crew frustration may have led to a different vehicle and crew responding to an incident

04/21/23 Dr Andy Brooks 13

Crash on November 4• after 27 October, it was decided to revert to a semi-manual

operation• computer-based call taking and the gazetteer were for the most

part reliable• radio voice channels were available to resolve any mobilisation

misunderstandings• additional call taking staff were allocated to each shift• but shortly after 2am on November 4th, the system slowed

significantly and then locked up• attempts to reboot the system failed• a decision was made to revert to a manual system with voice

mobilisations

04/21/23 Dr Andy Brooks 14

What went wrong on November 4?

• a minor programming error left a piece of code which used up a small amount of memory within the file server every time a vehicle was mobilised

• after 3 weeks all the available memory on the file server had been used!

• normal testing would probably not have caught this particular error

• the fallback to a second server was never implemented for the semi-manual operation!

Response times 26/27 October(The ORCON specified response time from taking the call and arriving at the scene is 14 minutes.)

04/21/23 Dr Andy Brooks 15

Calls and average ring times 26 October(Average ring times peak at 10 minutes!)

04/21/23 Dr Andy Brooks 16

Total A&E Patients Carried, October(October 26/27 were normal in terms of demands on LAS services. Excessive demands was not responsible for the poor response times.)

04/21/23 Dr Andy Brooks 17

Calls recorded by call logger(On October 26/27 the number of calls were within the range within which it can be predicted 95% of calls will fall.)

04/21/23 Dr Andy Brooks 18