The Experience Conundrum: we want it, we lost it.

Patrick Hudson Tim Hudson

Hudson Global Consulting Good morning ladies and gentlemen. Before I get started I want to express my gratitude to the Flight Safety Foundation and CAAS for inviting us and to you, the audience, for listening, although you may have changed your minds by the end. [Slide 2] When I first flew in 1954 no one would have been that surprised if I hadn’t reached New York, my destination from London in a Constellation, after two stops for refuelling - Prestwick and Gander. People would have been very upset, maybe, but not completely surprised. I came back by boat, by the way. Today we regard flying as one of the safest means of transport and crack jokes about the trip to the airport in a car as being much more dangerous. How did we get there? Well we learned a lot about how to design, construct and maintain aircraft – the technology has improved immeasurably. But we also learned how to fly safely, both individually, as pilots and crew, and organizationally, as the world-wide aviation system. [Slide 3] [Slide 4] Today the system is so safe that we startle when something goes wrong. Whole types of accidents, like CFIT and Approach and Landing accidents, have reduced not just in absolute frequency, but also changed in relative ranking. The Flight Safety Foundation rightly deserves to take significant credit for these improvements. What is left are a small number of incidents, some disastrous, but also some successfully managed. They often defy categorisation, they currently have no acronym, except maybe Loss of Control (LOC), but they are weird. [Slide 5] By weird I mean we and those directly involved never conceived of them happening, even if in hindsight we may fool ourselves into thinking we saw them coming – a good example of hindsight bias. [Slide 6] The sort of disasters we are talking about include Air France 447, Asiana 214, Turkish Airlines TK 1951 and Colgan CO3407, while the successful recoveries include the Miracle on the Hudson, QF 32, BA 38 and Air Transat 236. All of which are you say, “that can’t happen today”. We do still have accidents of the ‘old’ sort, but the weird ones increasingly stand out. WEIRD can be an acronym – Wildly Erratic Incident Resulting in Disaster - for those types of incidents, whether disastrous or not. What makes the difference, in these sorts of incidents, between disaster and relief? When we read the reports, but more often the commentaries by experts, we see use of the word experience. The pilots while

technically competent, lacked or possessed respectively the experience either to avoid danger or manage what was almost always the unexpected consequence of some other failure. Yet all the pilots had enough hours to be considered sufficiently experienced; today we realise that while the number of hours logged used to be a good proxy for experience, today that link has become almost completely broken. In Canadian offshore helicopter operations, pilots flew an average of 450 hours a year, of which 14 were hands on. We don’t expect long-range commercial flying to be much better. [Slide 7] In another industry, shipping, we can see the same problem, exemplified by one Capt. E.J. Smith who said with absolute assurance:

“I cannot imagine any condition which would cause a ship to founder. I cannot conceive of any vital disaster happening to this vessel. Modern shipbuilding has gone beyond that.”

Captain Smith was, of course, the master of the Titanic. The solutions that are proposed have ranged from more or better instrumentation, such as angle of attack meters; more reliance on automation, less reliance on automation; demanding a reversion to much more hands-on flying, getting rid of pilots altogether. [Slide 8] The lack of a consistent message suggests we don’t really know what to do next. After Colgan the US Congress, a highly respected fount of wisdom on flight safety, may have spotted something and set a 1500 hour minimum flight hours requirement for commercial pilots. But the pilots already had 3379 and 2244 hours. The standard explanations for incidents still seem to come down to pilot error; all contain references to one or another breakdown in CRM, today’s acceptable expression for old-fashioned pilot error. There is an underlying thread that goes “A more experienced crew wouldn’t have had that happen to them, they personally would have followed the procedures, detected and solved the problem earlier etc” [Slide 9] As Tigger might have said: “The wonderful thing about hindsight, is hindsight’s a wonderful thing”. Now, I agree that a more experienced, dare I say cynical, flight deck crew might never get into these sorts of problems and, if they do, would come out of them better. But what we are really challenging is the idea that today we are getting the experience hindsight demands from the ways we go about developing pilots. Let’s distinguish competence, what we can define, examine and generate with formal systems and to which we devote a lot of resources, and experience, what we get when we get scared and promise never to repeat – the squeaky bottom stuff. Do you remember your first skid? I’ll bet it was after you passed the driving test, so you were officially competent, and you learned more than a whole heap of lessons, you gained experience. What we have come to expect is

that training delivers competence that can be equated to experience, but maybe it’s not as simple as that. [Slide 10] Likewise, experience may not always get us competence, the two can and should be distinguished and conflating them may have consequences that we would rather avoid. [Slide 11] If we graph them then we might have competence and experience on different axes. We expect the two to go up together, but there is a limit to competence training. [Slide 12] What we wanted can be called the experience gap. So how might we get it? By separating competence and experience we can examine both in their relevant domains and understand how we need both. [Slide 13] This should allow us to better understand the shortcomings in the current system and enable us to explore how we could acquire the sort of experience we are talking about without, to put it crudely, having to have the squeaky bottom moments, with all the risk they imply, that used to give us the experience we now lack. In other words, how can we acquire experience without actually undergoing any of the hazardous situations that used to be what gave us the experience we crave? Can we also systematise the acquisition of experience rather than the haphazard learning driven by events as they happen? In short we want to solve the experience conundrum. Today’s rigorous attention to competence and technical excellence has made air travel one of the safest forms of travel, but also robbed us of the experience to successfully navigate the curveballs that are still occasionally thrown our way. In other words, we’ve solved most of our problems but created some new ones in the process, or rather the problems that we used to overlook have now become salient and in need of attention. What got us where we are today won’t get us further. To do this I have to take a diversion into some theory, stand back and look at how accidents are caused. I must stress that we are talking about an extremely small number of events in terms of the number of sectors being flown today. Next we want to look at what we are actually training. We want to examine decision-making that goes beyond CRM and simple Situation Awareness. Finally we want to make some proposals about the sort of areas we will have to look at and approaches we could consider, gathering insights from outside aviation, which is usually held up as the paragon example for other hazardous industries, particularly medicine.

Accident causation models [Slide 14] Most of the accidents we could possibly have are really pretty simple; their causes are what we call deterministic (If A happens, then B will follow) and are linear (the effects of actions increase as a linear function of the strength of the action). Very crudely - Push the stick down and the plane goes down; the more you push, the more it goes down. I have argued elsewhere that 80% - 90% of the possible accidents you could have are of this sort – we call them Type I. We can use the word cause without losing sleep over it at this level. Note I said possible, not probable, because we have learned what to do to prevent the possible becoming the probable - most of these possible accidents never get to happen. [Slide 15] But that leaves 10%-20% over and these are still deterministic but the cause-effect relationship is non-linear, we can better talk about contributing factors and this is really what Swiss Cheese, the Organisational Accident, is about. Holes have to be big enough to let the hazard through, but once they are really big, size doesn’t matter any more. These accidents, called Type II, probably cover 80%-90% of the residual, so totalling 96% to 99%, leaving us with between 1% and 4% over that we don’t quite catch. The remaining possible accidents have non-deterministic causes, they only make the results more or less likely, as well as being non-linear – These we call Type III. [Slide 16] Here we can talk at best about factors that impact on the results, avoiding the cause word altogether. Fatigue is a good example of what we mean by non-linear, non-deterministic causation. [Slide 17] In hindsight, looking back after an incident, all the factors look pretty simple, deterministic and linear, but looking forward in time, they may be anything but straightforward, making it almost impossible to predict what will actually happen in the future. Type II is a reasonable approximation even if all accidents are really Type III. Likewise, Type I is a reasonable approximation of Type II, so almost everything can be pretty well approximated as Type I accidents, with a few inconvenient details we can safely ignore most of the time. Hindsight involves looking back and accepting the approximations, glossing over the details. But the reality is that those details are just what we are now chasing. Prevention is better than cure, but it is increasingly hard to imagine what is needed for that small, weird, percentage to prevent accidents we don’t, and maybe can’t, see coming. Now, WEIRD accidents may form 1-4% of the possible accidents by frequency, they are uncommon. But it is not the absolute numbers of accidents, but the area of the space of possible accidents, scenarios, that

we are facing. The reduction in the absolute numbers necessarily implies a significant increase in the relative proportion of incidents that we now label as weird. [Slide 18] We can draw a circle and then distinguish segments separating out the different accident types (CFIT, ALA, mid-air, loss of control in flight, runway incursions etc). We can then draw circles outside the inner circle where the segments have the same type. [Slide 19] The innermost circle encloses the Type I scenarios, the next includes Type II. [Slide 20] This is best seen in three dimensions because it is the height that represents the frequency with which scenarios, points on the distribution, may occur. Now we get a Mexican Hat. [Slide 21] [Slide 22] The inner circle covers possible incidents that are common. We can expect them and therefore design aircraft and systems, and train to prevent them from actually happening. [Slide 23] If we scatter scenarios evenly across the whole of possibility space, there are only a limited proportion of scenarios available towards the centre. [Slide 24] The problem now becomes clear, the number of distinct scenarios actually gets larger, a lot larger, as we go out, even though they individually may be very unlikely to actually happen. Let’s examine the consequences of this representation. Competence and experience In the early days of high-hazard operations, like surgery and aviation, we relied on getting the experience to get the job done while not killing others, or ourselves. The more experience the better. We got the surgeon as hero, and still have to this day, at least in the eyes of their patients, or the ones who survive. [Slide 25] We had the highly experienced pilots who had flown everything, at least from a Dakota onwards, and collected thousands of hours in the process, scaring themselves as they went. We might crudely define experience as the number of exposures to potential disaster that an individual had successfully survived. And that became the problem. We had to have a high chance of not surviving to get those with the experience who had survived. As society no longer saw aviation as inherently dangerous, the implicit demand was created to ensure that squeaky bottom moments became a thing of

the past. But that means we have to get our experience some other way, and safely at that. Aviation led the way, going from ‘crash and learn’ to teaching pilots the skills we knew would keep them, and their passengers or cargo, out of harm’s way. First of all we codified the basic skills, stick and rudder, then we developed ways of exposing young pilots to situations that an instructor could create and manage as a lesson. Losing one engine on take-off is something we do in a simulator without a qualm, even if in real life it’s something else. [Slide 26] The arrival of flight simulators, first in the 2nd World War with the Link trainer, and later with simulators of increasing sophistication and verisimilitude, allowed pilots to acquire both the basic flying skills and type-specific knowledge, all without bending the aircraft or putting anyone at risk. Much later, simulators even appeared in medicine, especially for anaesthetics. [Slide 27] Doctors could practice hands-on rather than just absorbing the lessons of more experienced colleagues while hoping they didn’t kill the patient as they learned. Medicine, to this day, still relies on a pretty tight approach that leaves little room for error – See one, do one, teach one – but a crucial lesson that medicine has been learning is that surgeons and hospitals need to carry out substantial minimum numbers of operations for them to have enough experience to cope well with the unexpected. With the increase in training sophistication we appeared to have cracked the problem. We expect to gain experience without having any of the blood and much less sweat and tears than we used to have. We analyse the situations that we wish to be able to manage, usually based on the industry history of the most frequent incidents, and we create training and testing opportunities to ensure that pilots can handle them safely, dare I say, competently. These are the ‘inner circles’ of possible incidents and what we do to prevent those possibilities. [Slide 28] But what about the outer circle? We don’t train for them because: [Slide 29]

1) There are so many distinct scenarios, many or all are weird; 2) We can’t afford the resources to train for all of them; 3) We may not even be able to imagine them happening; 4) They are extremely infrequent, anyway; so, 5) We hope, that if we can manage and train for the inner circles,

the rest will generalise for a sufficiently competent pilot. Now we run into a problem that we still have to this day. The problems we train for, in simulators, are almost exclusively technical in origin and are reduced to drills we can learn and reproduce. And this is good. The improvement in general technical flying competence this allows forms a major reason why flying is so much safer today than it ever used to be. [Slide 30]

The identification of technical problems has always been at the forefront for accident investigators. This is not surprising as a major technical problem can bring down an aircraft regardless of how good the pilots are or what they manage to do. But we have learned that purely technical competence is not enough, accompanying the ‘hard’ issues that could be identified as causing accidents, we learned that there were a great many ‘soft’ issues that contributed to incidents - and so CRM and CRM training was developed. With CRM the skills required are soft, cognitive, rather than hard, but also the problems the skills are intended to manage are much more diverse and numerous in the second ring. Purely technical problems, once identified, can be left to the manufacturers. The pilots’ interactions with most technical failures can be designed and taught, often as drills, such as what to do with an engine fire. Competence is what we define as the ability to manage those specific and foreseeable technical problems, usually by rapid response to emergencies, act first, think later. However, in a ‘hard’ industry, with a long tradition based on engineering and physical flying, soft skills are hard to identify, hard to teach, hard to assess and, worst of all, liable to evaporate just when you need them. The approach to competence assessment is critical. Assessment has always been rigorous but, while you can assess technical performance in a simulator with little room for disagreement, the soft side proves to be more contentious. Performance on hard problems can be assessed objectively, pretty much like maths; performance on soft problems like “What’s going on?” in contrast, is much more subjective, like marking an essay. This may be one reason why the soft side has received less attention in terms of skill acquisition. If you fail a competence test, you are grounded. A hard metric is hard to miss, but there would be serious personal issues associated with people being grounded because they failed a test when the scoring is open to discussion. As a result, CRM is often assessed by whether you turned up on the day and, maybe, stayed awake. Even assessments in simulators, like LOFT, or line-checks, are often seen as arbitrary. Programs like NOTECHS, scoring behavioural indicators, have attempted to confront this problem, but it remains. Anyway people can usually behave ‘properly’ when there is someone observing them, and maybe even when they are observing themselves. There is really only one objective test of CRM, and that’s the hard one, when one or more real problems force themselves on the crew and there is no capacity for introspection. [Slide 31] A limited set of drills, which need to be practiced to the point of automaticity, covers Type I scenarios well. They are overwhelmingly technical. Early success with conquering Type I’s opened up the

realisation that we needed to manage the soft problems that now became apparent. The reason we continue to place so much emphasis on CRM is that CRM is what provides an essential antidote for most Type II accident scenarios. Now this is where I begin to have problems. We have expected current approaches to CRM to generalise to Type III. This may not always be the case.1 [Slide 32] The first limitation I can identify with today’s approach to CRM is that it is typically passive –talk and chalk multiplied by death by PowerPoint. Good for awareness, but not even for knowledge – real human factors is degree-level knowledge2. But what is only hinted at is the difference made by indulging in real, physical action. We do this in the simulator, but hardly get involved past some limited LOFT scenarios. Simulator time is precious but costly these days. But we know that physical involvement makes a massive difference: consider learning to drive by being in the passenger’s seat, even more so learn how to control a sudden skid on black ice in heavy traffic at dusk in a rental car by listening to a powerpoint presentation. The reasons for this limitation we cover elsewhere. Everything else has worked so well we have left the last 1% in the hope it will work out all right on the day. Unfortunately the accidents I talked about at the start were those where it didn’t. [Slide 33] But this is, to be honest, a luxury problem. I am only worrying about the Type III weird accidents because aviation has been so successful in preventing the simple Type I and then the more complicated organisational Type II accidents. Weird accidents either never got the chance, because the system was still pretty fragile, or were so rare that they got lost in the statistics. 1% is not a problem when you’re facing 20%, it’s only 5% of the 20% and 1% of 100%, but it is a problem when 100% of 1% is what you’ve got to worry about. Society has gone and moved the goalposts: what people once would have shrugged over has now become completely unacceptable. We are trying to prevent the ‘new’ accidents using more of the same ‘old’ tools. The bad news seems to be that we can’t throw out the old approaches; we need to do more and hope that it helps reduce the resources for the ‘old’ stuff when we know how to do it well. [Slide 34] Now this is where experience comes in. CRM training delivers the experience of sitting through a CRM course, but does everyone therefore learn what to do when unexpected and novel situations suddenly arise in a way that can be called skilled? As we see some do, some don’t. I expect that when we go back over the history of pilots like Dick de Crespigny or ‘Sully’ Sullenberger or the pilot of the Air Transat 330 who also happened to be a glider pilot, we find that their career was sprinkled with the unexpected and the novel that they never got from

courses. They will have been quite unlike Captain Smith, whose career had been all plain sailing. The whole point about Weird scenarios is exactly that, they are unusual and unexpected. The chances of coming up against an exact scenario is vanishingly small, so what you have to learn are generaliseable skills. Let’s see how many scenarios we might have to face. [Slide 35] The Rule of Three is a simple heuristic I developed for helicopter operations back in 1992, actually that was when I did my CRM course in Brunei. The idea is that we have about seven distinct dimensions such as crew, aircraft, ATC, weather, plan, etc that can be scored as green (go), orange (proceed with caution) or red (stop). If any three dimensions are orange – marginal – then the rule says that should be counted as a red. Either you ground the flight or manage one or more back into the green. [Slide 36] The important point here is that we have a 7-dimensional space within which we can generate massive numbers of scenarios, most of them seen as ‘impossible’ or benign until the day a bad one happens to some poor crew. With just 5 options per dimension we have 78000 scenarios, with only a few all ‘in the green’ or with only 1 or 2 oranges. With 10 per dimension, not unreasonable, we get 107, ten million! Some of these, quite a small restricted number, will be the standard, high frequency ones we recognise and have designed, built and trained for. The majority are benign but many will be the stuff of nightmares, the flight from hell, or weird accident scenarios if you will. Decision-making is difficult and there is a considerable scientific literature about it. The early approach, rational and normative – “how you should make decisions” – involves generating all the possible options, weighing them in a rational way and choosing the best, optimal, option, followed by execution. Fine if you have all the time in the world and doesn’t need much practice as a distinct skill. Back in the 60’s Herb Simon, a Nobel Prize winner, pointed out that we don’t often have the luxury of time and resources, instead we satisfice –picking the best under the conditions, maybe just good enough. If we can only examine 10 of a 1000 options, we pick the best of the ten, we can do no more. In the 70’s Danny Kahneman, another Nobel Prize winner, and Amos Tversky pointed out that much decision-making is anything but coldly rational, influenced by a whole range of cognitive biases3. What we are beginning to realise is that we need to be aware of what biases us, a meta-cognitive skill and this is intimately related to what experience gives us. [Slide 37] You may remember I was a bit rude about medicine; it’s not called practicing for nothing. These days medicine is continuously being told to look at aviation for a shining example. It’s true they don’t do the mundane things as well as they could, the horrendous patient safety

data supports this. They hardly use simulators even today, they learn a lot on the job, driven by what walks in the door rather than systematic analyses of what goes wrong. They don’t report if they can help it. They’re heavily laden on experience. But there is one area where they should be the shining light and that is trauma medicine. Situations which may arise suddenly, where the first interveners almost certainly won’t be senior trauma surgeons, where the diagnosis may be a mystery and where if you don’t get a move on, the patient dies. ATLS stands for Advanced Trauma Life Support and is designed for just the kinds of situation I have been saying aviation isn’t doing too well at - and it’s been around since 1980. [Slide 38] Some of the ATLS training looks suspiciously like a good CRM course, yes they have lectures and discussions too, but most of the time they get out of their chairs and physically practice scenarios, using dummies but also real actors (usually medical students). One comment I have found in ATLS training is that, after the book learning, when they try the first scenario in practice they find just how hard it is to put that knowledge into practice. In the active part of the training all too often things don’t go to plan, just like real life, and they have to find their way out. Assessments are frequent, often based on video shot during training – doctors are used to being assessed on the job these days. You can fail. The scenarios aren’t just drills, they are intended to extend the participants’ understanding of all the things that can go wrong, and always when you don’t want it. They have a lot of different scenarios to choose from. Sounds a bit like some of the weird accidents we want to manage better doesn’t it. Physical involvement, with some skin in the game, using a wide range of scenarios makes for more experienced individuals and teams who are there to save your life when things really go wrong. Why haven’t we heard this before, well I think they’re too busy saving lives to worry about academic arguments? So what can we actually do? Extra sessions in simulators, unfortunately, would cost money, especially as full flight simulators now cost a significant proportion of the real thing and times are already booked to the hilt to make the number of standard hours required by regulators. And that is good, that training really works. More realistic CRM training, with hands-on activities and real assessment, would also be more expensive and in the eyes of most regulators would be even harder to certify. Today’s CRM training also works pretty well, even if it could be improved. These approaches are certainly necessary, but we’ve done so well so far on what we already do, don’t we. Well we are arguing yes, for the Type I and Type II accidents we have succeeded in containing, but they are weak, even ineffective, with the weirdos. Now they are so infrequent we used to be able to ignore them, but the goalposts have been changed on us; in relative terms Loss of Control in flight, which forms the majority of weird accidents, has now overtaken CFIT as killer #14.

Two high-hazard activities exemplify the need to gain experience without running the risks of the real activity. [Slide 39] High speed contact sports, I might mention rugby union, cannot be trained for in the classroom past a certain point, yet if a player trains by getting real experience, he may not be able to play the game because of injury. Fitness and non-contact drills get you so far, delivering competence such as ball skills and knowing the rules of the game, but the difference between winning and losing is gained past that point, on the pitch. [Slide 40] Formula 1 is similarly limited. Practice on the circuit is actually severely limited for a variety of reasons and while you can learn the geography of the track on a simulator, that is nowhere like a real race, necessary but never sufficient. What top-class rugby, Formula 1 and trauma medicine have in common with aviation is the need to experience a wide range of scenarios without having to use expensive simulators, that are better reserved for engraining rapid emergency responses to the more likely and mundane problems. But how can we do this in an industry like commercial aviation? [Slide 41] One possibility is to use much cheaper simulators. We can put pilots into a whole range of scenarios without requiring the services of a full flight simulator. But this can quickly degrade into what we do today, requiring more expensive time spent in the training centre and following preset drills. The problem that has to be faced is that the number of possible scenarios is vast and while in foresight they will often seem unlikely or incredible, only in hindsight will we have usually seen them coming. What we need is to find ways of generating multiple scenarios and experiencing them at, essentially, no cost. We want to have a population of pilots who go past mere awareness, and even past knowledge, that there are scenarios out there that they may never encounter, but they had better be ready when they come. These pilots should be capable of standing back and working it out as it happens. [Slide 42] What top sport exponents use is a technique call Visualisation. You run a play, a sequence of corners, part of a match or a race even in your mind. A conscious disciplined exploration of a scenario, one you can run and rerun without getting hurt or damaging your car. Done properly this is no mere daydreaming, but a mental activity that, surprisingly, can be shown to have effects equivalent to having the physical experience. This is something in common with the trauma training with physical involvement and time stress. Experience with enough scenarios, which embeds itself much more effectively than sitting through a barrage of

powerpoint slides, can provide the basis for pattern recognition of the elements of scenarios, even those not specifically encountered. Experience with them can provide what we call meta-knowledge, knowledge about knowledge, that can be used to help in the specific diagnosis and response selection that weird scenarios demand in real life. This kind of understanding of the space of possible accidents, as a whole, can provide a firm basis for preventing mistakes & associated violations (i.e. we thought it was the best thing to do at the time) that worry us. So, how can we generate enough scenarios to be useful, because what we need is bulk and variety? We need imagination, a commodity hard to regulate for but I am sure most pilots have some. We can use the Rule of Three to generate weird combinations. We can probably better do it together, two or three of us, rather than trying to generate the unusual on our own. Where can we get two or three pilots together for long enough to come up with some good scenarios without calling them in and putting them in the training centre? Two locations come immediately to mind, but there will be more. The first is on the flight deck, whiling away the time by indulging in some active imagination of how things could go wrong, trying to come up with weird but feasible combinations of factors that a malign world might throw at you. The best can and should be recorded. People could share their wildest dreams and vote on the best, all ways of spreading the knowledge around the wider community. But, you will say, long-haul may have the luxury of pandering to their imagination, short-haul, especially LCCs, are too busy flying into and out of a multiple of airfields, negotiating a wide range of ATC regimes in places like Europe. And you would be right, but they are the ones getting the experience, albeit the hard way. [Slide 43] The second is a location where many pilots find themselves, and where they already show this kind of behaviour so they need little extra training: the bar or the gym after a long day’s flying. [Slide 44] Wherever two or more pilots are joined together they will talk about flying, let’s make use of that. Anyone who says, “that’s what we do already” is already agreeing with me, now the rest of the population and permission to turn speculation into officially recognised training! And then, when we have some weird scenarios, we can visualise them on our own. An individuals can visualise how they could recognise the situation, what would you do, how would the scenario develop? Just like the rugby game or the Formula 1 first lap. This can be done at home, on the flight deck, in a hotel room as you fight jetlag. It’s not as easy as it sounds, but it pays off; ask Lewis Hamilton or Seb Vettel. [Slide 45]

This is hard to regulate, but just because we can’t yet define assessment methods for use of scenarios doesn’t mean we shouldn’t try. So, in conclusion, I think we need to confront the small number of accidents we still have, but not by more of the same. We have to take up the challenge of what our success has created, a 1% problem. We have to consider changing what we do, and how we do it, while ensuring that we don’t throw the baby out with the bathwater. I think we can do it, and from my experience, tackling that last 1% can be made to be fun. Many of you may already be doing just this, inventing scenarios, visualising what to do about them, in my book that probably makes you more experienced than your less imaginative colleagues, regardless of their hours. All we need to do is to bring everyone else up to speed. [Slide 46]

1 Look at how we actually train the skills and ask what are the skills we may be missing. First of all everything I have seen, including the CRM training I got when I pretended to be a helicopter pilot in Brunei, is classroom based. In fact EASA recommends teaching off-site although there are concessions to simulator training, whenever practicable (AKA rarely or never as it is not mandatory). EASA state:

1. (1) CRM training should reflect the culture of the operator as well

as type of operation and be conducted by means of both

classroom training and practical exercises including group

discussions and accident and serious incident reviews to analyse

communication problems and instances or examples of a lack of

information or crew management.

2. (2) Whenever it is practicable to do so, consideration should be

given to conducting relevant parts of CRM training in FSTDs that

reproduce, in an acceptable way, a realistic operational

environment and permit interaction. This includes, but is not

limited to, appropriate line-oriented flight training (LOFT)

scenarios conducted in FSTDs.

3. (3) It is recommended that, whenever possible, initial CRM

training be conducted in a group session away from the pressures

of the usual working environment so that the opportunity is

provided for flight crew members to interact and communicate in

an environment conducive to learning.

2 Modern CRM training, I am afraid, has all too often become a drill exercise, reduced to the same methodology as technical drills. While it is accepted as crucial, it doesn’t to my mind receive the attention it still needs, in particular in the area of decision-making and handling of emergencies. I am fairly happy about the inter-personal and social issues; it’s what is needed even when those are good that concerns me. People also get taught so that they know what they should do, but what is often missing is the assurance that, after the training, they will actually behave in the best, or at least a good, way. In terms of a skills hierarchy most pilots attain a level of awareness of the problems and potential solutions (“I know it exists”), some knowledge of accident causation and human factors (“I know the theory, up to a point”), but little in the way of skill (“I can do it in practice”), let alone mastery (“I can teach others”). CRM has been extremely successful in creating awareness and having a crew population aware of the social and cognitive issues that has become the culture, especially once the old dinosaurs became extinct. 3 Biases include availability – what comes to mind first; anchoring and adjustment – where we start determines where we go next; regression to the mean – people are overly sensitive to the rule of small numbers – etc. There is a scientific literature about how people make good

decisions under non-ideal circumstances; we need to make more use of that. 4 The problem I face with weird scenarios is that there are so many of them, most are not even technical and even when we have perfect inter-personal interaction, the problem lies in recognising what is really going on and selecting what actions to take. In his studies of expert decision-making Gary Klein discovered that real experts don’t apply the rational decision-making model, but recognise the problem and know what to do pretty much immediately. Experience has taught them to recognise what is happening as one of a multitude of scenarios, pattern matching the situation and generating a limited number of actions that can be tested mentally. The few possible actions are run against the problem and the best selected as quickly as possible. This turns out to be a very efficient use of the two systems Danny Kahneman talks about; System 1 pattern matches the problem, System 2 selects the best course of action. To do this, however, we need sufficient experience in what happens and what and how things can go wrong, also called requisite variety. Without a basis of patterns to call on, our decision-making processes are hopelessly slow and can be hijacked by a variety of cognitive biases. Competence can be developed but is essentially, maybe sensibly, restricted to the most likely scenarios I described, as covered by Types I and II.