The old saying goes that there are only three certainties in life: “Birth, Death and
Taxes”. When it comes to Risk Management, we may need a fourth: Risk Matrices.
Risk Matrices1 are one of those management tools that have been around for so long that very few
people recall a time when they weren’t. They appear in pretty much every industry that requires a
professional or standardised approach to risk management: Aviation, HSE, Project and Programme
Management, Financial and Investment Risk etc. They often come in a variety of shapes and sizes and
commonly appear hand in hand with the equally ubiquitous ‘risk register’2.
Executives and Accountable Managers enjoy their simplicity and Regulators
routinely ask to see them as their ‘start point’ indication that risk management
is taking place.
In our experience ubiquitous management tools like Risk Matrices that have
a solid foothold in everyday operations often find themselves being adopted,
integrated and employed purely at face value. Their simplicity and prevalence
lends them to being viewed with an almost unconscious deference to their
assumed necessity and “best practice” credentials. Little consideration is given
to their original purpose, construct or ‘correct’ employment.
Professional Risk Management practitioners would call this unconscious and unquestioning acceptance a
blind spot, and blind spots worry us. They have a nasty habit of concealing inherent weaknesses, fragile
assumptions and practical drift3 in the management systems we employ to keep our business safe and
successful.
In this paper I’d like to take a moment to shine a light on some of the assumptions we naturally make
regarding the origins, purpose and functional benefits of Risk Matrices in our aviation industry. I’d like
to see if we can derive some good (and bad) practices from their construction and employment. I’d like
to highlight a few misnomers about their use and identify where, when and how they might add some
practical value to organisational ‘risk-based decision making’ if properly designed, consciously reviewed
and correctly employed.
Matrix Revisited
White PaperNovember 2019
Author: Mark Townend, Senior Consultant, Baines Simmons
Risk toleration by matrix alone is simply not going to survive the harsh scrutiny, detailed investigation and demands for accountability that will inevitably occur if and when risk taking goes bad. Nor should it.
1 A Risk Matrix in this paper is a tool that utilises the basic description of risk (a combination of an event’s likelihood and the severity of its impact) to establish some form of ‘risk classification’.
2 A Risk Register in this paper is a log or other recording medium in which a summary list of identified and classified risks is held.
3 “the unintended systematic adaptation of practice from written procedure”: (Snook, 2000)
At the most basic level, risk matrices have their roots bedded in the basic concept that the risk of undesired
or unintended event4 can be described and classified in terms of the probability / likelihood or frequency of its
occurrence (note the distinction5) and the impact / severity of whatever may result from it. In simple terms: “how
bad” on one axis and either “how likely” or “how often” on the other (see figure 1).
It is from this very simple baseline that risk matrices start to diverge in both form and function, and it is very
important to recognise from the outset that not all matrices are the same. Terminology, size, shape, content,
application and output can all differ hugely from one matrix to another even when they may all appear superficially
similar.
There is already a wealth of research, analysis and academic commentary on the “goods, bads and uglies” of risk
matrices across numerous industries and from many learned experts on the subject6. I won’t attempt to reiterate
their well-researched points here in full (their work speaks for itself ), but it may be worth highlighting a few of the
most significant findings across their collective works to kick off the conversation on what Risk Matrices can, cannot,
should, and should not be doing for us.
4 See para 4.3 of “Hazard Identification and Risk Management challenges throughout the Supply Chain”: Kritzinger (2018).
5 “Probability / likelihood” relates to the predicted chance of a single occurrence, whilst “frequency” addresses the number of occurrences over a specific time frame. Both classifications have their place in specific circumstances and against specific types of risk, but our advice is to never mix the two within a single Risk Matrix: intense confusion and misclassification can result.
6 Cox (2008), Thomas, Reidar and Bickel (2014), Hubbard (2009), Kahneman and Tversky (1972), Talbot (2011)
Likelihoodof
Occurance
Impact of Occurance
Low Risk
High Risk
Figure 1: The basic description of risk and risk classification
The Good } Risk Matrices are simple in construct, easy to use and visually appealing. They require little to no technical expertise and specialist training to utilise in their basic function of classifying risks. As such they can open up the risk management conversation to a wider, less specialist audience.
} Risk Matrices have achieved widespread employment across multiple fields, industries and disciplines of risk management whilst at the same time appearing superficially similar in look and employment. As such they can form a basic fulcrum of risk management effort, particularly where different definitions of risk exist across different disciplines (financial, enterprise, project, safety etc.).
} The presence of a Risk Matrix in a risk management discussion can inspire positive dialogue and lively debate around the nature, history, probability, severity and causal factors of a risk. The improved risk awareness, understanding and consciousness that results from the debate can often be more valuable to the organisation than the risk classification that results from it.
The Bad } Risk Matrices contain several inherent mathematical, logical and structural flaws that significantly undermine the accuracy and reliability of their output for risk-based decision making. The more complex, multifarious and superficially “incredible” the risk, the more unreliable their output may become.
} Some researchers suggest that the structure and application of a matrix-based tool in risk analysis is so fundamentally flawed that it can statistically produce more consistently “wrong” classifications of a risk than if the user just randomly selected a risk classification without any “analysis”. Or, to use Dr Tony Cox’s assessment: the matrix (as an analytical tool) can ultimately prove to be “worse than useless”.
} Risk Matrices are often employed at a senior executive level where they can encourage important risk management decision making to take place with (at best) a loose, incoherent and fragile connection to more objective and rigorous risk analysis methodologies.
The Ugly } Risk Matrices appear to have found themselves widely cited as “best practice” risk management tools with little scientific research or objective evidence to support such an accolade.
} The simple ‘x and y axes’ correlation of risk likelihood vs risk severity can easily mask the nuanced complexity of a risk and the most efficient / effective risk treatment measures to address it. In the worst cases it may even mask the need for treatment entirely, leaving the organisation unconsciously exposed to the harmful outcomes of its risk taking.
} Risk Matrices are, ultimately, a qualitative tool for risk measurement and prioritisation although their construct and terminology often fools users into believing they are providing some form of quantified certainty in support risk based decision making.
} Excessive and careless use of colour, definitions and terminology can exacerbate a Risk Matrix’s inherent vulnerabilities to decision making bias and cognitive dissonance.
Remembering the inherent weaknesses and flaws of risk matrices as described in Point 1 above, one of the more
common and contentious functions they have served is to define a risk’s “tolerability”. This has been an established
element of the ICAO 9859 matrix example for many years now and has percolated down into numerous National
Aviation Authority examples in kind.
The tolerability of any individual risk is an incredibly complex thing. For starters, no (real world) tolerability
argument can be built upon a purely qualitative or a purely quantitative analysis of risk probability and severity7. Both
approaches are needed (and must be mutually supportive) to form as accurate and robust an understanding of risk
exposure as possible. And this is just the beginning of the justification argument.
Any decision on risk tolerability is ultimately going to be formed on a wide array of factors that will influence, to
differing degrees and in different ways, the impressions of numerous different stakeholders to that risk. Employees,
shareholders, investors, sub-contractors, clients, society, governments and even the international community may all
experience differing degrees of benefit from the taking of the risk and differing degrees of harm or loss if the risk’s
undesirable outcomes manifest.
7 As Hubbard points out in “The Failure of Risk Management” (2009), the “scoring” of a risk that many matrices employ is often mistaken as a form of quantified analysis. The “risk score”, however, is most often derived on a (generally subjective) analysis of the qualities of a risk (likelihood/probability and severity). In reality therefore, and despite outward appearances, Risk Matrices can only really be described as a qualitative tool.
How likely is “probable”, how frequent is “often”?
Point 2 of our findings summarises the work of Hubbard, Kahneman, Tversky and others who have carried out
extensive research into how decision makers handle uncertainty, risks and probabilistic decision making. Once again,
their work speaks for itself and is well worth the reading time for those with a professional interest in this subject.
One of the factors that stood out in their research was the variability, interpretive difference and imprecision
identified in the responses of different people using the same “verbal scales” to analyse and classify a risk. Hubbard
describes one risk meeting where, having realised the huge variance in their interpretation of the term ‘very likely’,
“...a roomful of people looked at each other as if they were just realising that…they had been speaking different
languages all along”.
When attempting to classify a risk in terms of likelihood and severity, the words ‘likely’, ‘credible’, ‘improbable’,
‘critical’ etc. may fit with my definition of the terms in a very different way to your own. Depending on training,
experience, job role, culture, values, ethics and a thousand other factors our interpretation and application of these
words to the risk classification may be close, in the same ballpark or a thousand miles apart. It is for this reason that
such terms must be explicitly and objectively defined in either qualitative or quantitative terms (or both) and in clear
language to minimise the potential for confusion.
The old saying goes that “a picture paints a thousand words”: it can also speak a thousand languages. One significant
success we have noted in recent years is the use of risk visualisation models (e.g. the Bowtie: see below) to help
reduce some of the interpretive and subjective weakness described above. Risk visualisations that score highly in
‘accessibility’ and ‘communicative clarity’ can be useful supportive tools to a more objective, structured assessment
of risk exposure so long as they are used correctly, consciously and competently9.
Of the two traditional risk matrix axes, ‘risk likelihood’ has generally been the more difficult to assess. This
is particularly true when attempting to classify risks with low probabilities / occurrence rates and extreme /
catastrophic consequences: a common negative correlation that Cox10 identifies as particularly vulnerable to the
logical and mathematical fallacies of the basic risk matrix construct.
Against this challenge, risk probability assessments become a lot more structured and objective when utilising a
“barrier based” approach to risk management: a methodology that has gained significant traction in aviation over the
last decade11. We now routinely talk about barriers and control measures to prevent a risk manifesting as a harmful
consequence or mitigating the effects of that consequence if it does.
9 Well-made, competently employed Bowties as a leading “barrier-based” approach to risk management modelling are ideally suited to help answer difficult and detailed questions about risk exposure and risk mitigation presence, suitability, operation and effectiveness. See our Bowtie Basics (TS101) and Bowtie Advanced (TS107) courses for more details.
10 “What’s Wrong with Risk Matrices”: Cox, L.A. 2008
11 For those familiar with the ERC matrix of the ARMS methodology, question 2 of the tool employs the same barrier focused logic argument to assess the potential likelihood and severity of an historical occurrence even though the occurrence itself may have had no significantly harmful outcome. See https://www.skybrary. aero/bookshelf/books/1141.pdf.
Mark is a highly skilled and dynamic safety management consultant with specialties in risk assessment, safety performance, error management and SMS improvement as well as a pedigree in the emerging market of civil and military unmanned aviation. Mark excels at identifying bespoke and innovative approaches to managing a client’s risk portfolio, utilising his background in developing military best practice to improve the identification, assessment, communication and mitigation of current and future threats.
Expertise and capability
} Highly experienced in developing and training Risk Analysis and Risk Management best practice for the aviation industry
} Accomplished communicator and liaison across safety practitioners, senior management and “front line” operators who live and breathe the risk environment
} Experienced in facilitating efficient cooperation and practical action between organisations, stakeholder groups and industry cultures
} Practical and highly relevant experience in improving the understanding, articulation and ownership of risk for accountable managers
} Skilled in identifying simple but effective solutions to complex problems
Career Background and experience
Mark left the British Army in 2012 following an expansive career in military Unmanned Air System (UAS) operations, UAS capability integration and strike asset management as a ground commander and liaison for air and aviation platforms. Mark’s military career has been defined by his extensive operational experiences: proving his skills as a manager, communicator and adaptive thinker in some of the most complex, austere and pressurised military environments of modern times.
Mark’s consulting skills, advice and innovations for military SMS are widely recognised across the Defence air safety community as leading in best practice for air asset risk assessment and risk articulation, particularly in the successful employment of the Bowtie methodology.
About Baines SimmonsWe are specialists in aviation regulations, compliance and safety management and partner with the world’s leading civil and defence aviation organisations to improve safety performance.
As trusted advisors to businesses, armed forces, governments and regulators across all sectors of aviation, we help to advance best practice, shape safety thinking and drive continuous improvement to safety performance through our consulting, training and outsourced services.
