W W W . C H I C A G O L A N D R I S K F O R U M . O R G W W W . C H I C A G O L A N D R I S K F O R U M . O R G

What’s New in Risk Assessment?

2  2  

3  3  

Risk Management Depends on Risk Assessment The simplest definition of Risk Management involves 3 steps:

Step  1  

Step  2  

Step  3  

4  4  

Risk Assessment Has Many Moving Parts!

6  6  

Strategic Risk Assessment Question:

What is Important to Achieving Organizational Objectives and Not Under [complete] Control? Identifying threats and exposures without measurement generates lists of “Critical Risks” that frustrate management and fail to provide a blueprint for action. Strategic Risk Assessment starts with corporate objectives and considers assets at risk; identifies potential threats; and assesses the impact and the effectiveness of current controls to counter those threats.

– and points to new controls where objectives are threatened.

7  7  

Strategic Risk Assessment Issues To be effective, Risk Assessment must be connected to business strategy. •  Integrated in a way that provides timely and relevant risk

information to management.

•  Owned by the business units and be embedded within the business cycle, starting with strategic planning.

•  And: Risk assessment begins and ends with the organization’s specific objectives.

8  

Strategic  Risk  Assessment  

Qualita9ve  Analysis  Risk  Register  ◄  

Risk  Map  ◄  Risk  Categoriza9on  ◄  

Quan9ta9ve  Analysis  ► Decision  Tree  Analysis  ► Scenario  Analysis  ► FMEA  ► Simula9on  &  Modeling  

9  9  

Risk Assessment Basics •  It is a matter of widespread understanding that risks should be

assessed in terms of the likelihood (probability) that an uncontrolled event will occur and the consequences (impact) on achieving one or more organizational objectives. –  Applicable to both qualitative and quantitative methods of

assessment.

•  Strategic Risk Assessment involves pursuing a systematic, logical process to identify the magnitude of hazards and exposures, assess threats, and implement controls to mitigate, eliminate or control high-risk conditions.

Qualitative Methods & Risk Maps Highlight Critical Threats

Risk Maps Are Primarily Qualitative Assessments

! Data  gathering  &  representa9on  

! Select  appropriate  techniques(s)  

! Risk  analysis  &  modeling  ! Expert  judgment  

RISK

But Quantitative Methods are Often Required to Identify Corrective Actions

13  13  

Risk Assessment Tools & Techniques Are Rapidly Evolving – RM’s Must Keep Up! Risk Assessment must move beyond Probability x Severity, risk registers and risk maps to quantitatively evaluate emerging issues, detectability, and critical threats to strategic objectives. •  Over the past decade, developments in economic and financial

theory -- plus computing and “big data” advancements – provide new methods for quantitative risk assessment, as well as improvements to existing techniques.

•  Risk Managers should understand available risk assessment techniques and adopt a set of tools they can apply to their organization's unique Risk Management requirements.

Several of the analytical methods discussed in this presentation are contained in the new Institutes designation program, Associate in Insurance Data Analysis (AIDA).

14  14  

Three Basic Types of Quantitative Assessment Tools – In Order of Complexity

1.  Comparative methods; 2.  Temporal methods; and, 3.  Functional methods.

15  15  

Comparative Assessment Methods A Comparative Analysis takes an explicit standard – eg., “Best Practices” – and compares a system, process and/or set of procedures to that standard, producing a “Gap Analysis” as a result. A “good standard” is prepared and maintained as “the distillation of continually developing expert opinion and experience in the face of a continually changing environment”. •  One of the strengths of Comparative Analysis is its simplicity.

Comparative methods can be ideal for organizations just beginning to focus attention on specific systems, processes or threats.

•  A key weakness is the lack of an explicit list of threats as there is in other approaches.

16  16  

Sample “Best Practices” Matrix – Claims Handling

O Managerial OversightP PrimaryS SecondaryC Consultative InputD Data Resource

Directo

r  of  Insurance

Directo

r  of  Legal  Supp

ort  &

 Cla

ims

Executive

 Vice

 Preside

nt,  A

on

Senio

r  Vice

 Preside

nt,  Claims

Vice  Pr

eside

nt,  Claims

Assistant  VP,  Claim

s

Senio

r  Con

sultant,  Claims

Senio

r  Client  Sp

ecialist,  

Claims  (Megan)

Senio

r  Client  Sp

ecialist,  

Claims  (Martha)

Claim  Assistant

CLAIM  MANAGEMENT  PROCEDURES

1)Establish  formal  claims  service  standards  for  TPA's,  carriers  and  other  vendors   C P C P C C

2)Develop  annual  written  service  plan  for  TPA's  and  other  vendors  and  monitor  performance C P O P C C

3) Develop  written  Claims  Procedures  or  Manual C O C P C C

4)Establish  internal  claims  reporting  and  management  procedures  and  monitor  compliance C O C O P S

5)Develop  claim  reports,  distribute  and  review  with  business  units  as  necessary C O C O C P S S S D

6) Maintain  listing  of  all  insured  claims O O O C P S S S D7) Maintain  listing  of  all  self-­‐insured  claims O O O C P S S S D8) Establish  and  monitor  WC  post-­‐injury  management  program C O O O D P S9) Manage  claims  litigation  process C O O P D S D10) Administer  OCIP  claims C O O O P S11) Administer  non-­‐litigated  GL  claims O O O P S S D D12) Administer  auto  claims O O O P S S D D13) Administer  D&O,  fidelity,  fiduciary,  EPL C P C14) Administer  Litigated  GL  claims O O P D S D15) Administer  Property  claims O O O P D S D16) Pursue  subrogation  activities O O O P S S S D17) Review  losses  and  identify  trends C C C O C P S S S D18) Conduct/coordinate  periodic  claims  audits D D19) Monitor  large  loss  activity   C O O P D D

20)

Review  and  adjust  safety/loss  control  initiatives  as  needed  to  proactively  treat  risk  and  address  trends  observed  in  claims  management  activities

O C C C D C D

Management assures activ ity is addressed

Principally responsible for driv ing the activ ity

Responsible to perform or drive certain aspects of the activ ity , but is not the leader

Can prov ide guidance or feedback at a high level for activ ity

Prov ides data or information that is used in the activ ity

17  17  

Sample “Gap Analysis” – RM Strategy

18  18  

Temporal Analysis Methods A Temporal Assessment applies quantitative tests to a system, process or set of procedures. These “tests” involve analyzing the results of specific threats or attacks against actual protections and controls, subject to some constraints. •  Since it is often impractical to test a system directly, a model of the

system is generally used instead. –  However, a model introduces the question of fidelity: an

inaccurate model may not only confuse matters -- it may provide a false sense of security that is even worse than confusion.

•  A key weakness of a temporal method is that it is not possible to model – or even list -- all possible threats.

19  19  

One Temporal Method: Scenario Analysis

Scenario analysis considers the questions what might happen? and what should/would we do if it does?’ It not only highlights risks and opportunities in the short and long term; but also tests the effectiveness and efficiency of specific controls and plans. •  The central idea is to consider a variety of possible futures that

include many of the important uncertainties in the system, rather than to focus on the accurate prediction of any particular outcome.

•  A strength of scenario analysis is that it can consider “existential threats” that involve large swaths of the organization.

20  20  

Four Critical Components of Scenario Analysis 1.  Determine which factors the scenarios will be built around. In general,

analysts should focus on the two or three most critical factors. 2.  Determine the number of scenarios to analyze for each factor. That

depends upon how different the scenarios are, and how well the results of each scenario can be forecast.

3.  Estimate results – e.g., asset cash flows, control failures, unexpected breakdowns, etc. -- under each scenario.

4.  Assign probabilities to each scenario. Note: this may not make sense if the scenarios do not cover the full spectrum of possibilities; in that case, the probabilities will not add up to 100%

21  21  

Sample Scenario A Scenario Analysis to ensure effective and reliable insurance coverage. •  Typically involves meeting with brokers, underwriters, lawyers,

adjusters and managers to analyze and discuss how each insurance policy is expected to respond to different circumstances.

•  The results are compiled into systematic tables and charts that point out problem areas and suggest solutions.

One of the strengths of Scenario Analysis is that it tests the system itself (or a model), clearing away misconceptions and uncovering specific elements or issues needing attention.

22  22  

Other Temporal Analysis Methods

The most important Temporal Assessment techniques use Predictive Analytics to not only determine what might happen, but how much it could impact objectives. •  Two useful tools are:

–  Decision Tree Analysis; and, –  Simulation modeling.

23  23  

Decision Tree Analysis A Decision Tree is a logical structure in which each internal node represents a "test" on an attribute; each “branch” represents the outcome of the test; and each “leaf” represents a decision taken after computing all previous attributes. The paths from root to leaf represent classification rules:

–  A Root node represents the start of the decision tree, where a decision maker is faced with an uncertain outcome. The objective is to evaluate the overall net positive or negative outcomes at this node.

–  Event nodes represent outcomes based upon the probable occurrence of various events.

–  Decision branches represent choices that are made by the decision maker.

–  End nodes represent final outcomes where a payoff value is identified.

24  24  

Sample Decision Tree: Jenny Lind •  Jenny Lind is a writer of romance novels. A movie company

and a TV network have both offered to buy exclusive rights to one of her more popular works.

•  If she signs with the TV network, she will receive a definite lump sum; but if she signs with the movie company, the amount she will receive depends on the market response to her movie.

•  What should she do?

Jenny Lind Decision Tree

Small Box Office

Medium Box Office

Large Box Office

Small Box Office

Medium Box Office

Large Box Office

Sign with Movie Co.

Sign with TV Network

$200,000

$1,000,000

$3,000,000

$900,000

$900,000

$900,000

.3

.6

.1

.3

.6

.1

Es+mated  Outcomes  Es+mated  

Likelihood  

Root  Node  

Event  Nodes  

Jenny Lind Decision Tree - Solved Small Box Office

Medium Box Office

Large Box Office

Small Box Office

Medium Box Office

Large Box Office

Sign with Movie Co.

Sign with TV Network

$200,000

$1,000,000

$3,000,000

$900,000

$900,000

$900,000

.3

.6

.1

.3

.6

.1

Expected $900,000

Expected $960,000

Best Result $960,000

27  27  

Simulation Modeling Where Scenario Analysis and Decision Tree Analysis are techniques to assess discrete risk events, Simulation methods measure continuous risk exposures and outcomes. •  Simulations yield a distribution of outcomes rather than a single point

estimate. •  An important simulation tool is an “Exceedance Probability Curve”

that measures whether an outcome will exceed a specific estimate, based upon predetermined probabilities.

•  Simulation has few limitations in terms of events, probabilities and outcomes – very robust models may be constructed, evaluated and displayed graphically.

28  28  

Simulation Model Example: Quantifying the Risk of Natural Catastrophes How do companies prepare for the financial impact of natural catastrophes? How can they possibly determine the potential cost of events that haven't happened? Catastrophe Modeling can provide answers. A catastrophe model is generally composed of three parts: •  The Hazard Module looks at the physical characteristics of potential

disasters and their frequency. •  The Vulnerability Module assesses the vulnerability (“damageability”)

of buildings and their contents. •  The Damage Module determines the overall loss distribution for a

specific event by multiplying building values by potential damage.

Catastrophe Modeling Functions

Expected Frequency of Major Storms

0  2  4  6  8  10  12  14  16  18  20  

0%  

5%  

10%  

15%  

20%  

25%  

30%  

35%  

40%  

45%  

50%  

55%  

60%  

65%  

70%  

75%  

80%  

85%  

90%  

95%  

100%

 

Probability  of  #  Annual  Storms  

0  20  40  60  80  100  120  140  160  180  

0   1   2   3   4   5   6   7   8   9   10  11  12  13  14  15  16  17  18  19  20  

Expected  #  of  Annual  Storms  

At  least  1  major  storm  is  expected  in  2  years  out  of  3  

There is a 5% chance of 7 or more major

storms in 1 year

# of Annual Storms

# of

Ann

ual S

torm

s

# of

Yea

rs

Expected Business Interruption Losses Summary of Simulation Model Results

LIKELIHOOD EXPECTED LOSS ($MM) 1 in 10,000 $412.0

1 in 1,000 $103.2

1 in 100 $23.9

1 in 10 $3.5

1 in 3 No storm

Average Annual Loss $1.2

32  32  

Functional Assessment Methods A Functional Analysis focuses on specific threats and protections. •  A threat model -- a list of system vulnerabilities, and the likelihood of

successful threats against those vulnerabilities -- is weighed against organizational objectives, assets, and protections -- and the likelihood that available protections will successfully defend those assets against specified threats.

•  Temporal Assessment methods, such as statistical modeling; and Comparative Assessment techniques, such as expert systems, are often employed jointly.

•  The key strength of a Functional Assessment is its ability to specifically consider a wide range of threats, vulnerabilities, assets and countermeasures.

33  33  

Failure Mode & Effects Analysis (FMEA) FMEA identifies where & how failures can occur within processes and measures the impact of those failures. •  The FMEA Process has 4 basic steps:

1.  Determine the failure modes of specific process elements – that is, what (specifically) could go wrong;

2.  Analyze the likely effects of those failures on other elements and the overall system;

3.  Rank criticality; and, 4.  Identify existing and potential controls and their probable impact.

•  FMEA is particularly useful for evaluating critical risks in very complex systems.

FMEA Thought Process

FMEA Path Model Example

36  36  

Sample FMEA Template A c tion  R es ults

Item  /  F unc tionP o tential  F ailure  

Mode(s )

P o tential  E ffec t(s )  o f  F ailure

Sev

P o tential  C aus e(s )/  

Mechanism(s )  o f  F ailure

Prob

C urrent  D es ign  C ontro ls

Det

RPN

R ecommended  A c tion(s )

R es pons ibility  &  T arget  

C ompletion  D ateA c tions  T aken

New

 Sev

New

 Occ

New

 Det

New

 RPN

C oo lant  containment.  H o s e  connec tion.  C oo lant  fill.  M

C rack/break.  B urs t.  S ide  wall  flex.  B ad  s eal.  P oo r  ho s e  rete

Leak 8 Over  pres s ure 8 B urs t,  validation  pres s ure  c yc le.

1 64 T es t  inc luded  in  pro to type  and  produc tion  validation  tes ting.

J .P .  A guire      11/1/95      E .  E glin      8/1/96      

Response Plans and Tracking

Risk Priority Number - The combined weighting of Severity, Likelihood, and Detectability.RPN = Sev X Occ X Det

Likelihood - Write down the potential cause(s), and on a scale of 1-10, rate the Likelihood of each failure (10= most likely). See

Severity - On a scale of 1-10, rate the Severity of each failure (10= most severe). See Severity

Detectability - Examine the current design, then, on a scale of 1-10, rate the Detectability of each failure(10 = least detectable). See Detectability sheet.

Write down each failure mode and potential consequence(s) of that

37  37  

FMEA Technique: Fault Tree Analysis •  A Fault Tree is a logical diagram that starts with an actual or

predicted failure and works backward to identify all of the possible causes or origins of that failure.

•  Made up of branches connected by AND nodes and OR nodes; each node is assigned a pre-determined probability. –  ALL of the branches below an AND node must occur for the

event above the node to occur. –  Only ONE of the branches below an OR node needs to occur for

the event above the node to occur

38  38  

Fault Tree Example

Identified “Fault”

Both Required

Any of These

Required

39  39  

FMEA Technique: Event Tree Analysis •  An Event Tree is a logical diagram that starts with an actual or

potential event and works forward to identify all of the failures -- and possible corrective actions -- that could result.

•  Essentially the reverse of a Fault Tree; in an analysis, one Event Tree may lead to multiple Fault Trees and vice-versa.

•  Originally developed by engineers to find and fix vulnerabilities in nuclear power plants; it is applicable, and has been applied, to assess many complex processes.

.302  

.034  

.084  

.180  

.400  

41  41  

Summary – Strategic Risk Assessment Various quantitative risk assessment methods view the landscape from different heights, so to speak -- altitude is a tradeoff between scope and detail. •  The more abstract the method, the greater the scope but the

coarser the detail; the more concrete the method, the smaller the scope and the finer the detail.

•  Different objectives, systems, threats, perils, hazards, controls, etc. dictate the use of different assessment tools and methods.

•  Identifying the appropriate technique should be the first – and most important – step in risk assessment.

42  42  

And, Don’t Forget – the Real Objective is to Manage Risk •  Don’t waste time with complex analysis when you

KNOW what to DO – and everyone AGREES. •  The methods examined in this discussion, and many

other related techniques, are best used when there is considerable uncertainty about exposures, risks, perils, outcomes, hazards, etc. –  and about the best techniques to eliminate, mitigate,

control, or manage critical risks.

QUESTIONS?

Thank  you  very  much  for  listening!  

Backup

45  45  

Categorizing Risk Assessment Techniques

Three basic categories of assessment tools are: 1.  Comparative methods; 2.  Temporal methods; and, 3.  Functional methods.

Assessment techniques and tools can be classified on three axes: 1.  by their level of formality on a continuum from abstract to

concrete; 2.  the type of analysis performed; and 3.  the threats they are attempting to find and address.

46  46  

Types of Comparative Assessment Methods

•  A Principles Method type, like all of the Comparative types, is a list. This type asks the user to apply agreed-upon principles to their system.

•  A Best Practices list consists of directives: Do this, Don’t do that. This method asks the user to compare what they do—their current practice—with the best practice list: the list of differences represents the “Gaps” between actual practices and ideal.

•  An Audit is based on an explicit standard, such as a Best Practice list, a requirements list or a Principles list. This type asks the user to evaluate the effectiveness of the controls in place in fulfilling each item in the standard.

47  47  

Types of Temporal Assessment Methods

•  An Engagement consists of experts looking for any way, within given bounds, to compromise assets.

•  An Exercise links experts and owners together in order to test the protection of assets particular to a specific system.

•  Compliance Testing includes methods that the owner can execute him/herself without the aid of an expert.

48  48  

Types of Functional Assessment Methods •  Sequence Methods are the epitome of abstract methods. A simple

sequence method asks the questions: 1.  What can happen? (i.e., What can go wrong?) 2.  How likely is [it] that that will happen?

3.  If it does happen, what are the consequences?

•  An Assistant Method type keeps track of details. Best illustrations of this type “walk” the user through a process, prompting for inputs needed to populate and rank lists of threats, vulnerabilities and remedial actions.

•  A Matrix Method asks the user to select ranges for n dimensions – assets, threats, vulnerabilities and protections. The information in the cells of the corresponding n-dimensional subspace is the result of analysis.

–  An Expert System is one implementation that is representative of the functional approach.