Date post: | 07-Jan-2017 |
Category: |
Economy & Finance |
Upload: | global-association-of-risk-professionals-garp |
View: | 87 times |
Download: | 3 times |
Model Risk:How Model Risk Can Impact the Financial Markets&Model Risk at Banks
Bart Rikkert, Lead Model Validator, AEGON
Dr. Ebbe Negenman, Head Regulatory Risk, ABN AMRO Bank
Global Association of Risk Professionals
November 2016
2
The views expressed in the following material are the
author’s and do not necessarily represent the views of
the Global Association of Risk Professionals (GARP),
its Membership or its Management.
3 | © 2014 Global Association of Risk Professionals. All rights reserved.Helping people achieve a lifetime of financial security
Model Risk
‘How model risk can impact the
financial markets’
Amsterdam, 7 November 2016
Bart Rikkert
4 | © 2014 Global Association of Risk Professionals. All rights reserved.
What & Why
What exactly is a model and why do
we need it?
3
Model types
What are the key model types and
their sources of model risk?
7
Model Risk
Some recent examples of model risk
11
Model Risk - Agenda
Content
Discussion
Will model risk be controlled or grow
worse?
1
5
5 | © 2014 Global Association of Risk Professionals. All rights reserved.
Quantitative method, system, or approach that applies statistical,
economic, financial, or mathematical theories, techniques, and
assumptions to process input data into quantitative estimates.
A model consists of three components:
- Information input component – delivers assumptions and data to the model
- Processing component – transforms inputs into estimates
- Reporting component – translates the estimates into useful business information
Source: Supervisory Guidance on Model Risk Management, Board of Governors of the
Federal Reserve System, April 4, 2011
What is a model - Theory
The what and why behind models
Definition for a model
6 | © 2014 Global Association of Risk Professionals. All rights reserved.
Reporting
and Use of
resultsApplication/
Calculation
kernel
Selected
assumptions
Selected model
version
Selected model
points
Selected data
Per model run
Reports
User
validation
Reconciliation
processes
Reconciliation
processes
Data
processing
OutputOutputMethodology
Assumptions &
Expert Judgments
Data
Assumption input
process / interface
Data
input
process /
interface
Reporting
and Use of
results
Model
development
and Testing
Preparing for and Validating model runs
What does a model look like
Calculation
parameters
Calculation
parametersRunlist
The what and why behind models
7 | © 2014 Global Association of Risk Professionals. All rights reserved.
What is a models added value
What is the added value of models?
Defining a model
Make calculations and aggregations
Show impact of underlying Assumptions and Expert Judgments
Support reporting and control
Help to quickly assess the impact of changes/choices
Increase understanding of modelled products/risks
8 | © 2014 Global Association of Risk Professionals. All rights reserved.
What is a good Model Use
Key goal of a model is to consider all possible realities using:
► Market data
► Portfolio specific/client data
► Product characteristics & regulations
► Expert judgments & assumptions
► Advanced modelling skills
A model can predict based
on what we know and
can improve predictions:
While models can be further
improved, predictions stay
based on the past.
The what and why behind models
9 | © 2014 Global Association of Risk Professionals. All rights reserved.
What & Why
What exactly is a model and why do
we need it?
3
Model types
What are the key model types and
their sources of model risk?
7
Model Risk
Some recent examples of model risk
11
Model Risk - Agenda
Content
Discussion
Will model risk be controlled or grow
worse?
1
5
10 | © 2014 Global Association of Risk Professionals. All rights reserved.
Capital Models
Market Models
- Interest rate (volatility), Fixed income, Default, currency and hedges
Actuarial/Liability models:
- Longevity, Mortality, Policy holder behaviour, Cost and Operational Risk
Scenario and aggregation models
- Scenario engines, correlations, proxy models
and assessment/monitoring models to understand movements
Key Solvency II Internal Model components
Examples
11 | © 2014 Global Association of Risk Professionals. All rights reserved.
Valuation/Best Estimate models
Models are aligned with capital models
Use same key risk drivers as capital models.
Market Models for capturing market values
Valuation and DCF models for non-publically traded asset-valuation
Actuarial models to value insurance commitments for the coming 30-
50 years.
Cash flow and valuation models
Examples
12 | © 2014 Global Association of Risk Professionals. All rights reserved.
Other models
Key model types where we run model risk
Examples
Pricing / MCVNB models
- To determine value of new business and set pricing
- Biggest difference: Includes first cost
Strategic/Planning models
- For future plans and hedging
Supporting models
- Scenario models, (big) data
13 | © 2014 Global Association of Risk Professionals. All rights reserved.
What & Why
What exactly is a model and why do
we need it?
3
Model types
What are the key model types and
their sources of model risk?
7
Model Risk
Some recent examples of model risk
11
Model Risk - Agenda
Content
Discussion
Will model risk be controlled or grow
worse?
1
5
14 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk - examples
Liquidity completely dried up for RMBS market in 2008
Negative rates for Government Bonds (current)
Mortgage prepayment rates fluctuate due to government policy
Actuarial longevity tables changed considerably
Solvency II ratio difficult to predict by market/report on by companies
Methodology / Expert Judgment
Model Risk
15 | © 2014 Global Association of Risk Professionals. All rights reserved.
Residual Model Risk
• (Insurance) companies spend significant effort and resources on
containing model risk.
• Many controls are focussed on day to day use of the models and
focus on the day to day (small) movements.
• Concern is that most models can be quite accurate for small
movements, but become much less accurate for big movements.
• Senior Management and Regulators however often require
models to be able to handle all possible futures.
Model Risk
Controls around Model Risk
16 | © 2014 Global Association of Risk Professionals. All rights reserved.
Accept that the model has limitations
- Seems like a weak approach; ‘just accepting’
- Does not improve the technical model itself or the current output of the numbers
Focus on creating clarity on these model limitations
- Improves model understanding
- Gives clear boundaries to model use
- Add value to monitoring and links to management actions
Model risk and Model limitations
Model Risk
Managing residual Model Risk
17 | © 2014 Global Association of Risk Professionals. All rights reserved.
What & Why
What exactly is a model and why do
we need it?
3
Model types
What are the key model types and
their sources of model risk?
7
Model Risk
Some recent examples of model risk
11
Model Risk - Agenda
Content
Discussion
Will model risk be controlled or grow
worse?
1
5
18 | © 2014 Global Association of Risk Professionals. All rights reserved.
For discussion
Statement 1:
Technology and computing power is still growing strong, allowing millions of
scenarios to be run and hugely complicated models to be build and used in a
reasonably timeframe. A few ‘mathemagicians’ still understand the math, the
rest considers them magicians creating huge black boxes.
Statement 2:
For valuation and capital models the are becoming more and more similar with
spreading risk as a key assumption. This creates a huge systemic risk,
especially in combination with ever increasing capital requirements and the
quantitative easing of the central banks.
Developments and potential impact on model risk
For discussion
19 | © 2014 Global Association of Risk Professionals. All rights reserved.
19The end
20 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk: The Definitions In Use At ABN AMRO
A model:
‘a quantitative method that applies statistical, economic, financial, or
mathematical theories, techniques, and assumptions to process
input information into quantitative estimates.’
The use of models exposes the bank to model risk:
‘the potential loss the bank may incur, as a consequence of
decisions that could be principally based on the output of internal
models, due to errors in the development, implementation or use of
such models.’
A good read:
SR 11-7 Guidance on Model Risk Management
– https://www.federalreserve.gov/bankinforeg/srletters/sr1107.htm
21 | © 2014 Global Association of Risk Professionals. All rights reserved.
Models Are Widely Used And Grow In Importance
Currently e.g. used for:
identifying and measuring risks,
valuing exposures, instruments or positions,
pricing, liquidity and capital allocation,
conducting stress testing,
measuring compliance with internal limits,
meeting financial or regulatory reporting requirements
other
Growing importance, e.g., for
Monitoring customer accounts
Anticipate credit deterioration using automatic alert models
Fraud and money laundering detection
Customer on boarding, engagement and marketing campaign
models
Automated decisions making
22 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk: One Of The Largest Risks Of Banks
McKinsey on Risk Number 1, Summer 2016
“ Model risk. Banks’ increasing dependence on business modelling requires that risk managers
understand and manage model risk better. Although losses often go unreported, the consequences of
errors in the model can be extreme. For instance, a large Asia–Pacific bank lost $4 billion when it
applied interest-rate models that contained incorrect assumptions and data-entry errors. Risk
mitigation will entail rigorous guidelines and processes for developing and validating models, as well
as the constant monitoring and improvement of them.”
– http://www.mckinsey.com/business-functions/risk/our-insights/mckinsey-on-risk
23 | © 2014 Global Association of Risk Professionals. All rights reserved.
Increasing Regulatory Focus On Model Risk
Financial Times 16 Aug. 2015
“ECB doubles the time needed to review banks' risk models - Having originally hoped to
complete the review of banks' risk models within a year or two, the ECB has set a
deadline of four years for work on the project, according to a tender document seen by
the Financial Times.”
Likely regulatory questions to banks:
Does management body understand the degree of model risk
in credit, market & operational risk?
To what extend are models used to support significant
business decisions?
How significant is model risk, is there sensitivity, scenario’s &
stress testing?
How sound are model validation & review processes?
What are model risk control mechanism & how are these
tested?
24 | © 2014 Global Association of Risk Professionals. All rights reserved.
Q4 2008: Fortis Bank Posts a Net Loss of About €6 Billion due to Model Risk
25 | © 2014 Global Association of Risk Professionals. All rights reserved.
Capital Calculation At ING Group As It Used To Be
€3
€1 €1 €2
€5 €10 €8
€17 €13
€6 €8 €4
€25
26 | © 2014 Global Association of Risk Professionals. All rights reserved.
2007: ING Group Claims To Have €5bn Capital Excess
27 | © 2014 Global Association of Risk Professionals. All rights reserved.
October 2008: ING takes a €10bn Capital Injection from Dutch State
28 | © 2014 Global Association of Risk Professionals. All rights reserved.
In the Netherlands we have > € 700bn of mortgages. But client behaviour cannot be estimated
on historical data due to current fundamental macro economic disruptions.
Current Model Risk In Mortgages: Disruptive Client Behavior
3% mortgage
(after tax)4% on savings 1.25% mortgage
( after tax)0.4% on savings
Clients will maximize loan,
keep all savings Clients will invest excess savings
to reduce mortgage costs
World of 2006 World of 2016
29 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk Effect of Miscomputing An LTV
Den Helder -13%:
LTV =115.2%
Amsterdam: + 17.3%
LTV = 85%
+5%:
Den Helder : LTV = 95.2%
Amsterdam : LTV = 95.2%
30 | © 2014 Global Association of Risk Professionals. All rights reserved.
Effects Of Miscomputing An LTV Is Large
Effect on Risk can be large
• LTV is primary driver of LGD model, and expected loss computation.
Effect on Earnings of the bank can be large
• Clients with low LTV get a discount (or can get it during the lifetime)
31 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk Life Cycle at ABN AMRO
Effective Challenge is key: Model Validation is
involved in the Model Life Cycle at two stages:
after initial model development or re-development
of an existing model.
Model review is carried out risk based and in line
with regulatory requirements. Each risk model is
to be reviewed internally with a minimum
frequency of once a year, or additionally, when an
event has taken place which significantly impacts
the model performance
For market risk and ABN AMRO Clearing models,
the valuation models which serve as the input to
the risk models must be reviewed and revalidated
at least once every three years, but on an annual
basis an assessment is made if model reviews
should be moved forwards.
Model
development
Model
validation
Model
approval
Model
Implementation
Model
Use
• Inappropriate input
• Inappropriate methodology
• bad performance
• Poor model documentation
• Incomplete validation
• Poor documentation
• Misunderstanding
• Wrong decision
• Incorrect implementation/
deployment of
methodology
• Misuse of the model
• Incorrect run of model
• Inappropriate testing
32 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk Scorecards: Independent From Model Origination
Review Element
Risk Score
Raw data set(s) are defined and appropriate 1
Data checks are in place &
Outliers are recognized and treated appropriatelyN.A.
Data assumptions are identified and supportable 1
Input models are identified and supportable 3
Modeling methodology and parameters are sound 2
Model assumptions are identified and supportable 1
Relevant model backtests are performed 1
Documentation is of good quality 2
Model performance backtesting results are adequate 1
Regulatory and Policy requirements are satisfied 1
There are no concerns regarding Market outlook 2
There are no concerns regarding model implementation 1
0
MoC MoC level used in the model 0
Severity What is the severity of the detected "high risks"? 2
1.92
Medium
Model Risk Rating
(including MOC)Medium
Size of Portfolio Size of the portfolio 176 148
Model category Model category ALM(IR BB-ILAAP)
Overdue MAG date 6/6/2016
Number of action points Low Risk 1
Medium Risk 3
High Risk
Additional factors
Number of Action Points
Client Rate Model (ANMLWW08)
Component Risk Score
Data & Model inputs 3.00
Model Methodology 1.50
Model Assessment 1.25
Model Risk Rating
Enter the name of the model
(model code in the parentheses)
& the name of the validator
Enter the Value in million
For credit risk, use Outstanding
Enter the MAG date when the model is/will be APPROVED
(keep the format the same as the example: DD-MM-YYYY)
(The cell will assign:
White = MAG date is in the future
Green=Model is not overdue.
Amber=Model is overdue less than 6 month.
Red=Model is overdue more than 6 month.)
Enter the MoC in percentage
(without the percentage
sign).
Enter scores for the
review elements:
1: Good
2: Moderate
3: Poor
Determine the severity of the
detected high risk (=red)
elements. The final model risk
score will be adjusted
accordingly.
33 | © 2014 Global Association of Risk Professionals. All rights reserved.
Integrated Model Risk Reporting At ABN AMRO
Direct resource costs (economic and human) and development and implementation time
34 | © 2014 Global Association of Risk Professionals. All rights reserved.
Model Risk – Herd Behavioral Risk
Since it is costly (both in time and money) to develop a model that is eventually rejected by
the regulator, banks will have a tendency to choose risk modelling techniques that have been
proven to be acceptable by the regulator. In other words banks are more identical in their
models than you would expect from real competitors.
35 | © 2014 Global Association of Risk Professionals. All rights reserved.
There is Always Hope
Essentially, all models are wrong,
…but some are useful.”
George Edward Pelham Box
(18 October 1919 –
28 March 2013)
C r e a t i n g a c u l t u r e o f
r i s k a w a r e n e s s ®
Global Association of
Risk Professionals
111 Town Square Place
14th Floor
Jersey City, New Jersey 07310
U.S.A.
+ 1 201.719.7210
2nd Floor
Bengal Wing
9A Devonshire Square
London, EC2M 4YN
U.K.
+ 44 (0) 20 7397 9630
www.garp.org
About GARP | The Global Association of Risk Professionals (GARP) is a not-for-profit global membership organization dedicated to preparing professionals and organizations to make
better informed risk decisions. Membership represents over 150,000 risk management practitioners and researchers from banks, investment management firms, government agencies,
academic institutions, and corporations from more than 195 countries and territories. GARP administers the Financial Risk Manager (FRM®) and the Energy Risk Professional (ERP®)
Exams; certifications recognized by risk professionals worldwide. GARP also helps advance the role of risk management via comprehensive professional education and training for
professionals of all levels. www.garp.org.
36 | © 2014 Global Association of Risk Professionals. All rights reserved.