Date post: | 14-Jun-2018 |
Category: |
Documents |
Upload: | vuongkhanh |
View: | 218 times |
Download: | 0 times |
Hidden Markov Models for Default Risk
Wai-Ki CHING
Department of Mathematics
The University of Hong Kong
Abstract: Default risk in commercial lending is one of the majorconcerns of the creditors. In this article, we introduce a new Hid-den Markov Model with multiple observable sequences (MHMM),assuming that all the observable sequences are driven by a commonhidden sequence, and utilize it to analyze default data in a networkof sectors. Efficient estimation method is then adopted to esti-mate the model parameters. To further illustrate the advantagesof MHMM, we compare the hidden risk state process obtained inMHMM with those from the traditional HMMs using credit defaultdata.
A joint work with Terence H. Leung. Research Supported inPart by HK RGC Grant 7017/07P.
1
Introduction
(1) Motivations and Objectives.
(2) The Idea of HMM Through an Example.
(3) Parameters Estimation.
(4) Application to Credit Default Data.
(5) Concluding Remarks.
2
1. Motivations and Objectives.
• Hidden Markov Models (HMMs) are widely used in many areas.
-Speech Recognition. L. Rabiner, A Tutorial on Hidden MarkovModels and Selected Applications in Speech Recognition, Proceed-ings of the IEEE, 77 (1989) 257–286.
-Computer Vision. H. Bunke and T. Caelli, Hidden Markov Mod-els : Applications in Computer Vision, Editors, Horst Bunke, TerryCaelli, Singapore, World Scientific (2001).
-Bioinformatics. T. Koski, Hidden Markov Models for Bioinfor-matics, Kluwer Academic Publisher, Dordrecht (2001).
-Finance. Rogemar S. Mamon and Robert J. Elliott, Hidden MarkovModels in Finance, New York : Springer (2007).
3
• In a HMM, there are two types of states: observable states and
hidden states. The hidden states follow a Markov chain process
and the observable states are driven by the hidden states.
• To define a HMM, one has to define the number of both types
of states and also their transition probabilities.
• The major problem in constructing a HMM is to determine the
transition probabilities of the hidden states because the transitions
among the hidden states are supposed to be unobservable. Ad-
vanced methods based on EM-like algorithm will be proposed to
solve the problem.
• Here we propose a HMM for multiple observation sequences.
The model is then applied to modeling credit default data. Hid-
den states concerns the risk level (economic condition) and the
observable states are the number of default bonds.
4
2. The Idea of HMM Through an Example.
•We consider the process of choosing dice and recording the number
of dots by throwing the dice.
• Suppose we have two dice A and B such that Die A is fair and
Die B is bias.
• The probability distributions of the dots obtained by throwing Dice
A and Dice B are given in the table below.
Die 1 2 3 4 5 6A 1/6 1/6 1/6 1/6 1/6 1/6B 1/6 1/6 1/3 1/6 1/12 1/12
Table 1.
5
• Each time a die is chosen, with probability α, Die B is chosengiven that Die A was chosen last time. And with probability β,Die A is chosen given that Die B was chosen last time. It is a2-state Markov chain process having transition probability matrix:
Die ADie B
(1− α α
β 1− β
)
• This process is hidden because we don’t know actually which dieis being chosen.
• The chosen die is then thrown and the number of dots (thisis observable) obtained is recorded. The following is a possiblerealization of the process:
A︸︷︷︸Hidden
→ A︸︷︷︸Hidden
→ B︸︷︷︸Hidden
→ B︸︷︷︸Hidden
→ A︸︷︷︸Hidden
→ · · ·
↓ ↓ ↓ ↓ ↓ · · ·
1︸︷︷︸Observable
3︸︷︷︸Observable
6︸︷︷︸Observable
5︸︷︷︸Observable
4︸︷︷︸Observable
· · ·
6
The followings are the model parameters of our HMM.
N , number of hidden statesK, number of observable sequencesT , length of the observation periodM , number of distinct observable statesS = {S1, . . . , SN}, the set of hidden statesqt, hidden state at time tV = {v1, . . . , vM}, the set of observable statesOk = (ok
1, ok2, . . . , ok
T ), kth observation sequencewk, the weighting of the kth observation sequence
Q = (q0, q1, q2, . . . , qT ), the sequence of hidden stateaij, transition probabilities from hidden State i to hidden State jbkj (v), the probability of symbol v being observed at state j in the
k-th sequenceπ, initial state distributionλ = (A, B, π), the model training parameters (N , M are fixed)
7
3. Model Parameter Estimation.
In order to define an HMM, one has to solve THREE problems:
(I) To efficiently compute P (O|λ), the likelihood of of a given ob-servation sequence, when we are given the model λ = (A, B, π) andthe observation sequence O = O1O2 . . . OT .Forward and Backward Algorithm, (Baum, 1972).
(II) To find the most likely hidden sequence.Viterbi algorithm (Viterbi, 1967), a DP approach.
(III) To adjust the parameters λ = (A, B, π) of the model so as tomaximize P (O|λ).Baum-Welch algorithm (Baum-Welch, 1970), an EM-like algo-rithm.
8
• For the classical HMM, (one observation sequence) (I), (II) and
(III) can be solved by using EM algorithm, Rabiner (1989) and
MacDonald and Zucchini (1999).
• A HMM has been proposed by Li et al. (2000) for multiple se-
quences.
• In our HMM, instead of having K hidden sequences as in Li et
al. (2000), here we only consider one common (global) hidden
sequence with weighting given to each observed sequence. We
define the likelihood as
P (O, Q|λ)) = πq0
T∏
t=2
[aqt−1,qt
K∏
k=1
(bkqt(ok
t ))wk]. (1)
Here wk > 0 is the weighting for the k-th sequence and∑K
k=11wk
= 1.
9
4. Application to Credit Default Data.
• We present the estimation results of our MHMM (multiple ob-
servable sequences) and also the classical HMM (one observable
sequence) using the observed default data extracted from the fig-
ures in Giampieri et al. (2005)
• Giampieri et al. (2005) apply the HMM to the quarterly bond
defaults data of four sectors ((i) Consumer, (ii) Energy, (iii)
Media and (iv) Transportation) in the United States taken from
Standard & Poors’ ProCredit6.2 database. The data set covers the
period from January 1981 to December 2002.
• The total number of bonds in January 1981 was 281 while the
total number of bonds in December 2002 was 222. At the beginning
there are 1024,420,650 and 281 non-default bonds in the above
sectors respectively. At the end of observation period, there are
251,71,133 and 59 default bonds in the above sectors respectively.
10
• In our study, we assume that there are two hidden states 0 (nor-
mal risk) and 1 (high risk) for the global risk state.
• We further assume that there are 3 observable states for the
credit default sequences and they are b1 (0 or 1 default), b2 (2 or
3 defaults) and b3 (4 or more defaults).
• We remark that this classification of observable states is subjective
and can be adjusted.
• Moreover, we assume that all the weighting wi are equal in our
calculation and again this can also be adjusted.
11
• All computations were done on a Pentium 4HT PC with MATLAB.
• We denote the model parameters (Am, B(i)m , πm) for our proposed
HMM and (Ai, Bi, πi), i = 1,2,3,4 for the classical HMM of individual
sequences 1,2,3,4 respectively.
• Since the problem is highly non-linear. The EM algorithm may
end up with local optima.
• We run the algorithm 100 times with different initial guesses
of the model parameters and get the best estimate.
• The stopping criteria is that the difference between successive
estimates in ||.||2 is less than 10−6.
12
• The model parameters of our model MHMM (multiple sequences)
and the traditional HMM (individual sequence):
Am =
(0.9660 0.03400.0395 0.9605
)πm =
(0.96070.0393
)
A1 =
(0.9367 0.06330.0629 0.9371
)π1 =
(0.93430.0657
)
A2 =
(0.9502 0.04980.0487 0.9513
)π2 =
(0.08700.9130
)
A3 =
(0.9622 0.03780.0390 0.9610
)π3 =
(0.95400.0460
)
A4 =
(0.9882 0.01180.0000 1.0000
)π4 =
(1.00000.0000
)
13
B(1)m =
(0.6130 0.3040 0.08300.0000 0.2768 0.7232
)B(1) =
(0.7089 0.2911 0.00000.0321 0.3013 0.6666
)
B(2)m =
(0.8171 0.1664 0.01660.6036 0.3964 0.0000
)B(2) =
(0.9561 0.0439 0.00000.4418 0.5298 0.0284
)
B(3)m =
(0.8759 0.1065 0.01760.1134 0.3824 0.5042
)B(3) =
(0.9035 0.0806 0.01590.1437 0.4008 0.4555
)
B(4)m =
(0.9505 0.0495 0.00000.5656 0.3259 0.1085
)B(4) =
(0.8588 0.1412 0.00000.0000 0.0000 1.0000
).
14
4.1 Inferring the Hidden Risk State
• We present the hidden sequences obtained from our MHMM for
multiple sequences Hm and also the classical HMM for individual
sequences Hi.
• We found that Hm is close to H3 (media sector) and H1 (consumer
sector). A bit different from H2 (energy sector).
• But Hm is significantly different from H4 (transportation sec-
tor). This is because large number of defaults only occurred in the
last three quarters in the transportation sector. Our MHMM can
give a consistent but a more holistic situation of the global eco-
nomic risk.
15
• The estimated hidden sequences (risk states: 0=normal risk and
1=high risk):
Hm : MHHM Model0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
H1 : Consumer Sector0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
H2 : Energy Sector0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
H3 : Media Sector0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
H4 : Transport Sector
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1
16
• Two separate periods were classified by our MHHM as en-
hanced risk. They are: from July 1989 to Dec. 1991 and from
July 1998 to Dec. 2002.
• In Giampieri et al., it is mentioned that their classical HMM esti-
mate the enhanced risk state overlaps with the recession periods dur-
ing the two most serious recessions: from July 1990 to Mar. 1991
and from Mar. 2001 to Nov. 2001 contractions of the business
cycle.
• But their classical HMM also estimate that one quarter in 1982
and one quarter in 1986 are in enhanced risk state with some
explanation. However, the enhanced risk state identified by our
MHHM can be justified by the mentioned recession periods.
• Results from our newly-proposed MHMM can model a more holis-
tic situation of the global economic risk.
17
4.2 A Comparison Based on VaR and ES
• We further compare our MHMM with the classical HMM in the
computation of VaR and ES using the same data set.
• We consider a 5-year (20 quarters) portfolio of 100 bonds and
each bond carries equally one unit (1 dollar).
• To apply the Binomial model, we have to determine the default
probability p of a bond in high risk and normal risk state.
• Since the most likely hidden sequences of all the sectors in both
MHMM and HMM are known. With the information of the number
of defaults in each quarter, one can estimate the default probability
in each of the sector and in each of the risk state in both of MHMM
and HMM. And they are presented in Tables 4.1 and 4.2.
18
Table 4.1 The Default Probability of a Bond in the MHMM.
Sector High Risk Normal RiskConsumer 2.916× 10−3 1.493× 10−3
Energy 4.151× 10−3 3.168× 10−3
Media 4.284× 10−3 1.874× 10−3
Transport 7.541× 10−3 4.347× 10−3
Table 4.2 The Default Probability of a Bond in the HMM.
Sector High Risk Normal RiskConsumer 2.817× 10−3 1.284× 10−3
Energy 4.559× 10−3 2.802× 10−3
Media 4.248× 10−3 1.851× 10−3
Transport 1.658× 10−2 4.967× 10−3
19
Table 4.3 The VaR and ES in the MHMM (Left) and HMM
(Right) at 95%.
Sector VaR ES VaR ESConsumer 8.42 9.82 8.10 9.32Energy 11.70 12.93 12.02 13.32Media 11.45 12.87 11.35 12.85Transport 17.81 19.66 34.32 36.54
Table 4.4 The VaR and ES in the MHMM (Left) and HMM
(Right) at 99%.
Sector VaR ES VaR ESConsumer 10.83 11.64 10.32 11.26Energy 13.76 14.80 14.20 15.52Media 13.83 14.82 13.80 15.00Transport 21.03 22.02 37.85 39.98
20
4.3 Prediction of Global Economic Risk State
• The prediction of economic risk is an important issue in risk man-
agement. Using our proposed MHMM, one can infer the economic
risk (0 or 1) of each quarter.
• We then can build a regression model (a prediction model) for
the risk state at quarter t+1 against the number of defaults in each
of the sectors at quarter t.
• Using the results (the sequence Hm: 0 for normal risk and 1
for high risk) obtained by our MHMM we found that we have to
exclude the energy sector in order to get a statistically significant
model. We get the following model through logistic regression.
21
• The probability that the risk state of the next quarter is 1 (high
risk) is given by
P (x1, x2, x3) =1
(1 + ey(x1,x2,x3))(2)
where
y(x1, x2, x3) = 4.4078− 0.3168x1 − 1.1041x2 − 1.3416x3 (3)
and x1, x2, x3 are the number of defaults in consumer sector, me-
dia sector and transportation sector in the current quarter re-
spectively.
22
• Figure 1 gives the prediction of the probability of high risk (‘◦’) by Model (2) and the inferred risk states (sequence Hm) by our
MHMM (‘ - ’ ).
• Furthermore, we have computed the correlation coefficient of the
two sequences (probabilities and sequence Hm) which is 0.8039.
This shows a high positive correlation between the prediction
probabilities and the risk state sequence.
0 10 20 30 40 50 60 70 80 900
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
The Prediction Probability and the Real Risk State
23
5. Concluding Remarks
• We proposed a HMM for modeling and analyzing default data in a
network of sectors (multiple sequences). The proposed model takes
into account of the correlations of the related observable sequences
sharing a common hidden sequence.
• The model was then applied to the calculation of VaR and ES of
a portfolio of bonds.
• A regression model was also built for the prediction of economic
risk.
24
6. References.
• Baum, L. (1972). An Inequality and Associated MaximizationTechniques in statistical Estimation for Probabilistic Function ofMarkov Processes, Inequality, 3, 1-8.
• Bunke, H. and Caelli, T. (2001). Hidden Markov Models : Ap-plications in Computer Vision, Editors, Horst Bunke, Terry Caelli,Singapore, World Scientific.
• Ching, W. and Ng, M. (2006). Markov chains : models, algorithmsand applications, International Series on Operations Research andManagement Science, Springer: New York.
• Ching, W., Ng, M. and Wong, K. (2004). Hidden Markov Modeland Its Applications in Customer Relationship Management, IMAJournal of Management Mathematics, 15 13-24.
25
• Ching, W., Fung, E., Ng, M., Siu, T. and Li, W. (2007). Inter-
active Hidden Markov Models and Their Applications, IMA Journal
of Management Mathematics, 18 85-97.
• Davis, M. and Lo, V. (1999). Infectious Defaults, Quantitative
Finance, 1 382-387.
• Davis, M. and Lo, V. (2001). Modeling Default Correlation in
Bond Portfolio, In C. Alescander (ed.) Mastering Risk Volume 2:
Applications. Financial Times, Prentice Hall. 141-151.
• Giampieri, G., Davis M. and Crowder M. (2005). Analysis of de-
fault data using hidden Markov models, Quantit. Finance, 5 27–34.
26
• Koski, T. (2001) Hidden Markov Models for Bioinformatics, Kluwer
Academic Publisher, Dordrecht.
• Levinson, S. Rabiner, L. and Sondhi, M. (1983). An introduction
to the application if theory of probabilistic functions of Markov pro-
cess to automatic speech recognition, Bell System technical journal,
62 1035–1074.
• Li, X., Parizeau, M. and Plamondon, R. (2000). Training hidden
Markov models with multiple observation - a combinatorial method,
IEEE Transaction on Pattern Analysis and Machine Intelligence, 22
371–377.
• MacDonald, I. and Zucchini, W. (1999). Hidden Markov and other
models for discrete-valued time series, Chapman & Hall: London.
27
• Mamon, R. and Elliott, R. (2007). Hidden Markov Models in Fi-
nance, New York : Springer.
•Rabiner, L. (1989). A tutorial on hidden Markov models and se-
lected applications in speech recognition, Proceedings of the IEEE,
77 257-286.
• Viterbi, A. (1967). Error Bounds for Convolutional Codes and an
Asymptotically Optimum Decoding Algorithm, IEEE Trans. Infor-
mation Theory, 13, 260-269.
28