Date post: | 25-May-2015 |
Category: |
Education |
Upload: | wmhan-farhan |
View: | 793 times |
Download: | 0 times |
SJEM2252 : STOCHASTIC PROCESSES
:
What is Markov Chain?
A stochastic model that describes the probabilities of
transition among the states of a system
Change of state depends probabilistically only on
the current state of the system
Independent of the past given that the present state
is known.
The behaviour depends on the structure of the
transition matrix P
Markov chain modelling is one of the most powerful
tools for analysing complex stochastic systems (Kim
and Smith, 1989)
Markov chain models have become popular in
manpower system planning. Several researchers
have adopted Markov chain models to clarify
manpower policy issues.
Method of Data Collection
• Based on the journal, the staff movement, in
steps, in this system is considered to be an
absorbing chain. A forty year data of staff
movements, as specified by state space
described above, was obtained from the
organization studied.
• Study Area: University of Benin, Nigeria
Table 1 : Original state space
Table 2 : Lumped state space
States 1970-2010 States 1970-2010
Contract 282 Special leave 5
Dismissal 2 Study leave 2
Late 109 Suspension 2
Leave of absence 65 Temporary 617
Left 108 Termination
appointment
3
Permanent 3051 Training leave 63
Resign 12 Visiting appointment 12
Retired 446 Withdrawal 7
Sabatical 49
States Manpower
Retirement 458
Wastage 227
Disciplinary case 463
Leave 191
Staff Stock 3109
Recruitment 911
Total 5359
State space was reduced to six from seventeen as follows:
(i) Recruitment (Rc)
(ii) Staff Stock (Ss)
(iii) Training leave (T)
(iv) Suspension (S)
(v) Wastage (W)
(vi) Retirement (R).
We developed the theory of canonical forms of transition probability
matrix of a finite aperiodic recurrent set decomposable Markov chain
of order n with steady state probability row vector.
By states, it is specifically implied the condition, the status, the
position or situation a Markov Chain (object or staff) undergoes in the
transition process towards the final position of retirement, if the staff
can ever get there.
• Retirement, (R) : is the point where a person stops
employment completely.
• Wastage(W): reduction in size of a workforce by
voluntary resignation.
• Suspension (S): To bar for a period from a privilege,
office, or position, usually as a punishment.
• Training Leave (T): company provide opportunity to
training or outside study.
• Staff stock(Ss) : official employee.
• Recruitment(Rc) : temporary worker.
Markov Transition Matrix:
A square matrix describing the probabilities of moving from one state to
another in a dynamic system.
In each row are the probabilities of moving from the state represented
by that row, to the other states. Thus the rows of a Markov transition
matrix each add to one.
Probabilities were organized in canonical form.
The data obtained appear pure/original although we cannot
guarantee for perfection in the way the records were kept.
The data obtained were used to compute the transition probability
matrix (TPM) which we depict in Table 1. The estimates of the transition
probabilities were based on frequency distributions or tabulations of the
number of transitions from one state to the other in the system
considered. The frequency were converted to TPM by dividing each row
by its total.
It is assumed that after this number of movements, the matrix T would have attained stationarity.
Transition matrix, P is
P =
18607.063501.003901.0004636.009355.0
17751.060581.003722.009022.0008924.0
18588.063436.003897.009447.004632.00
000100
000010
000001
Rc
Ss
T
S
W
R
R W S T Ss Rc
DECOMPOSITION OF A STATE SPACE INTO EQUIVALENCE CLASSES
• We decompose state space into equivalence classes
• States in the same class will have the same characterizing attributes.
• For state i, we let:
C(i) denote the class containing state i.
T(i) be the set of all states that are accessible from i.
F(i) be the set of states from which state I is accessible.
Algorithm:
• Let T(i)={i} and F(i)={Φ}
• For each state k in T(i), add to T(i) all states j such that pkj>0 (if k is not already there). Reiterate.
• Add state j to F(i) if state i is in T(j). Reiterate.
• C(i)= F(i) ∩ T(i).
Transient and Recurrent Classes
• State i is recurrent if i is accessible from every state that is accessible from i.
• If i communicate with j, but j does not communicate with I, then I is transient.
Canonical Form
• By grouping the states together in their classes and list the recurrent classes first, the transition matrix P can be written the following canonical form:
R 0Q T
Absorbing State:
pii=1
Stays forever
Recurrent
Forms a class by itself
Absorbing Chain:
All of its states are either transient or absorbing.
It implies that it is reducible chain
The transition matrix P can be organized into the following canonical form:
𝐈 𝐎𝐐 𝐓
PERIODIC STATES AND PERIODIC CLASS
• A state i has period d if the chain can only revisit it a multiple of d steps later.
• d is the greatest common divisor of the set of integers
• If d=1, then the state has period 1 and is said to be aperiodic. Otherwise, if d>1,the state has period more than 1 and is said to be periodic.
EXPECTED NUMBER OF VISITS TO A TRANSIENT STATE
• Let vij be the total number of visits the chain makes to state j throughout its life, starting from state i.
• Matrix U is called the fundamental matrix.
EXPECTED ABSORPTION TIME
• Let ui be the absorption time or the number of steps before the chain is absorbed into one of the absorbing states, starting from a transient state i.
• The expected absorption :
• In matrix form:
where e is a column vector with all elements equal 1.
ABSORBING PROBABILITIES
• Q - transient to absorbing portion
• T - transient to transient portion
• [UQ]i,j - probability that the chain starts from transient state i, and then moves to the absorbing state j rather than any other absorbing state regardless of the number of steps taken.
LIMITING DISTRIBUTIONS
• Dependent on initial conditions
• After infinite steps, probability of visiting transient state equals to 0
• State space, S = { Retirement,(R) , Wastage,(W) , Suspension(S) ,Training Leave,(T), Staff Stock,(Ss) , Recruitment,(Rc) }
• Transition matrix, P is
P =
18607.063501.003901.0004636.009355.0
17751.060581.003722.009022.0008924.0
18588.063436.003897.009447.004632.00
000100
000010
000001
Rc
Ss
T
S
W
R
R W S T Ss Rc
DECOMPOSITION OF STATE SPACE
State, i T ( i ) F ( i ) C ( i )
R R R, T, Ss, Rc R
W W W, T, Ss, Rc W
S S S T, Ss, Rc S
T T, W, S, Ss, Rc, R T, Ss, Rc T, Ss, Rc
Ss Ss, R, S, T, Rc, W T, Ss, Rc T, Ss, Rc
Rc Rc, R, W, T, Ss, S T, Ss, Rc T, Ss, Rc
• Closed class
I. E1 = { R }
II. E2 = { W }
III. E3 = { S }
• Non-closed class
T = { T, Ss, Rc }
R,W ,S are recurrent states
T, Ss, Rc are transient states
Absorbing state
{ R, W, S }
Transient state
{ T, Ss, Rc }
Absorbing chain
P =
P2 =
18607.063501.003901.0004636.009355.0
17751.060581.003722.009022.0008924.0
18588.063436.003897.009447.004632.00
000100
000010
000001
Rc
Ss
T
S
W
R
15459.015459.052760.006098.005679.016763.0
14749.050334.003092.014839.000995.015991.0
15444.052706.003238.015538.005674.007400.0
000100
000010
000001
R W S T Ss Rc
P3 =
P4 =
12844.043835.002693.011164.006546.022917.0
12254.041820.002569.019672.001822.021862.0
12831.043791.002690.020599.006540.013548.0
000100
000010
000001
10672.036421.002238.015373.007266.028031.0
10181.034746.002135.023688.002509.026741.0
10661.036383.002235.024804.007260.018656.0
000100
000010
000001
P5 =
P6 =
08867.030260.001859.018870.007865.032279.0
08459.028869.001774.027025.003080.030794.0
08858.030229.001857.028298.007857.022901.0
000100
000010
000001
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0.26427 0.08354 0.31201 0.01543 0.25116 0.07359
0.34161 0.03555 0.29797 0.01474 0.23986 0.07028
0.35809 0.08362 0.21776 0.01545 0.25142 0.07367
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0.29357 0.08767 0.33612 0.01282 0.20868 0.06114
0.36959 0.03949 0.32100 0.01224 0.19928 0.05839
0.38742 0.08775 0.24190 0.01283 0.20889 0.06121
P7 =
P8 =
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0.31791 0.09110 0.35616 0.01065 0.17338 0.05080
0.39284 0.04276 0.34013 0.01017 0.16558 0.04852
0.41178 0.09118 0.26196 0.01066 0.17356 0.05085
P9 =
P10 =
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0.33813 0.09394 0.37281 0.00885 0.14405 0.04221
0.41216 0.04548 0.35603 0.00845 0.13757 0.04031
0.43203 0.09403 0.27863 0.00886 0.14420 0.04225
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0.35494 0.09631 0.38664 0.00735 0.11969 0.03507
0.42820 0.04774 0.36924 0.00702 0.11430 0.03349
0.44885 0.09640 0.29247 0.00736 0.11981 0.03511
For i = 1,2,3,4,5,and 6
Pii(n) > 0 ,
when n = 1,2,…,10,…
Greatest common divisor d is 1. Hence d=1.
That is, all the states are aperiodic state.
Hence, the chain is aperiodic.
EXPECTED NUMBER OF VISIT TO TRANSIENT STATES
U = ( I – T )-1
= 1 0 00 1 00 0 1
−0.03897 0.63436 0.185880.03722 0.60581 0.177510.03901 0.63501 0.18607
-1
𝑇 𝑆𝑠 𝑅𝑐
= 𝑇𝑆𝑠𝑅𝑐
1.23040 3.75028 1.098890.22004 4.58150 1.049430.23064 3.75413 2.10002
If we are currently on training leave,T : We are expected to go for the next training leave 1.23040 steps
later.
We are expected to be official employee 3.75028 steps later
We are expected to be recruited as a temporary worker 1.09889
steps later
If we are now an official employee , Ss : We are expected to go for next training leave 0.22004 steps later.
We are expected to stay as an official employee 4.58150 steps
later
We are expected to be recruited to be as a temporary worker
1.04943 steps later
If we are now recruited, Rc : We are expected to go for next training leave 0.23064 steps later.
We are expected to be an official employee 3.75413 steps later
We are expected to be recruited again 2.10002 steps later
EXPECTED ABSORPTION TIME
ui = Ue
= 1.23040 3.75028 1.098890.22004 4.58150 1.049430.23064 3.75413 2.10002
111
= 𝑇𝑆𝑠𝑅𝑐
6.0795775.8509626.084779
If we are now on training leave (T), we will move
6.079577 number of steps before getting
absorbed into retirement (R),wastage (W) or
suspension (S)
If we are now an official employee (Ss), we will
move 5.850962 number of steps before getting
absorbed into retirement (R),wastage (W) or
suspension (S)
If we are now recruited (Rc), we will move
6.084779 number of steps before getting
absorbed into retirement (R),wastage (W) or
suspension (S)
ABSORBING PROBABILITIES
A= UQ
= 1.23040 3.75028 1.098890.22004 4.58150 1.049430.23064 3.75413 2.10002
0 0.04632 0.094470.08924 0 0.090220.09355 0.04636 0
𝑅 𝑊 𝑆
= 𝑇𝑆𝑠𝑅𝑐
0.43748 0.10794 0.454590.50703 0.05884 0.434130.53148 0.10804 0.36049
If we are currently on training leave,T : We will end up in retirement with probability of 0.43748
We will end up in wastage with probability of 0.10794
We will end up being suspended with probability of 0.45459
If we are now an official employee , Ss : We will end up in retirement with probability of 0.50703
We will end up in wastage with probability of 0.05884
We will end up being suspended with probability of 0.43413
If we are now recruited, Rc : We will end up in retirement with probability of 0.53148
We will end up in wastage with probability of 0.10804
We will end up being suspended with probability of 0.36049
LIMITING
DISTRIBUTIONS
𝑃(𝑘) = 𝐼 0𝐴 𝑇𝑘
𝑃(∞)= 𝐼 0𝐴 0
The limiting probabilities, 𝑃(∞)
𝑃(∞) =
After infinite years, employees must go to Retirement(R ) ,Wastage(W), and Suspension(S)
00036049.010804.053148.0
00043413.005884.050703.0
00045459.010794.043748.0
000100
000010
000001
Rc
Ss
T
S
W
R
R W S T Ss Rc
Arising from the foregoing analysis and discussion, it is evident that the Markov Chain model applied has been able to provide useful insight into the behaviour pattern for a long run manpower policy for the organization studied.
The transition probability matrix, after several sets of transitions, shows the properties of stochastic regularities and aperiodic.The fundamental matrix, U when multiplied with the Q, do, gave us a dependable probability matrix that provides reliable manpower forecast outcomes in line with the theory of Markov Chain.
The results of this study appear useful for understanding the long run manpower policy for a university system.
• A.C. Igboanugo & M.K. Onifade(2011). Markov Chain Analysis of Manpower Data of a Nigerian University. Journal of Innovative Research in Engineering and Science 2, 2(2), 107-123.
• Adriana Irawati Nur Ibrahim. (2013). SJEM2252: Stochastic Processes, week 5-11 notes [pdf].
• Retrieved from http://www.thefreedictionary.com