Stationary Probability Vectorof a Higher-order Markov Chain

By Zhang Shixiao

Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan

Content

• 1. Introduction: Background

• 2. Higher-order Markov Chain

• 3. Conclusion

1. Introduction: Background

• Matrices are widely used in both science and engineering.

• In statisticsStochastic process: flow direction of a particular system or process.Stationary distribution: limiting behavior of a stochastic process.

Discrete Time-HomogeneousMarkov Chains

• A stochastic process with a discrete finite state space S

Pr (𝑋 𝑡+1= 𝑗∨𝑋 𝑡=𝑖 , 𝑋𝑡 −1=𝑖𝑡−1 , 𝑋 𝑡−2=𝑖𝑡− 2 ,… , ,𝑋 1=𝑖1 , 𝑋 0=𝑖0 )¿ Pr (𝑋 𝑡+1= 𝑗∨𝑋 𝑡=𝑖 )=𝑝𝑖𝑗

𝑃= (𝑝𝑖𝑗 , 𝑖 , 𝑗∈𝑆 )

• A unit sum vector X is said to be a stationary probability distribution of a finite Markov Chain if PX=X where

Discrete Time-HomogeneousMarkov Chains

• In other words

a coutinuous function f: which preserves at least one fixed point.

2. Higher-order Markov Chain

• a stochastic process with a sequence of random variables, , which takes on a finite set called the state set of the process

• Definition 2.1 Suppose the probability independent of time satisfying

Pr (𝑋 𝑡+1=𝑖∨𝑋 𝑡=𝑖1 ,𝑋 𝑡− 1=𝑖2 , 𝑋𝑡 −2=𝑖3 ,…, ,𝑋 1=𝑖𝑡 , 𝑋 0=𝑖𝑡+1 )¿ Pr (𝑋 𝑡+1=𝑖∨𝑋 𝑡=𝑖1 ,𝑋 𝑡− 1=𝑖2 , 𝑋𝑡 −2=𝑖3 , …,𝑋 𝑡 −𝑚+1=𝑖𝑚)

¿𝑝𝑖 ,𝑖1 , 𝑖2 ,⋯ , 𝑖𝑚

2. Higher-order Markov Chain

• Definition 2.2 Write to be a three-order n-dimensional tensor

where and define an n-dimensional column vector

2. Higher-order Markov Chain

• Example: is a three-order 2-dimensional tensor where and

Conditions forInfinitely Many Solutions over the Simplex

• Theorem 2.1 Now we are consideringwhere all

𝑥 ( 𝑎1 𝑏1

1−𝑎1 1−𝑏1)( 𝑥

1−𝑥)+ (1−𝑥 )( 𝑎2 𝑏2

1 −𝑎2 1 −𝑏2)( 𝑥

1−𝑥 )=( 𝑥1−𝑥)

Conditions forInfinitely Many Solutions over the Simplex

• Then one of the following holds

If , then we must have two solutions or to the above equation.If , then we must have infinitely many solutions, namely, every with is a solution to the above equation.

Otherwise, we must have a unique solution.

Conditions forInfinitely Many Solutions over the Simplex

• Then we want to extend the condition for infinitely many solutions for case

Main Theorem 2.2

would have infinitely many solutions over the whole set

if and only if

Main Theorem 2.2

12 1

121

1

1

1

1

n

n

a a

aA

a

12

12 23 2

2 23

2

1

1

1

1

n

n

a

a a a

A a

a

Main Theorem 2.2

1

2

1,

,1 , 12 1,

, 1

,

1

1

1

1

1

1

i

i

i i

ii ni i ii i i

i i

i n

a

a

aA

aa aa a

a

a

Main Theorem 2.2

1

2

1,

1 2 1,

1

1

1

1

n

n

n

n n

n n n n

a

a

A

a

a a a

Main Theorem 2.2

Proof:Sufficiency:For , infinitely many solutions

Main Theorem 2.2

Main Theorem 2.2

12 1, 1

121

1, 1

1

1

1

n

n

a a

aA

a

12

12 23 2, 1

232

2, 1

1

1

1

1

n

n

a

a a a

aA

a

1

2

1,

, 11 , 12 1,

, 1

, 1

1

1

1

1

1

1

i

i

i i

ii ni i ii i i

i i

i n

a

a

aA

aa aa a

a

a

1, 1

2, 1

1

2, 1

1, 1 2, 1 2, 1

1

1

1

1

n

n

n

n n

n n n n

a

a

A

a

a a a

Main Theorem 2.2

12 13 1

12 22 2

13 22

1,

1,1 2

1

1

1

1

n

n

n n

n nn n

a a a

a a a

a aM

a

aa a

Other

• Given any two solutions lying on the interior of1-dimensional face of the boundary of the simplex, then the whole 1-dimensional face must be a set of collection of solutions to the above equation.

• Conjecture: given any k+1 solutions lying in the interior of the k-dimensional face of the simplex, then any point lying in the whole k-dimensional face, including the vertexes and boundaries, will be a solution to the equation.

3. Conclusion

Thank you!