Markov ChainAndrew Wang

Yum

Probability

0.2

0.8

0.3

0.7

Monte Carlo Simulation

Examples:

What are the most commonly visited spots in the game of monopoly?

Drunkard's Walk

Markov Chain

Each day, you choose to eat either grapes, cheese, or lettuce:

1. Choice today affects preferences tomorrow

2. cheese => tomato (0.5) || lettuce (0.5)

3. grapes => grapes (0.1) || cheese (0.4) || lettuce (0.5)

4. lettuce => grapes (0.4) || cheese (0.6)nom

Terminology

Absorbing

Expected Number

Probability

Linear System

How is this even related to CS?

Gaussian EliminationGaussianElimination[m_?MatrixQ, v_?VectorQ] :=Last /@ RowReduce[Flatten /@ Transpose[{m, v}]]

2x + y - z = 8

-3x - y + 2z = -11

-2x + y + 2z = -3

[ 2 1 -1 | 8 ] [ 1 (1/3) (-2/3) | (11/3) ]

[ -3 -1 2 | -11 ] ===> [ 0 1 2/5 | (13/5) ]

[ -2 1 2 | -3 ] [ 0 0 1 | -1 ]

Terminology

Row Echelon Form

Reduced Row Echelon Form

Gaussian Elimination

what on Earth?

Example problems

Flip a coin a bunch of times:

Expected number of flips before getting 6 heads in a row?

Roll a 6 sided die a bunch of times:

Expected number of rolls before getting 6 consecutive identical values in a row?

POTW (medium) 30 points

N (N<1000, N even) players sit around a table; the game begins with two opposite players having one die each. On each turn, the two players with dice roll them.

If a player rolls a 1, he passes the die to his neighbour on the left; if he rolls a 6, he passes the die to his neighbour on the right; otherwise, he keeps the die for the next turn.

The game ends when one player has both dice after they have been rolled and passed; that player has then lost.

What is the expected number of turns the game lasts?

Give your answer rounded to ten significant digits.

POTW (medium) 30 points

N (N<1000, N even) players sit in circle.

Players at opposite sides start with 6-side die

Players roll the dice at the same time:

• 1 => pass die to the left

• 6 => pass die to the right

• 2,3,4,5 don't mean anything

Find the expected number of times until 1 player has both dice

POTW (hard) 50 points

An infinitely long random string of digits:

p1p2p3p4p5p6p7p8p9p10p11p12p13p14p15...

Every integer will occur as a substring in X at some index Q

Ex: X = 12501672... Q2501= 2 , Q67 = 6

Given integer N ( N has <= 18 digits ),

Find the expected value of QN

Hint: use Knuth-Morris-Pratt Pattern Matching

POTW (hard) 50 points, hints

It can be proven that the expected value is always integer (doesn't mean I know how)

For large N:

built-in double is not precise enough

Use high precision decimals (BigDecimal) or integers during gaussian elimination

Hint: Use gaussian elimination for small N

then find pattern.