Monte Carlo Methods for Random/StochasticPDE

Jingchen Liu

Department of StatisticsColumbia University

Summer School in Monte Carlo Methods for Rare EventsBrown University, Providence RI

June 17, 2016

1 / 42

Overview

I Ordinary/partial differential equations

I Imprecision and measurement error

I Stochastic and random partial differential equation

2 / 42

Overview

I Ordinary/partial differential equations

I Imprecision and measurement error

I Stochastic and random partial differential equation

2 / 42

Overview

I Ordinary/partial differential equations

I Imprecision and measurement error

I Stochastic and random partial differential equation

2 / 42

Random PDE

u : D → R

I Differential equationL(u) = 0

subject to certain boundary conditions.

I Randomness L(u, ξ) = 0

I u(x , ω) : D ×Ω→ R

3 / 42

Random PDE

u : D → R

I Differential equationL(u) = 0

subject to certain boundary conditions.

I Randomness L(u, ξ) = 0

I u(x , ω) : D ×Ω→ R

3 / 42

Random PDE

u : D → R

I Differential equationL(u) = 0

subject to certain boundary conditions.

I Randomness L(u, ξ) = 0

I u(x , ω) : D ×Ω→ R

3 / 42

Interesting quantities

I The tail probabilities of functional P(Γ(u) > v)

I Some examples

Γ(u) = supx∈D

u(x), ‖u(x)‖p, supx∈D|∇u(x)|...

I Level crossing of random functions

4 / 42

Interesting quantities

I The tail probabilities of functional P(Γ(u) > v)

I Some examples

Γ(u) = supx∈D

u(x), ‖u(x)‖p, supx∈D|∇u(x)|...

I Level crossing of random functions

4 / 42

Interesting quantities

I The tail probabilities of functional P(Γ(u) > v)

I Some examples

Γ(u) = supx∈D

u(x), ‖u(x)‖p, supx∈D|∇u(x)|...

I Level crossing of random functions

4 / 42

Asymptotic regimes

I L(u, ξ(ω)) = 0⇒ uξ(x)

I Fixed noise and large deviations P(Γ(uξ) > v) as v → ∞.

I Small noise: L(u, σξ) = 0 as σ→ 0.

5 / 42

Asymptotic regimes

I L(u, ξ(ω)) = 0⇒ uξ(x)

I Fixed noise and large deviations P(Γ(uξ) > v) as v → ∞.

I Small noise: L(u, σξ) = 0 as σ→ 0.

5 / 42

Asymptotic regimes

I L(u, ξ(ω)) = 0⇒ uξ(x)

I Fixed noise and large deviations P(Γ(uξ) > v) as v → ∞.

I Small noise: L(u, σξ) = 0 as σ→ 0.

5 / 42

Korteweg-de Vries (KdV) Equation

I The wave u(x , t)

u : R× [0,T ]→ R

I Korteweg-de Vries (KdV) equation

∂tu − 6u∂xu + ∂xxxu = 0

6 / 42

Korteweg-de Vries (KdV) Equation

I The wave u(x , t)

u : R× [0,T ]→ R

I Korteweg-de Vries (KdV) equation

∂tu − 6u∂xu + ∂xxxu = 0

6 / 42

Wave

7 / 42

Wave

8 / 42

Wave

9 / 42

Wave

10 / 42

Solitary wave

11 / 42

Solitary wave

12 / 42

Solitary wave

13 / 42

Solitary wave

u(x , t) =W(x − vt)

14 / 42

Solitary wave

I ∂tu − 6u∂xu + ∂xxxu = 0

I u(x , t) =W(x − vt)

⇒ d2Wdx2

= 3W2 + vW + c0

15 / 42

Solitary wave

I ∂tu − 6u∂xu + ∂xxxu = 0

I u(x , t) =W(x − vt)

⇒ d2Wdx2

= 3W2 + vW + c0

15 / 42

Solitary wave

1

1The picture is published at http://kaheel7.com

16 / 42

Stochastic Korteweg-de Vries (KdV) Equation

I The stochastic version

∂tu − 6u∂xu + ∂xxxu = ξ(t)

where ξ(t) is a Gaussian process.

I P(supt∈[0,T ] u(x , t) > b), for a given x

17 / 42

Stochastic Korteweg-de Vries (KdV) Equation

I The stochastic version

∂tu − 6u∂xu + ∂xxxu = ξ(t)

where ξ(t) is a Gaussian process.

I P(supt∈[0,T ] u(x , t) > b), for a given x

17 / 42

Solitary wave

2

2The picture is published at http://whybecausescience.com

18 / 42

Initial condition

I The equation

u(x , t = 0) = −2κ2sech2(κx)

where sech(x) = 2ex+e−x .

I If ξ(t) = 0, then

u(x , t) = −2κ2sech2κ(x − 4κ2t)

19 / 42

Initial condition

I The equation

u(x , t = 0) = −2κ2sech2(κx)

where sech(x) = 2ex+e−x .

I If ξ(t) = 0, then

u(x , t) = −2κ2sech2κ(x − 4κ2t)

19 / 42

Solution to the random PDE

∂tu − 6u∂xu + ∂xxxu = ξ(t)

u(x , 0) = −2κ2sech2(κx)

I The solution

u(x , t) = f (t)− 2κ2sech2

κ(x − 4κ2t) + 6κ∫ t

0f (t)dt

where f (t) =

∫ t0 ξ(s)ds

20 / 42

Hueristics

u(x , t) = f (t)− 2κ2sech2

κ(x − 4κ2t) + 6κ∫ t

0f (s)ds

I Upper bound u(x , t) ≤ f (t)

I supt u(x , t) > v ⊂ sup f (t) > v

I P(supt u(x , t) > v) ≤ P(sup f (t) > v)

21 / 42

Hueristics

u(x , t) = f (t)− 2κ2sech2

κ(x − 4κ2t) + 6κ∫ t

0f (s)ds

I Upper bound u(x , t) ≤ f (t)

I supt u(x , t) > v ⊂ sup f (t) > v

I P(supt u(x , t) > v) ≤ P(sup f (t) > v)

21 / 42

Hueristics

u(x , t) = f (t)− 2κ2sech2

κ(x − 4κ2t) + 6κ∫ t

0f (s)ds

I Upper bound u(x , t) ≤ f (t)

I supt u(x , t) > v ⊂ sup f (t) > v

I P(supt u(x , t) > v) ≤ P(sup f (t) > v)

21 / 42

The relative difference

I Given sup f (t) > v

supt

f (t)− v = O(1/v)

andsupt

f (t)− supt

u(x , t) = O(e−εv )

22 / 42

The relative difference

I The relative difference

1− e−εb ≤ P(supt u(x , t) > v)

P(supt f (t) > v)≤ 1

I Approximation

P(supt

u(x , t) > v) ∼ P(supt

f (t) > v) ∼ Cv βe−v2/2σ2

T

where σ2T = supt Var(f (t))

23 / 42

The relative difference

I The relative difference

1− e−εb ≤ P(supt u(x , t) > v)

P(supt f (t) > v)≤ 1

I Approximation

P(supt

u(x , t) > v) ∼ P(supt

f (t) > v) ∼ Cv βe−v2/2σ2

T

where σ2T = supt Var(f (t))

23 / 42

Simulation from the change of measure

I Simulate τ ∈ Dτ ∼ h(t)

I Simulate f (τ) according to gτ(x)

I Simulate f (t) : t 6= τ given f (τ) under P

24 / 42

Simulation from the change of measure

I Simulate τ ∈ Dτ ∼ h(t)

I Simulate f (τ) according to gτ(x)

I Simulate f (t) : t 6= τ given f (τ) under P

24 / 42

Simulation from the change of measure

I Simulate τ ∈ Dτ ∼ h(t)

I Simulate f (τ) according to gτ(x)

I Simulate f (t) : t 6= τ given f (τ) under P

24 / 42

The change-of-measure-based analysis

I Let P be the original measure.

I The change of measure Q

dQ

dP=∫t∈D

gt(f (t))

ϕt(f (t))h(t)dt

where ϕt(x) is the marginal density of f (t), h(t) is a densityon D, and gt(x) is an alternative density.

25 / 42

The change-of-measure-based analysis

I Let P be the original measure.

I The change of measure Q

dQ

dP=∫t∈D

gt(f (t))

ϕt(f (t))h(t)dt

where ϕt(x) is the marginal density of f (t), h(t) is a densityon D, and gt(x) is an alternative density.

25 / 42

Choice of h and gt

I Let γ = v − 1/v

I The distribution

gt(x) = I (x > γ)ϕt(x)

P(f (t) > γ)

I The distribution

h(t) =P(f (t) > γ)∫

D P(f (t) > γ)dt

26 / 42

Choice of h and gt

I Let γ = v − 1/v

I The distribution

gt(x) = I (x > γ)ϕt(x)

P(f (t) > γ)

I The distribution

h(t) =P(f (t) > γ)∫

D P(f (t) > γ)dt

26 / 42

Choice of h and gt

I Let γ = v − 1/v

I The distribution

gt(x) = I (x > γ)ϕt(x)

P(f (t) > γ)

I The distribution

h(t) =P(f (t) > γ)∫

D P(f (t) > γ)dt

26 / 42

Rare-event simulation and importance sampling

I The change of measure:

dQγ

dP=

mes(Aγ)∫D P(f (t) > γ)dt

where Aγ = t : f (t) > γ.

27 / 42

Importance sampling (Li and L. 2015)

I Strongly efficient in computing the tail probabilities ofsup f (t) and sup u(x , t).

I Discretization

28 / 42

Importance sampling (Li and L. 2015)

I Strongly efficient in computing the tail probabilities ofsup f (t) and sup u(x , t).

I Discretization

28 / 42

Material Failure

I u(x): the shape of the material

I Ou(x): strain

I p(x): pressure

I a(x): material-specific coefficients

29 / 42

Material Failure

I The partial differential equation: x ∈ D

−O · a(x)Ou(x) = p(x)

I Dirichlet boundary condition u(x) = 0 for all x ∈ ∂D.

I log a(x) = ξ(x)

I Failure probability

P(

supx∈D|∇u(x)| > b

)

30 / 42

Material Failure

I The partial differential equation: x ∈ D

−O · a(x)Ou(x) = p(x)

I Dirichlet boundary condition u(x) = 0 for all x ∈ ∂D.

I log a(x) = ξ(x)

I Failure probability

P(

supx∈D|∇u(x)| > b

)

30 / 42

Material Failure

I The partial differential equation: x ∈ D

−O · a(x)Ou(x) = p(x)

I Dirichlet boundary condition u(x) = 0 for all x ∈ ∂D.

I log a(x) = ξ(x)

I Failure probability

P(

supx∈D|∇u(x)| > b

)

30 / 42

Material Failure

I The partial differential equation: x ∈ D

−O · a(x)Ou(x) = p(x)

I Dirichlet boundary condition u(x) = 0 for all x ∈ ∂D.

I log a(x) = ξ(x)

I Failure probability

P(

supx∈D|∇u(x)| > b

)

30 / 42

Material Failure – one dimensional example

I The ordinary differential equation: x ∈ [0, 1]

(a(x)u′(x))′ = −p(x)

I Spatial variation: a(x) = eξ(x), where ξ(x) is a Gaussianprocess.

31 / 42

The failure probability

I The failure probability

P

(supx∈D|u′(x)| > b

)I The displacement u(x) depends on the process a(x).

32 / 42

The failure probability

I The failure probability

P

(supx∈D|u′(x)| > b

)I The displacement u(x) depends on the process a(x).

32 / 42

Material Failure – Dirichlet condition

I Dirichlet condition: u(0) = u(1) = 0

I The solution:

u(x) =∫ x0 F (y)a−1(y)dy −

∫ 10 F (y )a−1(dy )dy∫ 1

0 a−1(dy )

∫ x0 a−1(y)dy ,

where F (x) =∫ x0 p(y)dy .

33 / 42

Material Failure – Dirichlet condition

I Dirichlet condition: u(0) = u(1) = 0

I The solution:

u(x) =∫ x0 F (y)a−1(y)dy −

∫ 10 F (y )a−1(dy )dy∫ 1

0 a−1(dy )

∫ x0 a−1(y)dy ,

where F (x) =∫ x0 p(y)dy .

33 / 42

Material Failure – Dirichlet condition

I The strain

u′(x) = a−1(x)

(F (x)−

∫ 10 F (y)a−1(y)dy∫ 1

0 a−1(y)dy

)= a−1(x)

[F (x)− Eξ(F (Y ))

]where a−1(x) = eξ(x).

34 / 42

Theorem: approximation for continuous force (L. and Zhou2011)

I The external force p(x) is a continuously differentiablefunction.

I x∗ = arg supx p(x).

Then, we have the approximation

P( supx∈[0,1]

|u′(x)| > b)

∼ P(|u′(0)| > b) + P(|u′(1)| > b) + P( sup|x−x∗|<ε

|u′(x)| > b).

35 / 42

Exact asymptotic approximation for continuous body force

I Let p(x∗)r−1er−12 = b. Then,

P( sup|x−x∗|<ε

|u′(x)| > 0) ∼ κ∗ × r−1/2 exp−r2/2.

I Let H0r0−1/2er0 = b. Then,

P(|u′(0)| > b) = κ0 × r0−1e−r0

2/2

I Let H1r1−1er1 = b. Then,

P(|u′(1)| > b) = κ1 × r1−1e−r1

2/2

36 / 42

Exact asymptotic approximation for continuous body force

I Let p(x∗)r−1er−12 = b. Then,

P( sup|x−x∗|<ε

|u′(x)| > 0) ∼ κ∗ × r−1/2 exp−r2/2.

I Let H0r0−1/2er0 = b. Then,

P(|u′(0)| > b) = κ0 × r0−1e−r0

2/2

I Let H1r1−1er1 = b. Then,

P(|u′(1)| > b) = κ1 × r1−1e−r1

2/2

36 / 42

Exact asymptotic approximation for continuous body force

I Let p(x∗)r−1er−12 = b. Then,

P( sup|x−x∗|<ε

|u′(x)| > 0) ∼ κ∗ × r−1/2 exp−r2/2.

I Let H0r0−1/2er0 = b. Then,

P(|u′(0)| > b) = κ0 × r0−1e−r0

2/2

I Let H1r1−1er1 = b. Then,

P(|u′(1)| > b) = κ1 × r1−1e−r1

2/2

36 / 42

Key components of the conditional distribution

I Where does the break occur or arg sup u′(x) =?I Where does ξ(x) attain it maximum?

I At what level?

37 / 42

Choice of h and gt

I The distribution h(t): mixture of three components

I The distribution gt(x).

38 / 42

Choice of h and gt

I The distribution h(t): mixture of three components

I The distribution gt(x).

38 / 42

High-dimensional PDE (L. et al. 2014)

−O · a(x)Ou(x) = p(x)

I Random index h(τ) ∼ Uniform(D)

I The distributiongτ(x) ∼ N(lτ, 1)

39 / 42

High-dimensional PDE (L. et al. 2014)

−O · a(x)Ou(x) = p(x)

I Random index h(τ) ∼ Uniform(D)

I The distributiongτ(x) ∼ N(lτ, 1)

39 / 42

The choice of lτ

I Given that ξ(τ) = lτ, the conditional field is

ξ(x) = lτ × C (x − τ) + r(x − τ)

where E [r(x)] = 0.

I Set r(x) ≡ 0 for the asymptotic analysis

I Choose ξ(x) = lτC (x − τ) such that sup |∇u(x)| = v .

40 / 42

The choice of lτ

I Given that ξ(τ) = lτ, the conditional field is

ξ(x) = lτ × C (x − τ) + r(x − τ)

where E [r(x)] = 0.

I Set r(x) ≡ 0 for the asymptotic analysis

I Choose ξ(x) = lτC (x − τ) such that sup |∇u(x)| = v .

40 / 42

The choice of lτ

I Given that ξ(τ) = lτ, the conditional field is

ξ(x) = lτ × C (x − τ) + r(x − τ)

where E [r(x)] = 0.

I Set r(x) ≡ 0 for the asymptotic analysis

I Choose ξ(x) = lτC (x − τ) such that sup |∇u(x)| = v .

40 / 42

The computation of lt

1. Set the initial guess l(0)τ = log v .

2. For l(n)τ , set

ξ(t) = l(n)τ C (t − τ)

and compute u(n)

l(n+1)τ = l

(n)τ − log sup |∇u(n)|+ log v .

3. Repeat step 2 until convergence.

41 / 42

The computation of lt

1. Set the initial guess l(0)τ = log v .

2. For l(n)τ , set

ξ(t) = l(n)τ C (t − τ)

and compute u(n)

l(n+1)τ = l

(n)τ − log sup |∇u(n)|+ log v .

3. Repeat step 2 until convergence.

41 / 42

The computation of lt

1. Set the initial guess l(0)τ = log v .

2. For l(n)τ , set

ξ(t) = l(n)τ C (t − τ)

and compute u(n)

l(n+1)τ = l

(n)τ − log sup |∇u(n)|+ log v .

3. Repeat step 2 until convergence.

41 / 42

Visualizing lt

Figure: Contour plot of the excursion level function lτ with homogeneousexternal force p ≡ 1 (left panel) and the discontinuous external forcep(x) = sign(x < 0.5) (right panel) for b = 4.

42 / 42