Approximate Bayesian inference for discretelyobserved continuous-time multi-state models

Andrea Tancredi

Sapienza Universita di Roma

Padova, 18-2-2019

Multi-state modelsMulti-state Markov modelsMulti-state semi-Markov models

Approximate Bayesian computationSequential ABCABC model choiceThe summary statistics for Markov and semi-Markov models

Examples and applicationsABC for Markov models: simulated dataABC for semi-Markov models: simulated dataReal data example: ABC for Markov and semi-Markov models

Multi-state models

• Continuous time processes {X (t), t ≥ 0} with state spaceS = {1, 2, . . . ,S}.

• Multi-state models can be defined via the transition intensityfunctions

qrs(t,Ft) = limδt→0

P{X (t + δt) = s|X (t) = r ,Ft}δt

where Ft is the past history up to time t.

• Partially observed paths. Let xi = (xi 0, xi 1, . . . , xi M) be theobserved states at the times ti 0 < ti,1 < · · · < ti,M for the i-th unit.

x(t)

●

●

●

●

●12

3

0 2 4 8 12

x(t)

●

●

●

●

●

12

3

0 2 4 8 12

x(t)

●

●

12

3

0 2 4 8 12

x(t)

●

●

●

12

3

0 2 4 8 12

x(t)

●

●

●

●

●● ●

● ● ●

12

3

0 2 4 8 12

x(t)

●

●

●

●

●

●

● ● ● ●

12

3

0 2 4 8 12

x(t)

●

●

●

● ● ● ●

12

3

0 2 4 8 12

x(t)

●

●

●

●

● ●

● ●

12

3

0 2 4 8 12

Multi-state Markov models

Markov model:

P{X (t + δt) = s|X (t) = r ,Ft} =

{γrsδt + o(δt) s 6= r1 + γrrδt + o(δt) s = r

γrs is the rate at which transitions from r to s occur,∑

s γrs = 0 ∀r .

Let G be the matrix

G =

γ11 . . . γ1S

.... . .

...γS1 . . . γSS

then prs(t) = P(Xu+t = s|Xu = r) = P(Xt = s|X0 = r) is the (r , s)element of the exponential matrix exp (tG ) =

∑∞m=0(tG )m/m!

• Likelihood inference (Kalbfleish and Lawless, 1985, JASA) providedby the R-package msm (Jackson, 2011, JSS)

Multi-state semi-Markov models

Semi-Markov processes generalize Markov models by removing theconstant transition rates assumption

qrs(t,Ft) = qrs(u) = limδt→0

P{X (t + δt) = s|X (t) = r ,T ∗ = t − u}δt

where T ∗ denotes the time of entry into the current state

• qrs(u) = prsq(u;αr , γr ) where q(u;αr , γr ) is the hazard function ofa Weibull or Gamma r.v.

• Markov models are nested

• Full paths easy to simulate (sojourn times + Markov chain)

• Untractable likelihood when the full trajectories are not observed!!

Hidden semi-Markov models

• Instead of the true xij the state oij is reported.

• Let ers = P(O(t) = r |X (t) = s) be the misclassificationprobabilities.

• Hidden Markov and semi-Markov models may be fitted to theobservations oij

• Hidden Markov and semi-Markov models still easy to simulate.(sojourn times + Markov chain+misclassification)

Approximate Bayesian Computation

• Approximate Bayesian Computation (ABC) is a particularAccept-Reject algorithm well suited for Bayesian inference

• Let π(θ) be the prior distributiion and let f (x |θ) be the likelihoodfunction

• Exact likelihood free rejection sampling (x discrete r.v).

I For i = 1, . . .N doI repeat θ′ ∼ π() and z ∼ f (·|θ′)I until z = xI set θi = θ′

• The accepted values θi are i.i.d. draws from π(θ|x)

• ABC rejection sampling

I For i = 1, . . .N doI repeat θ′ ∼ π() and z ∼ f (·|θ′)I until d{s(z), s(x)} ≤ εI set θi = θ′

The accepted values (θ, z) are i.i.d. draws from

πε(θ, z |x) =π(θ)f (z |θ)IAε,x (z)∫

Θ

∫Zπ(θ)f (z |θ)IAε,x (z)

dzdθ

where Aε,x = {z : d(s(z), s(x)) ≤ ε} and

πε(θ|x) =

∫πε(θ, z |x)dz ≈ π(θ|x)

• Sequential ABC (Population Monte Carlo-ABC)Beaumont et al. 2009, BKA, Marin et al. 2012, Stat. & Comp.

1. For t = 1 obtain a regular ABC sample of size N with threshold ε1

I Let ω1i = 1/N for i = 1, . . . ,N

I Let Σ1 be 2 times the empirical variance of the θ1i .

2. For t = 2, . . . ,T , fix εt ≤ εt−1

I For i = 1, . . .N do

I repeat θ′ ∼∑N

j=1 ω(t−1)j N

(θ

(t−1)j ,Σt−1

)and z ∼ f (·|θ′)

I until d{s(z), s(x)} ≤ εI set θ

(t)i = θ′

I set ω(t)i ∝ π(θ

(t)i )/

∑Nj=1 ω

(t−1)j φd

(θ

(t)i ; θ

(t−1)j ,Σt−1

)I Let Σt be 2 times the empirical weighted variance of the θ

(t)i

The weighted sample{θ

(t)i , ω

(t)i

}∼ πεt (θ|x)

ABC and model choice

Suppose we need to compare the two models

M1 x ∼ f1(x |θ1) θ1 ∈ Θ1 θ1 ∼ π1(θ1)

M2 x ∼ f2(x |θ2) θ2 ∈ Θ2 θ2 ∼ π2(θ2).

with

B(s(x)) =

∫Θ1

f s1 (s(x)|θ1)π1(θ1)dθ1∫Θ2

f s2 (s(x)|θ2)π2(θ2)dθ2=

m1(s(x))

m2(s(x))

(mi (s(x)) is the acceptance rate of the exact ABC performed with s(x))

B(s(x)) is able to select asymptotically the right model if under thewrong model, the asymptotic mean of the summary statistics s(x) cannotbe equal to its true mean under the correct model, except for the nestedcase. (Marin et a.JRSSB 2014)

By the importance sampling identity

mε(s(x)) =

∫Θ

∫DIAε,x (z)f (z |θ)π(θ)dzdθ

=

∫Θ

∫DIAε,x (z)

π(θ)

q(θ)f (z |θ)q(θ)dzdθ

we can estimate mε(s(x)) by

mε(s(x)) =1

Nt

Nt∑`=1

IAε,x (z t` )π(θt`)

q(θt`).

where (z t` , θt`) for ` = 1, . . . , Nt , are the Nt draws (z , θ) performed during

the t-th step to obtain the N accepted points. That is

mε(s(x)) =1

Nt

N∑i=1

π(θ(t)i )∑N

j=1 ω(t−1)j φd

(θ

(t)i ; θ

(t−1)j ,Σt−1

)

Summary statistics

Let nr (tj) be the frequency of the state r at time tj and nrs(tj , tj′) be thetransition count from state r at time tj to state s at time tj′ , that is

nr (tj) =n∑

i=1

I (xij = r) and nrs(tj , tj′) =n∑

i=1

I (xij = r , xij′ = s)

Summary statistics are the empirical marginal and conditionalprobabilities

pr (tj) =nr .(tj)

nprs(tj , tj′) =

nrs(tj , tj′)

nr .

for j ′ > j (type 1) or j ′ = j + 1 (type 2)

The distance between the observed vector p(x) and the simulated p(z) is

d(p(x), p(z)) = (p(x)− p(z))tΣ−1d (p(x)− p(z))

where Σd is the estimated covariance of the p(x)− p(z) under aMultinomial sampling

Summing the distances for the occupation probabilities with those for theconditional probabilities, we obtain the chi-square statistics for testinghomogeneity between the transition matrices n(tj , tj′ ; x) and n(tj , tj′ ; z).

The total distance d(s(x), s(z) is the sum of the distances d(p(x), p(z))

M∑j=1

S∑r=1

n(tj , x)n(tj , z)

nr.(tj , x) + nr.(tjz)(pr (tj , x)− pr (tj , z))2 +

M−1∑j=1

M∑j′=j+1

S∑r,s=1

nr.(tj , tj′ , x)nr.(tj , tj′ , z)

nrs(tj , tj′ , x) + nrs(tj , tj′ , z)(prs(tj , tj′ , x)− prs(tj , tj′ , z))2

Irregular visit patterns

• Missing and censoring patterns are respected through ABCsimulation

• When individuals have different observation times tij due to irregularvisit patterns with respect to tj , summary statistics may be obtainedby projecting on tj the state observed at time t∗ij where

t∗ij = arg minl |tj − til |

• If the panel data are obtained in the same way on the simulatedtrajectories, the simulated occupation and transition probabilitieswill be the result of the same data reduction of the observed ones.

ABC for Markov models

With 2 states and ti j − ti j−1 = ∆ it is possible to obtain analytically theexponential matrix exp ∆G

1

γ12 + γ21

(γ21 + γ12e

−{∆(γ21+γ12)} γ12(1− e−{∆(γ21+γ12)})γ12(1− e−{∆(γ21+γ12)}) γ12 + γ22e

−{∆(γ21+γ12)}

)and the likelihood function is

L(G) =(γ21 + γ12e

−{∆(γ21+γ12)})n11

(γ12(1 − e−{∆(γ21+γ12)})

)n12

×(γ12(1 − e−{∆(γ21+γ12)})

)n21(γ12 + γ22e

−{∆(γ21+γ12)})n22

× (γ12 + γ21)−(n11+n12+n21+n22)

0.1 0.2 0.3 0.4 0.5

0.2

0.4

0.6

0.8

1.0

γ12

γ 21

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

● ●

●

● ●

●● ●

●

●

● ●●

●

●●

●

●

●

●● ●

●

●

●

●●

●

● ●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●●●

●●

●

● ●

● ●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●●

●

●

●●

●

●

●

● ● ●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●●

●●●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●● ●●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●●

●

●●

●●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

● ●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●●

● ●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

● ●●

●

●

●●

●●

●

●●●

●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●●

●

●

●

●

●●●

●

●●●

●●

●

●

●●

●

●

●

●

●●

●●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●●●

●●

●●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●●●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●●●

●

●●

●

●

●●

●●●

●●●

●●

●●

●

●●●●

●

●

●

●

●

●●●

●

●

●

●●

●●●●

●●

●●●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●●

●

●●●●

●

●●

●

●●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●●●●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●● ●

●●

●●

●●

●

●●

●

●

●●● ●

●

●●

●

●

●●

●

●●

●●

●

●●

●●

●

●●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●●●●

●

●

●

●

●

●●

●●

●

●

●●

●

●●

●●

●●

●

●

●●●●●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●●

●

●●●

●

●●

●●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●●●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●●

●●●

●●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

●●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●●

●●●

●●

●

●●

●●

●

●

●●

●●

●

●●

●

●●●

●●

●

●

●

●

●●

●

●● ●

●●

●

●

●●

●●●

●

●●

●

●●

●●●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●●

●●●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●

●●

●

●●

●

●

●

●●

●●

●●

●

●●●●

●

●●●

●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●●

●

● ●

●●●●●●●

●●

●

●

●●

●●

●

●

●●

●●

●

●

●

●●

●●

●●

●

●●

●●

●

●●

●

●

●●●●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●●

●

●

●●●●

●

●●

●

●

●●

●

●

●●

●●

●

●●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

• Ni = 10, ∆ = 1, n = 20 γ12 = 1 γ21 = 0.5

• Excheangeable prior on the log parameters

• ABC-PMC with T = 5 iterations (ε5 = 0!!)

• Model comparison: M1 : γ12 > 0, γ21 > 0 vs M2 : γ12 = γ21, B = 3.4

● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●

0 2 4 6 8

010

2030

time

id

● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●

0 2 4 6 8

010

2030

time

id

● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ●● ●● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ●●● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ●●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●

0 2 4 6 8

010

2030

time

id

● ●● ● ●●● ● ● ●● ● ●● ● ● ● ● ●●●● ●● ● ● ●● ●●●● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ●● ● ● ●●● ● ● ● ●● ●● ● ● ● ●● ● ●● ●● ● ● ●● ●● ●● ● ●●● ● ● ● ●●● ●●● ● ● ● ● ●●● ●● ●● ●●● ● ●● ● ● ●● ● ● ●● ●● ● ●● ● ● ●● ● ●● ● ●● ● ●● ●●●●● ● ● ● ● ● ●● ●● ●● ●● ●● ● ● ●●● ● ● ● ● ●●● ●● ●● ●● ● ● ● ● ●● ●● ● ●● ● ●● ●● ●● ● ●● ● ●● ●● ● ● ●● ● ●● ●●● ● ● ● ●● ● ● ●●●● ●● ● ● ●● ● ●●● ● ●● ● ● ● ●●●● ●●● ●● ● ●●● ● ●● ● ●●● ● ●●● ● ●● ● ● ● ●●●● ●●● ● ● ●● ●● ● ● ● ● ●●● ● ●●● ● ● ●● ● ●● ●

0 2 4 6 8

010

2030

time

id

γ12

0.0 0.1 0.2 0.3 0.4 0.5

04

812

γ12

0.0 0.1 0.2 0.3 0.4 0.5

04

812

γ12

0.0 0.1 0.2 0.3 0.4 0.5

04

812

γ12

0.0 0.1 0.2 0.3 0.4 0.5

04

812

γ21

0.0 0.2 0.4 0.6 0.8 1.0

02

46

γ21

0.0 0.2 0.4 0.6 0.8 1.0

02

46

γ21

0.0 0.2 0.4 0.6 0.8 1.0

02

46

γ21

0.0 0.2 0.4 0.6 0.8 1.0

02

46

Irregular follow-up times: the first column shows the follow-up times andthe other two columns the ABC (histogram) and exact posterior (solidline) distributions for γ12 and γ21

ABC for semi Markov models: simulated data

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●●●●

●●●●●●●●●

●●

●

●

●

●

●●

1 2 3 4 5 6 7

−2

−1

01

2

log(γ12)

●

●

●●

●

●

●●●

●

●

●

●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●

●

●

●●

●

●●●●●●●●●●●●●

●

●

●

●

●

●●

●●

●●

●

●

●●●●●●●●●●●●

●

1 2 3 4 5 6 7

−1

01

23

4

log(α1)

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

1 2 3 4 5 6 7

−6

−5

−4

−3

−2

−1

log(γ13)

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●●●●●

●

●●●

●

●

●●

●

●

●

●

●●

●

●●●●●

●●●●

●●●

●

●

●

●

1 2 3 4 5 6 7

−5

−4

−3

−2

−1

01

log(γ31)

●

●

●●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●●●

●

●

●

●●●●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

●

●●●●●●●

1 2 3 4 5 6 7

−1

01

23

45

log(α2)

●●●●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●●

●●●

●

●●●●

●●●

● ●●

●

1 2 3 4 5 6 7

−4

−3

−2

−1

log(γ23)

• Three-states model ( healthy, ill, dead) with follow-up times at 3,6,12,24and 60 months

• 100 patient histories drawn from Weibull model qrs(u) = γrsαr (uγr )αr−1

with (γ12, α1, γ13, γ21, α2, γ23) = (0.25, 1.4, 0.05, 0.04, 0.7, 0.1)

• Evolution of the ABC-PMC algorithm

Type 1 summary statisticsn γ12 α1 γ13 γ21 α2 γ23

50 0.26 (0.05) 1.25 (0.19) 0.07 (0.02) 0.05 (0.03) 0.79 (0.16) 0.09 (0.02)100 0.25 (0.04) 1.29 (0.16) 0.07 (0.02) 0.04 (0.02) 0.77 (0.11) 0.09 (0.02)500 0.25 (0.02) 1.34 (0.08) 0.05 (0.01) 0.04 (0.01) 0.71 (0.05) 0.10 (0.01)1000 0.25 (0.01) 1.36 (0.07) 0.05 (0.01) 0.04 (0.01) 0.71 (0.04) 0.10 (0.01)

Type 2 summary statisticsn γ12 α1 γ13 γ21 α2 γ23

50 0.25 (0.05) 1.28 (0.19) 0.07 (0.02) 0.05 (0.02) 0.79 (0.13) 0.09 (0.02)100 0.25 (0.03) 1.32 (0.17) 0.06 (0.02) 0.05 (0.02) 0.75 (0.11) 0.09 (0.02)500 0.25 (0.02) 1.36 (0.09) 0.05 (0.01) 0.04 (0.01) 0.73 (0.06) 0.10 (0.01)1000 0.25 (0.01) 1.37 (0.07) 0.05 (0.01) 0.04 (0.01) 0.70 (0.04) 0.10 (0.01)

Mean and standard deviation for the ABC-PMC posterior mediansobtained with different sample sizes n ( 100 data sets for each n)

●●●●

●

●

●

●●●●

●●●●●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Semi Markov Statistics=Type 1

n

p(M

=S

emi M

arko

v|y) ●●

●●●●

●

●●

●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Semi Markov Statistics=Type 2

n

p(M

=S

emi M

arko

v|y)

●

●

●●●●●

●●

●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Hidden semi Markov Statistics=Type 1

n

p(M

=H

idde

n se

mi M

arko

v|y)

●●

●

●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Hidden semi Markov Statistics=Type 2

n

p(M

=H

idde

n se

mi M

arko

v|y)

●●

●

●

●●●

●

●

●

●●●●

●●●●●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Markov Statistics=Type 1

n

p(M

=S

emi M

arko

v|y)

●●

●

●

●

●●

●●

●●

●

●●

●

●

●

●

●●●●

●

●●●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Markov Statistics=Type 2

n

p(M

=S

emi M

arko

v|y)

●

●

●

●●

●

●●

●

●

●●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Hidden Markov Statistics=Type 1

n

p(M

=H

idde

n se

mi M

arko

v|y)

●

●

●

●●●

●

●

●

●

●

●●

●

●

●●

50 100 500 1000

0.0

0.2

0.4

0.6

0.8

1.0

Model=Hidden Markov Statistics=Type 2

n

p(M

=H

idde

n se

mi M

arko

v|y)

Semi-Markov vs Markov. Posterior probabilities for the semi-Markovmodel with data generated under both the models

Breast cancer data

0 10 20 30 40 50 60

01

23

4

Time (months)

Sta

te o

f the

cha

in

List of models

• M1 Markov model with γ12, γ13, γ21, γ23

• M2 Markov model with γ12, γ13 = 0, γ21, γ23

• M3 Weibull semi-Markov model with γ12, γ13, γ21, γ23, α1, α2

• M4 Weibull semi-Markov model with γ12, γ13 = 0, γ21, γ23, α1, α2

B21 = 5.1, B43 = 5.6 instantaneous transitions 1→ 3 cannot occur

0 5 10 20 300.00

0.15

µ10.0 0.2 0.4 0.6 0.8 1.0

02

4

p13

0 5 10 150.0

0.2

0.4

µ20.0 0.2 0.4 0.6 0.8 1.0

02

4p23

M1

0 5 10 20 300.00

0.15

µ1

0 5 10 150.0

0.3

µ20.0 0.2 0.4 0.6 0.8 1.0

02

4

p23

M2

0 5 10 20 300.00

0.10

µ10.0 0.2 0.4 0.6 0.8 1.00.

01.

53.

0

p13

0 5 10 150.00

0.20

µ20.0 0.2 0.4 0.6 0.8 1.0

02

p23

M3

0 5 10 20 300.00

0.10

µ1

0 5 10 150.00

0.20

µ20.0 0.2 0.4 0.6 0.8 1.0

02

4

p23

M4

Breast cancer data. Posterior distributions for the parameters of theMarkov and semi-Markov models Mi , i = 1, . . . , 4 with type 1 summarystatistics (solid line), type 2 (dashed line) and MCMC (dotted line, onlyfor Markov models). B42 = 1.2, no strong evidence for the semi-Markovmodel

Cardiac allograft vasculopathy data

• Large data set with 622 subjects and 2846 observations

• Irregular follow-up times (discretization needed, missing data)

• Four states with progressive transitions (reverse transitions recoding)

• List of models:

I M1 Markov modelI M2 Semi-Markov model with Weibull distributions for all

transient statesI M3 Semi-Markov model with Weibull distributions only for the

first state

• Corresponding hidden Markov and hidden semi-Markov models withmisclassification for adjacent states (without reverse transitionrecoding)

●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ● ● ● ●

0 5 10 15

0.0

0.4

0.8

P11(0, t)

years

●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ● ● ● ● ● ●

● ●

● ● ● ● ●● ●

●● ●

● ● ● ● ● ●

0 5 10 15

0.0

0.4

0.8

P12(0, t)

years

● ●

● ●

● ● ● ● ●● ●

●● ●

● ● ● ● ● ● ● ●●

●● ● ●

●● ●

●●

● ● ● ● ● ● ● ●

0 5 10 15

0.0

0.4

0.8

P13(0, t)

years

● ●●

●● ● ●

●● ●

●●

● ● ● ● ● ● ● ● ●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ●

0 5 10 15

0.0

0.4

0.8

P14(0, t)

years

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●● ● ● ● ●

0 5 10 15

0.0

0.4

0.8

P11(0, t)

years

●

●

●

●

●

●

●

●

●

●

●

●●

● ●● ● ● ● ● ● ●

● ●

● ● ● ● ●● ●

● ● ●● ● ● ● ● ●

0 5 10 15

0.0

0.4

0.8

P12(0, t)

years

● ●

● ●

● ● ● ● ●● ●

● ● ●● ● ● ● ● ● ● ●

●●

● ● ●● ● ●

● ● ● ● ● ● ● ● ● ●

0 5 10 15

0.0

0.4

0.8

P13(0, t)

years

● ●●

●● ● ●

● ● ●● ● ● ● ● ● ● ● ● ● ●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ●

0 5 10 15

0.0

0.4

0.8

P14(0, t)

years

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ●

CAV data occupation probabilities. The solid lines and the dotted linesrepresent the posterior mean and 100 realizations under the semi-Markovmodel M3. The dashed lines are the posterior means under the Markovmodel M1 and the dot-dashed lines are the empirical estimates.

B21 = 1.92, B31 = 4.07, B32 = 2.1, showing support for the semi-Markovmodel M3 with respect to M1. Similar indications wiith the hidden models

Discussion

• Pairwise likelihoods are characterized by the transition counts. ABCbased on these summaries produces an adjusted posterior based onthe intractable pairwise likelihood.

• Covariate extension

• ABC is not suit with a large number of states. In fact, thedimension of the summary statistics scales quadratically in thenumber of states, and ABC may fail

• MCMC for semi-Markov: path reconstruction with proposals basedon Markov approximations embedded in Metropolis-Hastings step(joint work wirh Rosario Barone)

References

• Beaumont, M. A., Cornuet, J. M., Marin, J.-M., and Robert, C. P.(2009). Adaptive approximate Bayesian computation. Biometrika 96,983-990.

• Jackson, C. H. (2011). Multi-state models for panel data: the msmpackage for R. Journal of Statistical Software 38, 1-29.

• Tancredi, A. (2019). Approximate Bayesian inference for discretelyobserved continuous-time multi-state models. Biometrics.

• Titman, A. C. (2014). Estimating parametric semi-Markov models frompanel data using phase-type approximations. Statistics and Computing24, 155-164.

• Titman, A. C. and Sharples, L. D. (2010). Semi-Markov models withphase-type sojourn distributions. Biometrics 66, 742-752.

THANK YOU!