Bayesian inference

Jean Daunizeau

Wellcome Trust Centre for Neuroimaging

16 / 05 / 2008

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Introduction

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Bayesian paradigm (1) : theory of probability

Degree of plausibility desiderata:- should be represented using real numbers (D1)- should conform with intuition (D2)- should be consistent (D3)

a=2b=5

a=2

• normalization:

• marginalization:

• conditioning :(Bayes rule)

Bayesian paradigm (2) : Likelihood and priors

generative model m

Likelihood:

Prior:

Bayes rule:

Bayesian paradigm (3) : Model comparison

“Occam’s razor” :

Principle of parsimony :« plurality should not be assumed without necessity »

mo

de

l evi

de

nce

p(y

|m)

space of all data sets

y=f(

x)y

= f(

x)

x

Model evidence:

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Specification of priors

lack of information/entropy

order/structure

priors = population behaviour / information available before having observed the data

• subjectivist approach : “informative” priors• objectivist approach : “non-informative” priors

Principle of maximum entropy :

find the probability distribution functionwhich maximizes the entropy under some constraints (normalization, expectation, …)

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Hierarchical models (1) : principle

••• hierarchy

causality

Hierarchical models (2) : directed acyclic graphs (DAGs)

Hierarchical models (3) : univariate linear hierarchical model

•••

prior posterior

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Sampling methods

MCMC example: Gibbs sampling

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Variational methods

VB/EM/ReML: find (iteratively) the “variational” posterior q(θ)which maximizes the free energy F(q)under some mean-field approximation:

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

aMRI segmentation

grey matterPPM of belonging to… CSFwhite matter

1

2

3

212

223

iiy

class variances

class priorfrequencies

ith voxel label

class means

ith voxel value

1

2

3

212

223

iiy

1

2

3

212

223

iiy

class variances

class priorfrequencies

ith voxel label

class means

ith voxel value

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

fMRI time series analysiswith spatial priors

observations

GLM coeff

prior varianceof GLM coeff

prior varianceof data noise

AR coeff(correlated noise)

ML estimate of W VB estimate of W

aMRI smoothed W (RFT)

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

Dynamic causal modelling

state-space formulation:

Overview of the talk

1 Probabilistic modelling and representation of uncertainty

1.1 Introduction

1.2 Bayesian paradigm

1.3 Specification of priors

1.4 Hierarchical models

2 Numerical Bayesian inference methods

2.1 Sampling methods

2.2 Variational methods (EM, VB)

3 SPM applications

3.1 aMRI segmentation

3.2 fMRI time series analysis with spatial priors

3.3 Dynamic causal modelling

3.4 EEG source reconstruction

EEG source reconstruction

Homoapriorius

Homopragmaticus

Homofrequentistus

Homosapiens

Homobayesianis

y ,y

y

y