Approximate Bayesian inference as a gauge theory

Biswa Sengupta 1 2 3 Karl Friston 4

AbstractIn a published paper (Sengupta et al., 2016),we have proposed that the brain (and other self-organized biological and artificial systems) canbe characterized via the mathematical apparatusof a gauge theory. The picture that emerges fromthis approach suggests that any biological sys-tem (from a neuron to an organism) can be castas resolving uncertainty about its external mi-lieu, either by changing its internal states or itsrelationship to the environment. Using formalarguments, we have shown that a gauge theoryfor neuronal dynamics – based on approximateBayesian inference – has the potential to shednew light on phenomena that have thus far eludeda formal description, such as attention and thelink between action and perception. Here, we de-scribe the technical apparatus that enables such avariational inference on manifolds. Particularly,the novel contribution of this paper is an algo-rithm that utlizes a Schild’s ladder for paralleltransport of sufficient statistics (means, covari-ances, etc.) on a statistical manifold.

1. IntroductionA gauge theory is a physical theory that predicts how one ormore physical fields interact with matter. Every gauge the-ory has an associated Lagrangian (i.e., a function that sum-marizes the dynamics of the system in question), which isinvariant under a group of local transformations. ConsiderNewton’s laws of motion in an inertial frame of reference(e.g. a ball in an empty room). These laws are valid at ev-ery point in the room. This means that the dynamics andthe Lagrangian do not depend on the position of the ball.In this system, the dynamics will be invariant under trans-lations in space. These transformations – that preserve the

1Dept. of Bioengineering, Imperial College London 2Dept.of Engineering, University of Cambridge 3Cortexica Vision Sys-tems, UK 4Wellcome Trust Centre for Neuroimaging, Univer-sity College London. Correspondence to: Biswa Sengupta<b.sengupta@imperial.ac.uk>.

ICML 2017 Computational Biology Workshop, Sydney, Australia,2017. Copyright 2017 by the author(s).

Lagrangian – are said to be equipped with gauge symmetry.In short, a symmetry is simply an invariance or immunityto changes in the frame of reference.

Gauge theories originate from physics; however, they couldbe applied to countless fields of biology: cell structure,morphogenesis, and so on. Examples that lend them-selves to a gauge theoretic treatment include recent simu-lations of embryogenesis (Friston et al., 2015) and the self-organisation of dendritic spines (Kiebel & Friston, 2011).Although these examples appear unrelated, both can be for-mulated in terms of a gradient ascent on variational freeenergy. In other words, we may be looking at the samefundamental behaviour in different contexts. Here, we fo-cus on the central nervous system (CNS). Can we sketch agauge theory of brain function?

When attempting to establish what aspect of CNS functionmight be understood in terms of a Lagrangian, the varia-tional free energy looks highly plausible (Friston, 2010).The basic idea is that any self-organizing system, at non-equilibrium steady-state with its environment, will appearto minimize its (variational) free energy, thus resisting anatural tendency to disorder. This formulation reducesthe physiology of biological systems to their homeostasis(and allostasis); namely, the maintenance of their states andform, in the face of a constantly changing environment.

If the minimisation of variational free energy is a ubiqui-tous aspect of biological systems could it be the Lagrangianof a gauge theory? This (free energy) Lagrangian hasproved useful in understanding many aspects of functionalbrain architectures; for example, its hierarchical organisa-tion and the asymmetries between forward and backwardconnections in cortical hierarchies. In this setting, the sys-tem stands for the brain (with neuronal or internal states),while the environment (with external states) is equippedwith continuous forces and produces local sensory pertur-bations that are countered through action and perception(that are functionals of the gauge field).

In summary, the free energy formalism rests on a statisticalseparation between the agent (the internal states) and theenvironment (the external states). Agents suppress free en-ergy (or surprise) by changing sensory input, by acting onexternal states, or by modifying their internal states throughperception. In what follows, we show that the need to

arX

iv:1

705.

0661

4v2

[q-

bio.

NC

] 1

2 N

ov 2

017

Approximate Bayesian inference as a gauge theory

minimize variational free energy (and hence achieve home-ostasis) acquires a useful logical-mathematical formalism,when framed as a gauge theory.

2. MethodsVariational free-energy formalism

The variational free energy formalism assumes that anagent minimizes the entropy of its sensory states s ∈ S.Only through its sensory receptors can a biological sys-tem access the states of its environment; in other words,sensory states form a veil (technically, a Markov blanket)between the system’s internal states θ ∈ Θ and its envi-ronment (external states) ψ ∈ Ψ. By bounding the en-tropy of its sensory states, the system confines the entropyof its environment. Under the assumption of ergodicity,this entropy is the long-term average of surprise. Crucially,the system cannot calculate this quantity directly becauseit has to marginalize over the external states that cause sen-sory input. The objective then becomes to obtain a lowerbound on the marginal likelihood ln p(s) by approximat-ing it using a parametric probability distribution (for ex-ample, a Gaussian distribution) q(ψ) over the (unknown)external states that are hidden behind the Markov blanket(i.e., hidden causes of sensations). In short, the LagrangianL = − ln p(s) is minimised by bounding the surprise us-ing the variational free energy F(s, θ) , F(s, q) of thedistribution q(ψ|θ) where θ are the sufficient statistics orparameters of the variational distribution. In the case of aGaussian distribution the sufficient statistics are simply themean and the co-variances {µ,

∑} ⊂ θ.

The variational distribution that minimises free energycan be expressed in terms of an Euler-Lagrange actionδS(F(s, q)) = 0 ⇔ ∇F = 0, implying that the gradientdescent on the variational free-energy manifold leads us

to the most optimal representation of the external states.A numerical scheme to solve such a variational problemis generalised (Bayesian) filtering (Friston et al., 2008).The Bayesian perspective follows because our Lagrangianis also known as (the negative logarithm of) Bayesianmodel evidence. In other words, minimising free energy isequivalent to maximising model evidence.

Variational inference on manifolds

The evolution of the variational free-energy could be de-scribed on a Riemann manifold by augmenting the first or-der gradient flow using a Fisher information metric (Amari,1995; Tanaka, 2001). On a Euclidean manifold, the mini-mization of variational free-energy involves

−∇F‖∇F‖

= limε→0

1

εarg mindθ:‖dθ‖6ε

F(θ + dθ) (1)

This simply says that the flow of parameters (e.g. meansand co-variance of a Normal distribution; {µ, σ, . . .} ∈ θ)will induce the largest change in free-energy under a unitchange in parameters. Notice that the inner products aredefined on a Euclidean manifold.

Classical results from information geometry (Cencov’scharacterisation theorem) tell us that, for manifolds basedon probability measures, a unique Riemannian metric ex-ists – the Fisher information metric. In statistics, Fisher-information is used to measure the expected value of theobserved information. Whilst the Fisher-information be-comes the metric for curved probability spaces, the distancebetween two distributions is provided by the Kullback-Leibler (KL) divergence. It turns out that if the KL-divergence is viewed as a curve on a curved surface, theFisher-information becomes its curvature:

KLsym(θ, θ′) = Eθ

[log

q(ψ |θ)q(ψ |θ′)

]+ Eθ′

[log

q(ψ |θ′)q(ψ |θ)

]= dθT g(θ)dθ +O(dθ3)

gij(θ) =

+∞∫−∞

q(ψ, θ)∂ ln q(ψ, θ)

∂θi

∂ ln q(ψ, θ)

∂θjdψ

KL[θ0 + δθ : θ0].=

1

2gij(θ0)(δθ)2

(2)

Gradient descent on such a manifold then becomes the so-lution of arg min

dθF(θ + dθ), subject to KLsym(θ, θ +

dθ) < ε; i.e., the direction of the highest decrease in the

free-energy, for the smallest change in the KL divergence.The solution of this optimization problem yields Amari’snatural gradient that replaces the Euclidean gradient ∇Fby its Riemannian counterpart ∇F = gij(θ) - 1∇F . This

Approximate Bayesian inference as a gauge theory

Өi-1

Өi+1

Өi

Hi

Hi-1

Gi

Ө i-1

Ө i+1

Ө i

H i

H i-1

G i

H i-1

Levi-Civita connection(gauge �eld)

H i-1

A B C

Figure 1. . Conjugate gradient-descent algorithm on manifolds.(A) New parameters (θ) are selected by performing gradient de-scent on orthogonal sub-spaces with gradient G and the descentdirection H . (B) On a Riemannian manifold, minimization alonglines (as in a Euclidean sub-space described in A) is replaced byminimization along geodesics. This creates a problem, in that Hi

and Hi−1 are in two different tangent spaces and thus cannot beadded together. (C) Vector addition as in Eqn. 3 is undefined on aRiemannian manifold. Addition is replaced by exponential map-ping followed with parallel transport described using a co-variantgauge field (Levi-Civita connection; see text)

derivative is invariant under re-parameterisation of the ap-proximate probability distribution, thereby helping us tobreak symmetries on the variational free-energy manifold.

This formulation has two important consequences – (a)from classical results in statistics, pre-conditioning of thefree-energy gradient by the Fisher-information tells us thatthe variance of the estimator is bounded from below bythe Fisher-information (Cramer-Rao bound) and (b) undera Normal distribution approximation of the posterior dis-tribution, precision-weighted prediction errors under a Eu-clidean manifold are replaced by asymptotic dispersion andprecision-weighted prediction errors under a Riemannianmanifold.

Such constructs are already instantiated in advancedBayesian filtering schemes, such as the Statistical Para-metric Mapping (SPM) code-base (available from http://www.fil.ion.ucl.ac.uk/spm/) using Fisher-scoring – the gradient of variational free-energy is pre-multiplied by the inverse Fisher information metric. Noticethat the metric in Fisher-scoring is simply the variance ofthe score function, while our derivation of the metric in-cludes not only the metric for the likelihood but also that ofthe prior (instantiated as the Hessian of the prior).

The question that we now ask is whether we can deducean optimization scheme that enables us to traverse the free-energy landscape? In other words, find the geodesic to localminima in the sub-manifold. There are two routes one cantake to increase the statistical efficiency of the implicit op-timisation – first, we can formulate the Hessian operator onthe Riemannian manifold in terms of the Laplace-Beltramioperator (Section S3.1 in (Sengupta et al., 2016)) or we canretain a first-order approximation and formulate descent di-rections that are orthogonal to the previous descent direc-

H i-1ExpӨ(Hi-1) Ө

TM

M

LogӨ(Hgi-1)

M

θI θFθ1 θ2 θ3

m1 m2 m3

n1 n2 n3 n4

H H

A B

Figure 2. Parallel transport on Riemannian manifolds. (A) TheRiemann exponential map is used to map a vector field H fromTM → M whilst a logarithmic map is used to map the vec-tor field from M → TM. (B) Graphical illustration of paralleltransporting a vector field H using a Schild’s ladder (see text fordetails).

tions. Such Krylov sub-spaces are well-known in numeri-cal analysis with the conjugate gradient-descent algorithmproviding one such example (Figure 3). Routinely usedin optimization, conjugate gradient descent methods havebeen used for gradient descent on manifolds traced out byenergy functions such as the variational free-energy (Hens-man et al.; Honkela et al., 2010). Simply such a schemeamounts to,

θi = θi−1 + αHi

Hi = −Gi + βHi−1

β =∇Ti ∇i∇Ti−1∇i−1

Riemannize→ ∇Ti ∇i∇Ti−1∇i−1

(3)

For β we have used the Fletecher-Reeves instantiation on acurved manifold; other update rules such as Polak-Ribiere,Hestenes-Stiefel or Dai-Yuan can be similarly lifted to aRiemannian manifold by altering the implicit norm. All ofthese conjugate gradient descent formulas have a problem– one cannot add two vector fields Hi and Hi−1 on a Rie-mannian manifold. This is because they exist on differenttangent manifolds. Hi−1 should undergo parallel transportto the tangent manifold containingHi using a connection (agauge) field. In our case, this is the Levi-Civita connectiondescribed in Section S2 in (Sengupta et al., 2016).

Parallel-transport requires the solution of a second-orderdifferential equation. Analysis shows us that the natu-ral gradient is the first order approximation of the paral-lel transport – that we pursue in terms of solving geodesicequations for the sufficient statistics. For the Laplace ap-proximation, we could derive the Christoffel symbols ana-lytically (Section S4 in (Sengupta et al., 2016)), while formore complicated probability distributions we need to re-sort to a generic transport procedure (Figure 2). Namely,we use the Riemann exponential map for mapping the vec-

Approximate Bayesian inference as a gauge theory

tor field on the tangent manifold to the geodesic describedon the manifold (TM → M), whereas a Riemann loga-rithmic map represents the transformations of vector-fieldsfrom the manifold to the tangent manifold (M→ TM),

expθ(A) = θ1/2 exp(θ−1/2Aθ−1/2)θ1/2

logθ(A) = θ1/2 log(θ−1/2Aθ−1/2)θ1/2

(4)

The geodesic is first approximated using standard projec-tion method (Hairer et al., 2004). Then using the expo-nential and logarithm maps a Schild’s ladder (Misner et al.,1973) is instantiated as following: Let θI and θF denotethe initial and final points on the geodesic that the vec-tor field is to be transported to. We start by calculatingn1 = expθI (H) and the midpointm1 between the geodesicsegment joining n1 and θ1. We then trace out the geodesicfrom θI through m1 for twice its length, tracing out a newpoint n2. This scheme is repeated until we reach θF . Af-ter the vector field H has been parallel transported to θF ,we are in a position to use parameter updates as detailed inEqn. 3. This scheme will be made available in upcomingversions of SPM for a variety of dynamical systems.

3. ResultsSensory entropy as a Lagrangian

The variational free energy formalism uses the fact that bi-ological systems must resist the second law of thermody-namics (i.e. a tendency to disorder), so that they do not de-cay to equilibrium. In a similar vein to Maxwell’s demon,an organism reduces its entropy through sampling the envi-ronment – to actively minimise the self information or sur-prise of each successive sensory sample (this surprise is up-per bounded by free energy). By doing so, it places a boundon the entropy of attributes of the environment in which itis immersed. Variational free energy operationalises thisbound by ensuring internal states of the system become areplica (i.e., a generative model) of its immediate environ-ment. This can be regarded as a formulation of the goodregulator hypothesis, which states that every good regula-tor of a system must be a model of that system.

We know that a gauge theory would leave the La-grangian invariant under continuous symmetry transforma-tions. Therefore, a gauge theory of the brain requires the in-teraction among three ingredients: a system equipped withsymmetry, some local forces applied to the system and oneor more gauge fields to compensate for the local perturba-tions that are introduced. The first ingredient is a systemequipped with symmetry: for the purposes of our argu-ment, the system is the nervous system and the Lagrangian

is the entropy of sensory samples (which is upper-boundedby variational free energy, averaged over time). The lo-cal forces are mediated by the external states of the world(i.e., through sensory stimuli). The gauge fields can thenbe identified by considering the fact that variational free-energy is a scalar quantity based on probability measures.

How does neuronal activity follow the steepest descent di-rection to attain its free energy minimum? In other words,how does it find the shortest path to the nearest minimum?As the free energy manifold is curved there are no orthonor-mal linear co-ordinates to describe it. This means the dis-tance between two points on the manifold can only be de-termined with the help of the Fisher information metric thataccounts for the curvature. Algebraic derivations (S3 in(Sengupta et al., 2016)) tell us that, in such free-energylandscapes, a Euclidean gradient descent is replaced bya Riemann gradient, which simply weights the Euclideangradient by its asymptotic variance.

In the free energy framework, when the posterior prob-ability is approximated with a Gaussian distribution (theLaplace approximation; S4 in (Sengupta et al., 2016)), per-ception and action simply become gradient flows driven byprecision-weighted prediction errors. Here, prediction er-rors are simply the difference between sensory input (localperturbations) and predictions of those inputs based uponthe systems internal states (that encode probability distri-butions or Bayesian beliefs about external states that causesensory input). Mathematically, precision-weighted pre-diction errors emerge when one computes the Euclideangradient of the free energy with respect to the sufficientstatistics. In a curvilinear space, the precision-weightedprediction errors are replaced by dispersion and precisionweighted prediction errors. This says something quite fun-damental – perception cannot be any more optimal thanthe asymptotic dispersion (inverse Fisher information) re-gardless of the generative model. In statistics, this result isknown as the Cramer-Rao bound of an estimator. In otherwords, the well-known bound (upper limit) on the precisionof any unbiased estimate of a model parameter in statisticsemerges here as a natural consequence of applying infor-mation geometry. In the context of the Bayesian brain, thismeans there is a necessary limit to the certainty with whichwe can estimate things. We will see next, that attaining thislimit translates into attention.

Notice that the definition of a system immersed in its envi-ronment can be extended hierarchically, wherein the gaugetheory can be applied at a variety of nested levels. At everystep, as the Lagrangian is disturbed (e.g., through changesin forward or bottom-up sensory input), the precision-weighted compensatory forces change to keep the La-grangian invariant via (backward or top-down) messages.In the setting of predictive coding formulations of varia-

Approximate Bayesian inference as a gauge theory

tional free energy minimisation, the bottom-up or forwardmessages are assumed to convey prediction error from alower hierarchical level to a higher level, while the back-ward messages comprise predictions of sufficient statisticsin the level below. These predictions are produced to ex-plain away prediction errors in the lower level. From theperspective of a gauge theory, one can think of the localforces as prediction errors that increase variational free en-ergy, thereby activating the gauge fields to explain awaylocal forces.

In this geometrical interpretation, perception and actionare educed to form cogent predictions, whereby minimiza-tion of prediction errors is an inevitable consequence ofthe nervous system minimising its Lagrangian. Crucially,the cognitive homologue of precision-weighting is atten-tion, which suggests gauge fields are intimately related to(exogenous) attention. In other words, attention is a forcethat manifests from the curvature of information geometry,in exactly the same way that gravity is manifest when thespace-time continuum is curved by massive bodies. In sum-mary, gauge theoretic arguments suggest that attention (andits neurophysiological underpinnings) constitutes a neces-sary weighting of prediction errors (or sensory evidence)that arises because the manifolds traced out by the path ofleast free energy (or least surprise) are inherently curved.

4. DiscussionComplementary to our work in (Sengupta et al., 2016), thispaper advances the sketch of an algorithm that utilizes par-allel transport for statistical manifolds governed by vari-ational free-energy. It is well known that Riemann conju-gate gradient method differs from its Euclidean counterpart(i.e., for small step-sizes where the geometry is close to be-ing Euclidean) only by third order terms (Edelman et al.,1998; Bonnabel, 2013; Raskutti & Mukherjee, 2015) – en-abling these algorithms to converge quadratically near theextremum point. Nevertheless, our algorithm facilitates usto compute discrete approximations of the parallel trans-port, without requiring us to have any knowledge of thetangent structure of the manifold. This makes it tractablefor those manifolds where one need not assume the pres-ence of an ambient space.

From the point of neuroscience, we consider the principleof free energy minimization as a candidate gauge theorythat prescribes neuronal dynamics in terms of a Lagrangian.Here, the Lagrangian is the variational free energy, whichis a functional of a probability distribution encoded by neu-ronal activity. This probabilistic encoding means that neu-ronal activity can be described by a path or trajectory ona manifold in the space of sufficient statistics (variablesthat are sufficient to describe a probability distribution). Inother words, if one interprets the brain as making infer-

ences, the underlying beliefs must be induced by biophys-ical representations that play the role of sufficient statis-tics. This is important because it takes us into the realmof differential geometry (see S2 in (Sengupta et al., 2016)),where the metric space – on which the geometry is defined– is constituted by sufficient statistics (like the mean andvariance of a Gaussian distribution). Crucially, the gaugetheoretic perspective provides a rigorous way of measuringdistance on a manifold, such that the neuronal dynamicstransporting one distribution of neuronal activity to anotheris given by the shortest path. Such a free energy manifoldis curvilinear and finding the shortest path is a non-trivialproblem – a problem that living organisms appear to havesolved. It is at this point that the utility of a gauge theoreticapproach appears; suggesting particular solutions to theproblem of finding the shortest path on curved manifolds.The nature of the solution prescribes a normative theory forself-organized neuronal dynamics. In other words, solvingthe fundamental problem of minimizing free energy – interms of its path integrals – may illuminate not only howthe brain works but may provide efficient schemes in statis-tics and machine learning

Variational or Monte Carlo formulations of the Bayesianbrain require the brain to invert a generative model of thelatent (unknown or hidden) causes of sensations (see S3in (Sengupta et al., 2016)). The implicit normative the-ory means that neuronal activity (and connectivity) max-imises Bayesian model evidence or minimises variationalfree energy (the Lagrangian) – effectively fitting a genera-tive model to sensory samples. This entails an encoding ofbeliefs (probability distributions) about the latent causes, interms of biophysical variables whose trajectories trace outa manifold. In (deterministic) variational schemes, the co-ordinates on this manifold are the sufficient statistics (likethe mean and covariance) of the distribution or belief whilefor a (stochastic) Monte Carlo formulation, the coordinatesare the latent causes themselves. The inevitable habitat ofthese sufficient statistics (e.g., neuronal activity) is a curvedmanifold.

This curvature (and associated information geometry) mayhave profound implications for neuronal dynamics andplasticity. As the Lagrangian is a function of beliefs (prob-abilities), the manifold that contains this motion is neces-sarily curved. This means, neuronal dynamics, in a localframe of reference, will (appear to) be subject to forcesand drives (i.e., Levi-Civita connections). For example, themotion of synaptic connection strengths (sufficient statis-tics of the parameters of generative models) depends uponthe motion of neural activity (sufficient statistics of beliefsabout latent causes), leading to experience-dependent plas-ticity. A more interesting manifestation may be attentionthat couples the motion of different neuronal states in a waythat depends explicitly on the curvature of the manifold (as

Approximate Bayesian inference as a gauge theory

Figure 3. Screenshot of the SPM toolbox that primarily uses vari-ational free energy minimization for a wide-variety of problemsin neuroscience, non-linear dynamics, reinforcement learning,amongst others.

measured by Fisher information).

In brief, a properly formulated gauge theory should, inprinciple, provide the exact form of neuronal dynamics andplasticity. These forms may reveal the underlying simplic-ity of many phenomena that we are already familiar with,such as event-related brain responses, associative plasticity,attentional gating, adaptive learning rates and so on.

A. SoftwareVariational algorithms for (nonlinear) regression, proba-bilistic graphic models of varying complexity and vari-ational reinforcement learning (active inference; MarkovDecision Processes) have been released via the statisticalparametric mapping (SPM) toolbox http://www.fil.ion.ucl.ac.uk/spm/. Figure 3 provides a screenshotof a wide variety of demos that detail a variety of problems.Along with variational methods the SPM suite also in-cludes stochastic methods (mci toolbox) such as LangevinMonte Carlo, Manifold Monte Carlo, Riemannian MarkovChain Monte Carlo (MCMC), Hamiltonian MCMC (Sen-gupta et al., 2015a), population MCMC (Sengupta et al.,2015b), geometric Annealed Importance Sampling algo-rithms (Penny & Sengupta, 2016), amongst others. Paralleltransport for variational models shall be made available insubsequent releases.

ReferencesAmari, S. Information geometry of the EM and EM algorithms

for neural networks. Neural Networks, 8(9):1379–1408, 1995.

Bonnabel, Silvere. Stochastic gradient descent on Riemannianmanifolds. IEEE Transactions on Automatic Control, 58(9):2217–2229, 2013.

Edelman, A., Arias, T.A., and Smith, S.T. The geometry of algo-rithms with orthogonality constraints. SIAM Journal on MatrixAnalysis and Applications, 1998.

Friston, K. J., Trujillo-Barreto, N., and Daunizeau, J. DEM: avariational treatment of dynamic systems. Neuroimage, 41(3):849–885, 2008.

Friston, Karl, Levin, Michael, Sengupta, Biswa, and Pezzulo,Giovanni. Knowing one’s place: a free-energy approach topattern regulation. Journal of the Royal Society Interface, 12(105):20141383, 2015.

Friston, K.J. The free-energy principle: a unified brain theory?Nat Rev Neurosci, 11(2):127–38, 2010.

Hairer, E., Lubich, C., and Wanner, G. Geometric Numerical In-tegration: Structure Preserving Algorithms for Ordinary Dif-ferential Equations. Springer, 2004.

Hensman, J., Rattray, M., and Lawrence, N. D. Fast variationalinference in the conjugate exponential family. In NIPS.

Honkela, Antti, Raiko, Tapani, Kuusela, Mikael, Tornio, Matti,and Karhunen, Juha. Approximate Riemannian conjugate gra-dient learning for fixed-form variational bayes. Journal of Ma-chine Learning Research, 11:32353268, 2010.

Kiebel, Stefan J and Friston, Karl J. Free energy and dendriticself-organization. 2011.

Misner, C., Thorne, K., and Wheeler, J. Gravitation. W.H. Free-man, 1973.

Penny, Will and Sengupta, Biswa. Annealed importance samplingfor neural mass models. PLoS Comput Biol, 12(3):e1004797,2016.

Raskutti, Garvesh and Mukherjee, Sayan. The information geom-etry of mirror descent. IEEE Trans. Information Theory, 61(3):1451–1457, 2015.

Sengupta, B., Friston, K. J., and Penny, W. D. Gradient-basedMCMC samplers for dynamic causal modelling. Neuroimage,2015a.

Sengupta, B., Friston, K. J., and Penny, W. D. Gradient-freeMCMC methods for dynamic causal modelling. Neuroimage,112:375–81, 2015b.

Sengupta, Biswa, Tozzi, Arturo, Cooray, Gerald K., Douglas,Pamela K., and Friston, Karl J. Towards a neuronal gauge the-ory. PLoS Biology, 14(3):1–12, 03 2016.

Tanaka, T. Information geometry of mean-field approximation,pp. 259–273. The MIT Press, 2001.