Contributions of Canonical ModelingContributions of Canonical ModelingTo To StochasticsStochastics and Statisticsand Statistics
Eberhard O. VoitBiomedical Engineering
Georgia Tech and Emory3 May 2007
0 90 1800.9
1
1.1
Canonical Modeling in a Stochastics Seminar?
Voit talking about Stochastics?
Other Mismatches?
http://imagecache2.allposters.com/images/pic/AWI/NR9512~Target-1974-Posters.jpg
στóχος στοχαστικóς
“able to hit the target”“of keen mind”
“smart”Jasper Johns
Part 1: Canonical ModelingDefinitionPower-law modelsRecasting
Part 2: Deterministic versus StochasticDualityDeterministic chaos
Part 3: S-DistributionPropertiesModeling possibilities
Overview
Canonical Modeling
Definition (Merriam Webster Dictionary):
Model (verb): “to produce a representation or simulation of
<using a computer to model a problem>”
Canonical:“conforming to a general rule
or acceptable procedure”
Canonical Modeling
Examples:
Linear Dynamic Model:
Nonlinear Canonical (Dynamic) Models:Combinations of:
DifferentiationLogarithmSum
UXAX +⋅=&
Canonical Modeling
Examples:
Lotka-Volterra:
∑
∑
∑
=
=
=
+=
+==
+==
n
jjiji
i
n
jjijiiii
i
n
jjiijiii
i
Xbadt
Xd
XbaXXXdt
dX
XXbXaXdt
dX
1
1
1
ln
// &
&
Canonical Modeling
Examples:
Log-Lin Model:
Lin-Log Model:
Simplified: )ln(
ln1/
lnln1/
1
10
00
00
10
00
00
j
mn
jiji
i
mn
j j
jij
i
ii
i
i
i
mn
j j
jij
i
ii
i
i
i
Xbadt
dX
XX
ee
Jdt
dXJv
XX
ee
Jdt
dXJv
∑
∑
∑
+
=
+
=
+
=
+=
⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎟⎠
⎞⎜⎜⎝
⎛+==
⎟⎟⎠
⎞⎜⎜⎝
⎛+⎟⎟
⎠
⎞⎜⎜⎝
⎛+==
ε
ε
Canonical Modeling
Examples:
“Half-System”
= “Riccati-System”:
∑
∏
+
=
+
=
+=
=
mn
jjijii
mn
j
fjii
XfX
XX ij
1
1
)ln()ln()ln( γ
γ
&
&
Formulation of a Canonical Model for Complex Systems
Xi
Vi+ Vi
–−+ −== ii
ii VV
dtdX
X&
Approximate Vi+ and Vi
– in a logarithmic coordinate system
Biochemical Systems Theory (BST); Canonical ModelingSavageau, 1969
Log Xi
Log Vi+/-
Why not Use “True” Mechanisms and Rate Functions
E(1)
EAB(3)
EQ(4)
EA(2)
k12
k23
k41
k34
k14
k43
k21
k32 EPQ
A+B P+Q
(B)(P)(Q)AB coef.
coef.BB coef.
coef.BQBQ coef.
coef.BPQ
(A)(B)(P)AB coef.
coef.ABP
(P)(Q)AB coef.A coef.
A coef.constant
constantcoef.Q
coef.Qcoef.PQ
(B)(Q)AB coef.
coef.BB coef.
coef.BQ(A)(P)AB coef.
coef.AA coef.
coef.AP
(Q)AB coef.A coef.
A coef.constant
constantcoef.Q
(P)AB coef.A coef.
A coef.AP coef.
AP coef.coef.P(A)(B)
AB coef.AB coef.
(B)AB coef.B coef.(A)
AB coef.A coef.
AB coef.A coef.
A coef.constant
(P)(Q)num.1num.2
AB coef.num.1 (B)(A)
AB coef.num.1
⎟⎟⎠
⎞⎜⎜⎝
⎛××+
⎟⎠⎞
⎜⎝⎛+
⎟⎟⎠
⎞⎜⎜⎝
⎛×××+
⎟⎠⎞
⎜⎝⎛ ×+⎟
⎠⎞
⎜⎝⎛ ×+
⎟⎠⎞
⎜⎝⎛ ××+
⎟⎠⎞
⎜⎝⎛ ××+⎟
⎠⎞
⎜⎝⎛+
⎟⎠⎞
⎜⎝⎛+⎟
⎠⎞
⎜⎝⎛+⎟
⎠⎞
⎜⎝⎛ ×
⎟⎠⎞
⎜⎝⎛ ×⎟
⎠⎞
⎜⎝⎛
=-
v
from Schultz (1994)
Is an approximation “allowed”?
A+B P+Q
How Bad of an Approximation is this?
On one hand...
“True”process:
Molecular dynamics“Firing” of some individual reaction Ri in ΔtChemical Master EquationMild assumption on medium homogeneityStochastic mass action descriptionAverage rate: Poisson processNo Brownian motion: Langevin equationScale time: Itô formalismLarge numbers of molecules:
deterministic mass action lawQuasi-steady-state assumption:
Michaelis-MentenLocally approximate with power-law
How Bad of an Approximation is this?
On the other hand...
Only two assumptions:Deterministic formulation is o.k.System is (locally) spatially homogeneous
Then:Taylor theory guarantees that power-law is: perfect at operating point;good close to operating point.
Biological experience shows that range ofvalid approximation is often quite wide
Alternative Power-Law Formulations
mniiimniii hmn
hhi
gmn
ggii XXXXXXX ++
++ β−α= ,21,21 ... ... 2121&
S-system Form:
Xi
Vi1+ Vi1
–
Vi,p+ Vi,q
–
−+ ∑∑ −== ijiji
i VVdt
dXX&
Alternative Power-Law Formulations
mniiimniii hmn
hhi
gmn
ggii XXXXXXX ++
++ β−α= ,21,21 ... ... 2121&
S-system Form:
Xi
Vi1+ Vi1
–
Vi,p+ Vi,q
–
−+ ∑∑ −== ijiji
i VVdt
dXX&
Generalized Mass Action Form:
∑ ∏±= ijkfjiki XX γ&
Structure Determined by Parameter Values
X1 X 2 X 3 X 4
X1 X 2 X 3 X 4
2112gXα
4121412gg XXα g41 < 0
Crucial consequence :Identification of structure becomes parameter estimation
g41 = 0
Interesting Feature of S-systems:Steady-State Equations Linear
Define Yi = log(Xi):
mnmniiii
mnmniiii
YhYhYhYgYgYg
++
++
+++β=
+++α
,2211
,2211
loglog
S-system highly nonlinear, but steady-state equations linear.
IIDDD YAAbAY ⋅⋅−⋅= −− 11
0 ... ... ,21,212121 =β−α= ++
++mniiimniii h
mnhh
ig
mngg
ii XXXXXXX&
Characterization of Steady States
IIDDD YAAbAY ⋅⋅−⋅= −− 11
Slight change in an input exclusively affects YI.
ID
D
AA
bY
⋅=
∂∂
−1 Gains (w.r.t. independent variables)
1−= DA
∂∂ D
YI
Y
Sensitivities (w.r.t. rate constants)
Computation of eigenvalues for stability analysis; easy criteria for Hopf bifurcations
Applications of S-systems and BST
Pathways: purines, glycolysis, citric acid, TCA, red blood cell,...metabolic engineering; optimization, general design and operating principles
Genes: circuitry, regulation,…
Genome: explain expression patterns upon stimulus
Growth, immunology, pharmaceutical science, forestry, ...
Math: recasting, function classification, bifurcation analysis,...
Statistics: S-system representation, S-distribution, trends;applied to seafood safety, marine mammals, health economics
Big Question:
Canonical models appear to be very restrictive. So, what is the range of dynamics that can be modeled with them (e.g., S-systems)?
“Recasting”
mniiimniii hmn
hhi
gmn
ggii XXXXXXX ++
++ β−α= ,21,21 ... ... 2121&
S-system Form:
Various phenomena are special cases:
radioactive decaymass-action kineticsallometryexponential growth
Observations
2NKrrNN −=&E.g., logistic growth:
Gompertz growth function:
Auxiliary Variables
)]exp(exp[)( tVtV f αβ −−=
)exp( tk ααβ −=
kk
kVV
α−=
=&
&
(special case of an S-system)
t-Distribution
2/)1(2
1
2
21
1)(+−
⎥⎦
⎤⎢⎣
⎡+
⎟⎠⎞
⎜⎝⎛Γ
⎟⎠⎞
⎜⎝⎛ +
Γ=
ν
νν
ν
πνttf
(special case of an S-system)
)(,, 32
21 tfXtXctX =+=+= ν
31
2131
23
12
1
)1()1(
22
1
XXXXXcX
cXX
X
−− +−+=
−=
=
νν&
&
&
Noncentral t-Distribution
)(
2
21
)2/exp(),,(2/)1(
2
2
tSt
tf+−
⎥⎦⎤
⎢⎣⎡
+⎟⎠⎞
⎜⎝⎛Γ
⎟⎠⎞
⎜⎝⎛ +
Γ−
=ν
νν
ν
ν
πνδδν
j
j tt
j
j
tS ⎥⎦
⎤⎢⎣
⎡
+⎟⎠⎞
⎜⎝⎛ +
Γ
⎟⎠⎞
⎜⎝⎛ ++
Γ= ∑
∞
=2
0
2
!2
12
1
)(ν
δν
ν
Noncentral t-Distribution (cont’d)
j
j tt
j
j
ttf ⎥
⎦
⎤⎢⎣
⎡
+⎟⎠⎞
⎜⎝⎛ +
Γ
⎟⎠⎞
⎜⎝⎛ ++
Γ⋅⎥⎦
⎤⎢⎣⎡
+⎟⎠⎞
⎜⎝⎛Γ
⎟⎠⎞
⎜⎝⎛ +
Γ−
= ∑∞
=
+−
20
2/)1(
2
2 2
!2
12
1
2
21
)2/exp(),,(ν
δν
ν
νν
ν
ν
πνδδν
ν
532
89
532
8
72
22
72
212
7
175
5.126
765.1
25
34
31
2131
23
12
1
)2/exp(),,(
)2/exp(
)1(
)1()1(
221
XXtfX
XXX
XXcXXXX
XXXX
XXXX
XX
XXXXXcX
cXXX
δδν
δ
νδνδ
δνν
νδ
νν
−==
−=
−=
+=
=
=
+−+=
−=
=
−−
−−
−
−−
&
&
&
&
&
&
&
&
central t densitycumulative central t
cumulative noncentral tnoncentral t density
Noncentral t-Distribution (cont’d)
Added Value:
Can compute:DensityCumulativeMean, Variance, MomentsQuantilesPower curves
Can do the same for virtually allprobability density functions!
Theorem (Savageau and Voit, 1987)
Let
be a set of differential equations, wherein fi is composed of sums and products of elementary functions, or nested elementary functions of elementary functions. Then there is a smoothchange of variables that recasts (*) into an S-system of the form
Recasting
),...,,( 21 nii ZZZfZ =&0)0( ii ZZ = ni ,...,2,1= (*)
∏∏==
−=m
j
hji
m
j
gjii
ijij XXX11
βα&0)0( ii XX = mi ,...,2,1=
where αi and βi are non-negative and gij and hij are real.
Recasting (cont’d)
Recasting “embeds” systems of arbitrary differential equationsin a higher-dimensional space.
The structure of the recast set is that of an S-system.
Initial values confine the solution within the higher-dimensional space onto a trajectory that correspondsexactly to the solution of the original system.
Of note: linearity of steady state equations is lost; system matrix AD no longer invertible; some variables 0.
Recasting (cont’d)
Not surprising, recasting is also possible to the GMA form
∏=
=m
j
kjii
ijXX1
γ&
Surprising, recasting can be continued to even simpler forms, such as the Half-system or Riccati system:
∑ ∏±= ijkfjiki XX γ&
or even:}1,0{
1
∈= ∏=
ij
m
j
ejii eXX ijη&
Recasting (cont’d)
It is also possible to recast to the Lotka-Volterra form:
∑=
+⋅=n
jjijiii XbbXX
10 )(& mi ,...,2,1=
(Peschel and Mende, 1986).
It is also possible to recast to the Generalized Lotka-Volterra form: GMA system where each equation contains every power-law term of the system (where necessary with kinetic order 0) (Hernández-Bermejo and Fairén, 1997)
Note: It seems not possible to recast to log-lin and lin-log forms.
All the above “canonical forms” are ultimately combinations of
∂, +, · , and log
Intriguing Consequence
Because of recasting, all differentiable nonlinearities are ultimately combinations of
∂, +, · , and log
Value Added through Recasting
Homogeneous mathematical formatPossibility of classificationCustomized methods, softwareEfficient Taylor algorithm for integration Efficient Taylor algorithm for dynamic sensitivity analysisStreamlined Lie-group analysis for conserved quantitiesStatistics:
quantilespower curvestime-varying distributions
e.g., dynamic confidence intervalsStability analysis of non-polynomial vector fields
(Papachristodoulou and Prajna, 2006)
Deterministic versus Stochastic
Well-known “duality”
Deterministic systems can be intrinsicallyunpredictable and indistinguishable
from (stochastic) chaos.
Stochastic systems have average features that arewell characterized and essentially deterministic.
Ultimate example: Gas laws.
Example: Rössler Oscillator
41
411
4
4131
14
13
13
122
4321
5.29
6.49465,1
5.0
XXXX
XXXXXX
XXX
XXXX
−−
−−−−−−
−=
−=
−=
−=
μμ
λλ μλμλ
μλ
&
&
&
&
μ/14
3
2
1
50)0(
1)0(47)0(25)0(
=
===
X
XXX
0 90 180
0
30
60
X1 X2
10 25 40
40
50
60
X1
X2
Not Chaotic Enough? Coupled Rössler Oscillators
0 90 180
0.9
1
1.1
X10
R1
R2
t
t
Y1
Y2
Y2
Blue-Sky Catastrophe:
Other Example
)sin(25.0 3 tAxxx ⋅=−+ &&&
GMA-form
325.0)2( xxysAyyx
−+−−⋅=
=
&
&
2)cos(2)sin(
+=+=
tcts
stccts
−=−=−==2)sin(
2)cos(&
&
New variables3
2121
21
25.0)2( −+−−⋅=
⋅+⋅=⋅=
xxysA
zzzzyzzy
&&&
S-system form1
13
2
121
21
)(
]25.0)2([−
−
⋅−=
⋅−−⋅=
⋅=
zxxz
zysAz
zzx
&
&
&
GMA form325.0)2( xxysAy
yx−+−−⋅=
=
&
&
Blue-Sky Catastrophe (cont’d)
stccts
−=−=−==2)sin(
2)cos(&
&
0 250 500
-1.5
0
1.5x
A = 0.26498 A = 0.264510 250 500
-1.5
0
1.5x
S-Distribution
Possible to recast essentially all known continuous univariatedistributions.
Problem: Often too many auxiliary variables
Alternative: Approximate cumulatives with one S-system equation, “S-distribution”:
)( hg FFF −⋅= α& 5.0)( =mF
Initial value F(m): location; α ~ 1/σ; g, h: shape
S-Distribution: Many Shapes Possible
0 10 20
0
0.25
0.5f1 f2 f3
001.0,1
)(
)(
)(
53
2.133
5.12
75.022
5.01
25.011
==
−=
−=
−=
iF
FFF
FFF
FFF
α
α
α
α
&
&
&
S-Distribution: Good Fits
Example: noncentral t-distribution
S-Distribution: Classification
S-Distribution: Classification
Distributions classifiable by two shape parameters (g, h):
S-Distribution: Other Features
Approximation of discrete distributionsRelatively unbiased data specification (identification
of appropriate distribution)Various estimation methods (e.g., MLE)Time-dependent confidence intervals (definition of
“normal” in health statistics)Extension to several variatesExtensions to multimodalityExtensions to greater shape flexibility
S-Distribution: Rather General Random Number Generator
Use “quantile form” of desired S-distribution
)(1)( hg
hg
FFdFdXFF
dXdF
−=−=
αα
S-Distribution: Rather General Random Number Generator
1. Generate uniformly distributed random numbers, ui ∈(0,1)2. Solve quantile equation to ui
3. Value is ri.4. Collection of ri are S-distributed.5. Great for Monte Carlo simulations
ui
ri0
1
Poisson Process per S-Distribution:
1. Known that inter-arrival time of Poisson process is exponentially distributed.
2. Thus, generate exponentially distributed random numbers from S-distribution and use them to determine next event.
3. Using dynamic rate for exponential distribution results in a non-homogeneous Poisson process.
Linear-Logistic Risk Modeland Cox’s Proportional Hazard Model
Can show that famous models in epidemiology (including
models for two-by-two tables, odds ratios, relative risks) are
direct consequences of disease models formulated as
S-systems.
S-Distribution: Liouville’s Problem
Can we solve a differential equation simultaneously under many different initial values?
Reformulate: How is a distribution of valuestransformed under the dynamics of a system ofdifferential equations? (Example: S-system)
Generic transformation of a distribution:
))(()()( 11 YfYdYdYf xy
−−= ϕϕ
S-Distribution: Liouville’s Problem
Question: What is the distribution of initial values after τ time units?
Suppose, X1 is of interest in the dynamic system
),...,(
),...,(
1
1
nii
nii
YYY
XXX
Ψ=
Ψ=&
&
),(),(
00,
00,
τ+=
=
tXYtXX
ii
ii
nini
,...,1,...,1
==
S-Distribution: Liouville’s Problem
),...,(),...,(
1
),...,(),...,(
11
111
1
1
11
111
nnii
nnii
YYYYdYdYdYdY
YYXXdYdX
−
−
ΨΨ=
=
ΨΨ= )( 00, tXX ii =
)( 00, τ+= tXY ii
Similar to the quantile equation, we can make Y1 theindependent variable by dividing the entire system by the equation of Y1
S-Distribution: Liouville’s Problem
Define distribution of X1 and express in terms of Y1:
),...,(),...,()()(
)(
11
11111
11
1
1
11
1
nnxxx
xx
YYXXFdYdX
FdYdF
FdXdF
−ΨΨ==
=
ϑϑ
ϑ
Example
25.6
16
2
25.0
11
107
2.0
XXX
XXX−×−=
=&
&
1)0(10)0(
2
61
== −
XX
0 2 4
0
0.45
0.9f1
0 20 40
0
3
6X1
Question: If population is distributed at t = 0 with f, what is its size distribution at time t = τ ? Any guesses?
)(10 8.0 FFfF −==&
Result
Like
lihoo
d
Summary
Canonical power-law models were devised for continuous,deterministic descriptions of dynamic phenomena.
Canonical models seem restrictive, but are in truth very flexible
Recasting shows that canonical models capture virtually alldifferentiable nonlinearities, including probability distributions
S-distributions approximate traditional distributions and allowfor interesting applications