Statistical Inference for Food WebsPart I: Bayesian Melding
Grace Chiu∗
andJosh Gould
∗ Department of Statistics & Actuarial Science
CMAR-Hobart Science Seminar, March 6, 2009 1
Outline
Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)
Present ENA techniquesENA statistical inference
statistical perspective of mass balanceBayesian melding
Example: Chesapeake Bay Mesohaline NetworkSummary and Conclusion
CMAR-Hobart Science Seminar, March 6, 2009 2
Outline
Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)
Present ENA techniquesENA statistical inference
statistical perspective of mass balanceBayesian melding
Example: Chesapeake Bay Mesohaline NetworkSummary and Conclusion
CMAR-Hobart Science Seminar, March 6, 2009 2
Outline
Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)
Present ENA techniques
ENA statistical inferencestatistical perspective of mass balanceBayesian melding
Example: Chesapeake Bay Mesohaline NetworkSummary and Conclusion
CMAR-Hobart Science Seminar, March 6, 2009 2
Outline
Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)
Present ENA techniquesENA statistical inference
statistical perspective of mass balanceBayesian melding
Example: Chesapeake Bay Mesohaline NetworkSummary and Conclusion
CMAR-Hobart Science Seminar, March 6, 2009 2
Outline
Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)
Present ENA techniquesENA statistical inference
statistical perspective of mass balanceBayesian melding
Example: Chesapeake Bay Mesohaline Network
Summary and Conclusion
CMAR-Hobart Science Seminar, March 6, 2009 2
Outline
Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)
Present ENA techniquesENA statistical inference
statistical perspective of mass balanceBayesian melding
Example: Chesapeake Bay Mesohaline NetworkSummary and Conclusion
CMAR-Hobart Science Seminar, March 6, 2009 2
Aspects of a Food Web
CMAR-Hobart Science Seminar, March 6, 2009 3
Aspects of a Food Web
CMAR-Hobart Science Seminar, March 6, 2009 3
Aspects of a Food Web
... that was only for trophicrelations ...
CMAR-Hobart Science Seminar, March 6, 2009 3
Aspects of a Food Web: (1) Trophic
Food web, trophically
structure of interdependence among speciespredator-prey links =⇒ feeding patternslinks can be weighted:e.g. predation frequency
Existing work for trophic analyses:examine interactions between providers (prey) andbenefactors (predators)(semi-)quantitative techniques for systematic extraction ofthe meaning of these interactionse.g. trophic hierarchy / compartments
CMAR-Hobart Science Seminar, March 6, 2009 4
Aspects of a Food Web: (1) Trophic
Food web, trophicallystructure of interdependence among speciespredator-prey links =⇒ feeding patternslinks can be weighted:e.g. predation frequency
Existing work for trophic analyses:examine interactions between providers (prey) andbenefactors (predators)(semi-)quantitative techniques for systematic extraction ofthe meaning of these interactionse.g. trophic hierarchy / compartments
CMAR-Hobart Science Seminar, March 6, 2009 4
Aspects of a Food Web: (1) Trophic
Food web, trophicallystructure of interdependence among speciespredator-prey links =⇒ feeding patternslinks can be weighted:e.g. predation frequency
Existing work for trophic analyses:examine interactions between providers (prey) andbenefactors (predators)(semi-)quantitative techniques for systematic extraction ofthe meaning of these interactions
e.g. trophic hierarchy / compartments
CMAR-Hobart Science Seminar, March 6, 2009 5
Aspects of a Food Web: (1) Trophic
Food web, trophicallystructure of interdependence among speciespredator-prey links =⇒ feeding patternslinks can be weighted:e.g. predation frequency
Existing work for trophic analyses:examine interactions between providers (prey) andbenefactors (predators)(semi-)quantitative techniques for systematic extraction ofthe meaning of these interactionse.g. trophic hierarchy / compartments
CMAR-Hobart Science Seminar, March 6, 2009 5
Aspects of a Food Web
CMAR-Hobart Science Seminar, March 6, 2009 6
Aspects of a Food Web
and now, for whole-systemrelations...
CMAR-Hobart Science Seminar, March 6, 2009 6
Aspects of a Food Web
CMAR-Hobart Science Seminar, March 6, 2009 6
Aspects of a Food Web: (2) Whole System
Ecological Network (whole-system food web)
trophic compartments and susbstance / energy throughputthis interdependence is subject to system balancenotion of balance based on thermodynamics
Existing Work: Ecological Network Analysis (ENA)deterministic biophysical theory in a balance modelquantity of exchange of substance / energy amongcompartmentsextract characteristics of these quantities
=⇒ describe interactions among compartments
CMAR-Hobart Science Seminar, March 6, 2009 7
Aspects of a Food Web: (2) Whole System
Ecological Network (whole-system food web)trophic compartments and susbstance / energy throughputthis interdependence is subject to system balancenotion of balance based on thermodynamics
Existing Work: Ecological Network Analysis (ENA)deterministic biophysical theory in a balance modelquantity of exchange of substance / energy amongcompartmentsextract characteristics of these quantities
=⇒ describe interactions among compartments
CMAR-Hobart Science Seminar, March 6, 2009 7
Aspects of a Food Web: (2) Whole System
Ecological Network (whole-system food web)trophic compartments and susbstance / energy throughputthis interdependence is subject to system balancenotion of balance based on thermodynamics
Existing Work: Ecological Network Analysis (ENA)deterministic biophysical theory in a balance modelquantity of exchange of substance / energy amongcompartmentsextract characteristics of these quantities
=⇒ describe interactions among compartments
CMAR-Hobart Science Seminar, March 6, 2009 7
Disclaimer
Images of food webs were generated by aGoogle search.
CMAR-Hobart Science Seminar, March 6, 2009 8
Trophic analysis and ENA
Main Goal:to understand / predict (e.g. over time) within-web interactionsbased on the quantities associated with the edges (arrows)between pairs of species / compartments
ISSUES:randomness of quantities ignoredno formal statistical inference of interaction patterns orpredictionsfor ENA,
randomness =⇒ observed quantities don’t balancesome quantities are unobservable from field —computer algorithms generate missing quantities tominimize imbalance (e.g. EcoSim / EcoPath)
=⇒ further complicating any inference attempts!
CMAR-Hobart Science Seminar, March 6, 2009 9
Trophic analysis and ENA
Main Goal:to understand / predict (e.g. over time) within-web interactionsbased on the quantities associated with the edges (arrows)between pairs of species / compartments
ISSUES:randomness of quantities ignoredno formal statistical inference of interaction patterns orpredictionsfor ENA,
randomness =⇒ observed quantities don’t balancesome quantities are unobservable from field —computer algorithms generate missing quantities tominimize imbalance (e.g. EcoSim / EcoPath)
=⇒ further complicating any inference attempts!
CMAR-Hobart Science Seminar, March 6, 2009 9
Trophic analysis and ENA
Main Goal:to understand / predict (e.g. over time) within-web interactionsbased on the quantities associated with the edges (arrows)between pairs of species / compartments
ISSUES:randomness of quantities ignoredno formal statistical inference of interaction patterns orpredictionsfor ENA,
randomness =⇒ observed quantities don’t balancesome quantities are unobservable from field —computer algorithms generate missing quantities tominimize imbalance (e.g. EcoSim / EcoPath)
=⇒ further complicating any inference attempts!
CMAR-Hobart Science Seminar, March 6, 2009 9
A statistician’s ideas...
Alternative Trophic Analysis:take a completely quantitative regression approach:
accounts for randomness of data!accounts for substance exchange! and other variables
proper inference possible from regression modellingincludes prediction inference —for pairwise links and compartments AND over time
simple scatterplots to identify and interpret compartments
that’ll be Statistical InferencePart II
CMAR-Hobart Science Seminar, March 6, 2009 10
A statistician’s ideas...
Alternative Trophic Analysis:take a completely quantitative regression approach:
accounts for randomness of data!accounts for substance exchange! and other variables
proper inference possible from regression modellingincludes prediction inference —for pairwise links and compartments AND over time
simple scatterplots to identify and interpret compartments
that’ll be Statistical InferencePart II
CMAR-Hobart Science Seminar, March 6, 2009 10
A statistician’s ideas...
Alternative Trophic Analysis:take a completely quantitative regression approach:
accounts for randomness of data!accounts for substance exchange! and other variables
proper inference possible from regression modellingincludes prediction inference —for pairwise links and compartments AND over time
simple scatterplots to identify and interpret compartments
that’ll be Statistical InferencePart II
CMAR-Hobart Science Seminar, March 6, 2009 10
Statistical Inference Part I
ENA Statistical Inference:can overcome empirical imbalance by incorporatingrandomness through Bayesian Meldingcan fill in missing quantities by prediction inference withinBayesian frameworkcan be extended to temporal model without explicitcalibration (as opposed to, e.g. morphing multiple staticanalyses into a single dynamics model)
CMAR-Hobart Science Seminar, March 6, 2009 11
Statistical Inference Part I
ENA Statistical Inference:can overcome empirical imbalance by incorporatingrandomness through Bayesian Meldingcan fill in missing quantities by prediction inference withinBayesian frameworkcan be extended to temporal model without explicitcalibration (as opposed to, e.g. morphing multiple staticanalyses into a single dynamics model)
CMAR-Hobart Science Seminar, March 6, 2009 11
Statistical Inference Part I
ENA Statistical Inference:can overcome empirical imbalance by incorporatingrandomness through Bayesian Meldingcan fill in missing quantities by prediction inference withinBayesian frameworkcan be extended to temporal model without explicitcalibration (as opposed to, e.g. morphing multiple staticanalysis into a single dynamics model)
CMAR-Hobart Science Seminar, March 6, 2009 12
Conventional ENA
Simple e.g. from Ulanowicz (Comput. Biol. & Chem., 2004):
Fix a transfer medium, e.g. nitrogen or heat. Let
i , j = compartment labelTij = rate of transfer of medium from i to jXi = rate of exogenous input of medium to iEi = rate of external transfer of medium from iRi = rate of dissipation of medium from i
Balance Equation:
total inflow rate = total outflow rate=⇒ Xi +
∑j
Tj i =∑
j
Ti j + Ei + Ri
Ideally, do this over all n compartments and K media.
CMAR-Hobart Science Seminar, March 6, 2009 13
Conventional ENA
Simple e.g. from Ulanowicz (Comput. Biol. & Chem., 2004):
Fix a transfer medium, e.g. nitrogen or heat. Let
i , j = compartment labelTij = rate of transfer of medium from i to jXi = rate of exogenous input of medium to iEi = rate of external transfer of medium from iRi = rate of dissipation of medium from i
Balance Equation:
total inflow rate = total outflow rate=⇒ Xi +
∑j
Tj i =∑
j
Ti j + Ei + Ri
Ideally, do this over all n compartments and K media.
CMAR-Hobart Science Seminar, March 6, 2009 13
Conventional ENA
Simple e.g. from Ulanowicz (Comput. Biol. & Chem., 2004):
Fix a transfer medium, e.g. nitrogen or heat. Let
i , j = compartment labelTij = rate of transfer of medium from i to jXi = rate of exogenous input of medium to iEi = rate of external transfer of medium from iRi = rate of dissipation of medium from i
Balance Equation:
total inflow rate = total outflow rate=⇒ Xi +
∑j
Tj i =∑
j
Ti j + Ei + Ri
Ideally, do this over all n compartments and K media.
CMAR-Hobart Science Seminar, March 6, 2009 13
Conventional ENA: Ideally
System of n × K equations:
X (1)1 + T (1)
+1 = T (1)1+ + E (1)
1 + R(1)1
...X (1)
n + T (1)+n = T (1)
n+ + E (1)n + R(1)
n
X (2)1 + T (2)
+1 = T (2)1+ + E (1)
1 + R(2)1
...X (2)
n + T (2)+n = T (2)
n+ + E (1)n + R(2)
n
...X (K )
1 + T (K )+1 = T (K )
1+ + E (K )1 + R(K )
1...
X (K )n + T (K )
+n = T (K )n+ + E (K )
n + R(K )n
↑
Randomness =⇒ field data never satisfy equality
Worse yet, only some quantities are observable from field ...
CMAR-Hobart Science Seminar, March 6, 2009 14
Conventional ENA
: Ideally
System of n × K equations:
X (1)1 + T (1)
+1 = T (1)1+ + E (1)
1 + R(1)1
...X (1)
n + T (1)+n = T (1)
n+ + E (1)n + R(1)
n
X (2)1 + T (2)
+1 = T (2)1+ + E (1)
1 + R(2)1
...X (2)
n + T (2)+n = T (2)
n+ + E (1)n + R(2)
n
...X (K )
1 + T (K )+1 = T (K )
1+ + E (K )1 + R(K )
1...
X (K )n + T (K )
+n = T (K )n+ + E (K )
n + R(K )n
↑
Randomness =⇒ field data never satisfy equality
Worse yet, only some quantities are observable from field ...
CMAR-Hobart Science Seminar, March 6, 2009 14
Conventional ENA
: Ideally
System of n × K equations:
X (1)1 + T (1)
+1 = T (1)1+ + E (1)
1 + R(1)1
...X (1)
n + T (1)+n = T (1)
n+ + E (1)n + R(1)
n
X (2)1 + T (2)
+1 = T (2)1+ + E (1)
1 + R(2)1
...X (2)
n + T (2)+n = T (2)
n+ + E (1)n + R(2)
n
...X (K )
1 + T (K )+1 = T (K )
1+ + E (K )1 + R(K )
1...
X (K )n + T (K )
+n = T (K )n+ + E (K )
n + R(K )n
↑
Randomness =⇒ field data never satisfy equality
Worse yet, only some quantities are observable from field ...
CMAR-Hobart Science Seminar, March 6, 2009 14
Conventional ENA
: Ideally
System of n × K equations:
X (1)1 + T (1)
+1 = T (1)1+ + E (1)
1 + R(1)1
...X (1)
n + T (1)+n = T (1)
n+ + E (1)n + R(1)
n
X (2)1 + T (2)
+1 = T (2)1+ + E (1)
1 + R(2)1
...X (2)
n + T (2)+n = T (2)
n+ + E (1)n + R(2)
n
...X (K )
1 + T (K )+1 = T (K )
1+ + E (K )1 + R(K )
1...
X (K )n + T (K )
+n = T (K )n+ + E (K )
n + R(K )n
↑
Randomness =⇒ field data never satisfy equality
Worse yet, only some quantities are observable from field ...
CMAR-Hobart Science Seminar, March 6, 2009 14
Conventional ENA
Example for n=4, K =1Observe: Xi , Ei , Ri for all i ; Tij for all (i , j) except i=3Unknown: T31, T32, T34 =⇒ T3+
X1 + T21 + T31 + T41 = T12 + T13 + T14 + E1 + R1
X2 + T12 + T32 + T42 = T21 + T23 + T24 + E2 + R2
X3 + T13 + T23 + T43 = T31 + T32 + T34 + E3 + R3
X4 + T14 + T24 + T34 = T41 + T42 + T43 + E4 + R4
no balance =⇒ no theoretical solution for T3+
CMAR-Hobart Science Seminar, March 6, 2009 15
Conventional ENA
Example for n=4, K =1Observe: Xi , Ei , Ri for all i ; Tij for all (i , j) except i=3Unknown: T31, T32, T34 =⇒ T3+
X1 + T21 + T31 + T41 = T12 + T13 + T14 + E1 + R1
X2 + T12 + T32 + T42 = T21 + T23 + T24 + E2 + R2
X3 + T13 + T23 + T43 = T31 + T32 + T34 + E3 + R3
X4 + T14 + T24 + T34 = T41 + T42 + T43 + E4 + R4
no balance =⇒ no theoretical solution for T3+
CMAR-Hobart Science Seminar, March 6, 2009 15
Conventional ENA
Remedy: deduce T3j ’s from observable auxiliary quantities
e.g. theoretical relationship amongTijcompartment productionbiomass...
f (Pij , Bij , . . .) = Tij
even if a deduced Tij is free of uncertainty ...System of equations fails regardless ... what then?
CMAR-Hobart Science Seminar, March 6, 2009 16
Conventional ENA
Remedy: deduce T3j ’s from observable auxiliary quantities
e.g. theoretical relationship amongTijcompartment productionbiomass...
f (Pij , Bij , . . .) = Tij
even if a deduced Tij is free of uncertainty ...System of equations fails regardless ... what then?
CMAR-Hobart Science Seminar, March 6, 2009 16
Conventional ENA
Remedy: deduce T3j ’s from observable auxiliary quantities
e.g. theoretical relationship amongTijcompartment productionbiomass...
f (Pij , Bij , . . .) = Tij
even if a deduced Tij is free of uncertainty ...
System of equations fails regardless ... what then?
CMAR-Hobart Science Seminar, March 6, 2009 16
Conventional ENA
Remedy: deduce T3j ’s from observable auxiliary quantities
e.g. theoretical relationship amongTijcompartment productionbiomass...
f (Pij , Bij , . . .) = Tij
even if a deduced Tij is free of uncertainty ...System of equations fails regardless ... what then?
CMAR-Hobart Science Seminar, March 6, 2009 16
Conventional ENA
Computer software to the rescue!
tinker with quantities subject to certain criteriauntil equality (almost) holdscriteria built into software — can be mysterious to userrestrict tinkering to deduced quantities only, if possible
Lingo
(Program) Input: observed and deduced quantities(Program) Output: balanced quantities
CMAR-Hobart Science Seminar, March 6, 2009 17
Conventional ENA
Computer software to the rescue!
tinker with quantities subject to certain criteriauntil equality (almost) holdscriteria built into software — can be mysterious to userrestrict tinkering to deduced quantities only, if possible
Lingo
(Program) Input: observed and deduced quantities(Program) Output: balanced quantities
CMAR-Hobart Science Seminar, March 6, 2009 17
Conventional ENA
Computer software to the rescue!
tinker with quantities subject to certain criteriauntil equality (almost) holdscriteria built into software — can be mysterious to userrestrict tinkering to deduced quantities only, if possible
Lingo
(Program) Input: observed and deduced quantities(Program) Output: balanced quantities
CMAR-Hobart Science Seminar, March 6, 2009 17
Conventional ENA
In light ofim
(possible)
balancetheory-based deductioncoerced balance ...
Million $ question:How confident are we in the numbers??
CMAR-Hobart Science Seminar, March 6, 2009 18
Conventional ENA
In light ofim(possible)balancetheory-based deductioncoerced balance ...
Million $ question:How confident are we in the numbers??
CMAR-Hobart Science Seminar, March 6, 2009 18
Conventional ENA
In light ofim(possible)balancetheory-based deductioncoerced balance ...
Million $ question:How confident are we in the numbers??
CMAR-Hobart Science Seminar, March 6, 2009 18
Conventional ENA: confidence
Existing attempts: Sensitivity analysesperturb program inputmonitor behavior of program output
but ...
How to make inference for underlying network structure?
CMAR-Hobart Science Seminar, March 6, 2009 19
Conventional ENA: confidence
Existing attempts: Sensitivity analysesperturb program inputmonitor behavior of program output
but ...
How to make inference for underlying network structure?
CMAR-Hobart Science Seminar, March 6, 2009 19
Perspectives of Mass Balance
Physics: in = outStatistics: E( in ) = E( out )
Simplest example
Let Wi = Xi + T+i w/ mean µW
Ui = Ti+ + Ei w/ mean µU
Ri w/ mean µR
=⇒ balance model: µW = µU + µR
single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference
CMAR-Hobart Science Seminar, March 6, 2009 20
Perspectives of Mass Balance
Physics: in = out
Statistics: E( in ) = E( out )
Simplest example
Let Wi = Xi + T+i w/ mean µW
Ui = Ti+ + Ei w/ mean µU
Ri w/ mean µR
=⇒ balance model: µW = µU + µR
single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference
CMAR-Hobart Science Seminar, March 6, 2009 20
Perspectives of Mass Balance
Physics: in = outStatistics: E( in ) = E( out )
Simplest example
Let Wi = Xi + T+i w/ mean µW
Ui = Ti+ + Ei w/ mean µU
Ri w/ mean µR
=⇒ balance model: µW = µU + µR
single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference
CMAR-Hobart Science Seminar, March 6, 2009 20
Perspectives of Mass Balance
Physics: in = outStatistics: E( in ) = E( out )
Simplest example
Let Wi = Xi + T+i w/ mean µW
Ui = Ti+ + Ei w/ mean µU
Ri w/ mean µR
=⇒ balance model: µW = µU + µR
single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference
CMAR-Hobart Science Seminar, March 6, 2009 20
Perspectives of Mass Balance
Physics: in = outStatistics: E( in ) = E( out )
Simplest example
Let Wi = Xi + T+i w/ mean µW
Ui = Ti+ + Ei w/ mean µU
Ri w/ mean µR
=⇒ balance model: µW = µU + µR
single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference
CMAR-Hobart Science Seminar, March 6, 2009 20
Perspectives of Mass Balance
Physics: in = outStatistics: E( in ) = E( out )
Simplest example
Let Wi = Xi + T+i w/ mean µW
Ui = Ti+ + Ei w/ mean µU
Ri w/ mean µR
=⇒ balance model: µW = µU + µR
single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference
CMAR-Hobart Science Seminar, March 6, 2009 20
Bayesian Melding for ENA Inference
Elements of deterministic modelingdeterministic model M( )
model input θ
= E(observables)
model output φ
= E(unobservables)
M : θ 7−→ φ or φ := M(θ)
Rationale for choice of input/output:to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)
CMAR-Hobart Science Seminar, March 6, 2009 21
Bayesian Melding for ENA Inference
Elements of deterministic modelingdeterministic model M( )
model input θ
= E(observables)
model output φ
= E(unobservables)
M : θ 7−→ φ or φ := M(θ)
Rationale for choice of input/output:to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)
CMAR-Hobart Science Seminar, March 6, 2009 21
Bayesian Melding for ENA Inference
Elements of deterministic modelingdeterministic model M( )
model input θ
= E(observables)
model output φ
= E(unobservables)
M : θ 7−→ φ or φ := M(θ)
Rationale for choice of input/output:to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)
CMAR-Hobart Science Seminar, March 6, 2009 21
Bayesian Melding for ENA Inference
Elements of deterministic modelingdeterministic model M( )
model input θ = E(observables)model output φ = E(unobservables)
M : θ 7−→ φ or φ := M(θ)
Rationale for choice of input/output:
to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)
CMAR-Hobart Science Seminar, March 6, 2009 21
Bayesian Melding for ENA Inference
Elements of deterministic modelingdeterministic model M( )
model input θ = E(observables)model output φ = E(unobservables)
M : θ 7−→ φ or φ := M(θ)
Rationale for choice of input/output:to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)
CMAR-Hobart Science Seminar, March 6, 2009 21
Bayesian Melding for ENA Inference
Among Wi , Ui , Ri , suppose Ri is unobservable
=⇒ rewrite balance model as
µR = µW − µU
or φ = M(θ)
where φ = µR model output
θ = (µW , µU)′ model input
M(θ) = (1,−1)θ the model M : θ → φ
CMAR-Hobart Science Seminar, March 6, 2009 22
Bayesian Melding for ENA Inference
Among Wi , Ui , Ri , suppose Ri is unobservable=⇒ rewrite balance model as
µR = µW − µU
or φ = M(θ)
where φ = µR model output
θ = (µW , µU)′ model input
M(θ) = (1,−1)θ the model M : θ → φ
CMAR-Hobart Science Seminar, March 6, 2009 22
Bayesian Melding for ENA Inference
Among Wi , Ui , Ri , suppose Ri is unobservable=⇒ rewrite balance model as
µR = µW − µU
or φ = M(θ)
where φ = µR model output
θ = (µW , µU)′ model input
M(θ) = (1,−1)θ the model M : θ → φ
CMAR-Hobart Science Seminar, March 6, 2009 22
Bayesian Melding for ENA Inference
Bayesian Inference
specify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inference
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 23
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inference
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 23
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inference
Bayesian Melding (due to Poole & Raftery, 2000)
two priors for φ:specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 23
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inference
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 23
Bayesian Melding for ENA Inference
g(θ) h∗(φ)input prior induced output prior
θ(1) → M(θ(1)) = φ(1)
...θ(m) → M(θ(m)) = φ(m)
CMAR-Hobart Science Seminar, March 6, 2009 24
Bayesian Melding for ENA
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 25
Bayesian Melding for ENA
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 25
Bayesian Melding for ENA Inference
Melded prior for φ is
q̃(φ) ∝ h∗(φ)γ h(φ)1−γ
for some γ ∈ (0,1).
What’s γ?arbitrary in principlecan be specified to reflect expert opinions on relativereliability between h and h∗ (or M)
CMAR-Hobart Science Seminar, March 6, 2009 26
Bayesian Melding for ENA Inference
Melded prior for φ is
q̃(φ) ∝ h∗(φ)γ h(φ)1−γ
for some γ ∈ (0,1).
What’s γ?arbitrary in principlecan be specified to reflect expert opinions on relativereliability between h and h∗ (or M)
CMAR-Hobart Science Seminar, March 6, 2009 26
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, f (φ)induced prior, f ∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 27
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, f (φ)induced prior, f ∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of M
CMAR-Hobart Science Seminar, March 6, 2009 27
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, f (φ)induced prior, f ∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of MCMAR-Hobart Science Seminar, March 6, 2009 27
Bayesian Melding for ENA Inference
Bayesian Inferencespecify prior and likelihood for φ,θ
obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive
Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:
specified prior, f (φ)induced prior, f ∗(φ), from cranking prior for θ through M
combine both priors =⇒ melded prior for φ, q̃(φ)
crank melded prior q̃(φ) through M−1
=⇒ melded prior for θ, p̃(θ)
posterior inference based on p̃ and M
there are tricks to overcome non-invertibility of MCMAR-Hobart Science Seminar, March 6, 2009 27
Bayesian Melding for ENA Inference
posterior for θ: πθ(θ) ∝ p̃(θ) Lθ(θ) Lφ
(M(θ)
)
posterior for φ: M : πθ(θ) → πφ(φ)
Hallelujah!
CMAR-Hobart Science Seminar, March 6, 2009 28
Bayesian Melding for ENA Inference
posterior for θ: πθ(θ) ∝ p̃(θ) Lθ(θ) Lφ
(M(θ)
)posterior for φ: M : πθ(θ) → πφ(φ)
Hallelujah!
CMAR-Hobart Science Seminar, March 6, 2009 28
Bayesian Melding for ENA Inference
posterior for θ: πθ(θ) ∝ p̃(θ) Lθ(θ) Lφ
(M(θ)
)posterior for φ: M : πθ(θ) → πφ(φ)
Hallelujah!
CMAR-Hobart Science Seminar, March 6, 2009 28
Application: Chesapeake Bay Summer Network
based on
Baird & Ulanowicz (1989), Ecological Monographs 59, 329–364
and
http://www.cbl.umces.edu/˜ulan/ntwk/datall.zip
CMAR-Hobart Science Seminar, March 6, 2009 29
Application: Chesapeake Bay Summer Network
CMAR-Hobart Science Seminar, March 6, 2009 29
Application: Chesapeake Bay Summer Network
CMAR-Hobart Science Seminar, March 6, 2009 29
Application: Chesapeake Bay Summer Network
transfer medium: carbon (g/m2/summer)i=1,. . . , 36 compartments
Elements of Bayesian inferenceLikelihood:
Wi , Ui , Ri |θ, φ ∼ independent exponentials
specified prior:
(θ, φ)′ ∼ trivariate log-normal
=⇒ Wi , Ui , Ri are marginally dependent(as would be necessary due to theoretical balance)
CMAR-Hobart Science Seminar, March 6, 2009 30
Application: Chesapeake Bay Summer Network
transfer medium: carbon (g/m2/summer)i=1,. . . , 36 compartments
Elements of Bayesian inferenceLikelihood:
Wi , Ui , Ri |θ, φ ∼ independent exponentials
specified prior:
(θ, φ)′ ∼ trivariate log-normal
=⇒ Wi , Ui , Ri are marginally dependent(as would be necessary due to theoretical balance)
CMAR-Hobart Science Seminar, March 6, 2009 30
Application: Chesapeake Bay Summer Network
transfer medium: carbon (g/m2/summer)i=1,. . . , 36 compartments
Elements of Bayesian inferenceLikelihood:
Wi , Ui , Ri |θ, φ ∼ independent exponentials
specified prior:
(θ, φ)′ ∼ trivariate log-normal
=⇒ Wi , Ui , Ri are marginally dependent(as would be necessary due to theoretical balance)
CMAR-Hobart Science Seminar, March 6, 2009 30
Application: Chesapeake Bay Summer Network
NOTE: Instead of painstaking extraction of (unbalanced) datafrom the flow diagram, we adopted the online data (alreadybalanced) — would ideally use former.
CMAR-Hobart Science Seminar, March 6, 2009 31
Application: Chesapeake Bay Summer Network
W
0 100 300 500
0.00
00.
004
0.00
80.
012
U
0 100 300 500
0.00
00.
005
0.01
00.
015
R
0 50 100 150 200 250
0.00
0.02
0.04
0.06
θθ1
0 20 40 60 80 100
0.00
00.
015
0.03
00.
045
θθ2
0 20 40 60 80 100
0.00
0.02
0.04
φφ0 10 20 30 40 50 60
0.00
0.02
0.04
0.06
0.08
0.10
Data:
Specified prior ( —— ) and Melded Posterior ( —— ):
CMAR-Hobart Science Seminar, March 6, 2009 31
Application: Chesapeake Bay Summer Network
Estimates and 95% Confidence intervals:
θ1 = µW θ2 = µU φ = µRMelding
Posterior Mean 89.03 64.11 24.92HPD interval (72.69, 107.26) (47.99, 80.84) (17.57, 33.23)
ClassicalCLT interval (33.24, 146.06) (20.70, 110.56) (4.24, 43.80)
Classical intervals are MUCH WIDER!
CMAR-Hobart Science Seminar, March 6, 2009 32
Application: Chesapeake Bay Summer Network
Estimates and 95% Confidence intervals:
θ1 = µW θ2 = µU φ = µRMelding
Posterior Mean 89.03 64.11 24.92HPD interval (72.69, 107.26) (47.99, 80.84) (17.57, 33.23)
ClassicalCLT interval (33.24, 146.06) (20.70, 110.56) (4.24, 43.80)
Classical intervals are MUCH WIDER!
CMAR-Hobart Science Seminar, March 6, 2009 32
Application: Chesapeake Bay Summer Network
Dependence among θ1, θ2, φ:
60 70 80 90 100 110 120
4050
6070
8090
100
θθ1
θθ 2
60 70 80 90 100 110 120
010
2030
4050
60
θθ1
φφ
40 50 60 70 80 90 100
010
2030
4050
60
θθ2
φφ
High dependence is presumed among all 3 due totheoretical balance.Inference indicates dependence is only high betweenµW and µU =⇒ new insight!Note: inference on dependence structure NOT possiblewith classical inference (which treats µ’s as constants).
CMAR-Hobart Science Seminar, March 6, 2009 33
Application: Chesapeake Bay Summer Network
Dependence among θ1, θ2, φ:
60 70 80 90 100 110 120
4050
6070
8090
100
θθ1
θθ 2
60 70 80 90 100 110 120
010
2030
4050
60
θθ1
φφ
40 50 60 70 80 90 100
010
2030
4050
60
θθ2
φφ
High dependence is presumed among all 3 due totheoretical balance.Inference indicates dependence is only high betweenµW and µU =⇒ new insight!Note: inference on dependence structure NOT possiblewith classical inference (which treats µ’s as constants).
CMAR-Hobart Science Seminar, March 6, 2009 33
Summary
Physics Statistics(1) each quantity (variable) • no random variable needs
must satisfy within- to satisfy balancecompartment balance =⇒ no compartment needs=⇒ collapse compart- to satisfy balance§
ments to allow balance • random variables have=⇒ wrong biology expectations that satisfy
within-system balance=⇒ the more compartments(i.e. data) the better!
§ can be a need as long as replicated observations areavailable per compartment
CMAR-Hobart Science Seminar, March 6, 2009 34
Summary
Physics Statistics(1) each quantity (variable) • no random variable needs
must satisfy within- to satisfy balancecompartment balance =⇒ no compartment needs=⇒ collapse compart- to satisfy balance§
ments to allow balance • random variables have=⇒ wrong biology expectations that satisfy
within-system balance=⇒ the more compartments(i.e. data) the better!
§ can be a need as long as replicated observations areavailable per compartment
CMAR-Hobart Science Seminar, March 6, 2009 34
Summary
Physics Statistics(2) unobservable variables statistical prediction
are deduced from (inferential) possibleauxiliary theory in certain scenarios
e.g. variable observed forsome compartments
e.g. deduce unobservedthrough formal regression(both in progress)
(3) no formal inference / possible forconfidence statements (a) any quantity infor any quantity system-balance equation
(b) unobservable variablein some cases (see (2))
(4) multiple media on perceivably straight-same system hard forwardto impossible (in progress)
CMAR-Hobart Science Seminar, March 6, 2009 35
Conclusion
So why statistical inference for food webs?statistical models are more honest:
properly acknowledge uncertaintystatistical inference-based techniques are tractableproper prediction inference is possible
(straightforward within Bayesian framework)ENA with Bayesian melding additionally
overcomes empirical imbalance under theoretical balancesoundly integrates statistical practice with physical theory
— often preferred by scientists over purely empirically drivenapproaches
CMAR-Hobart Science Seminar, March 6, 2009 36
Conclusion
So why statistical inference for food webs?
statistical models are more honest:properly acknowledge uncertainty
statistical inference-based techniques are tractableproper prediction inference is possible
(straightforward within Bayesian framework)ENA with Bayesian melding additionally
overcomes empirical imbalance under theoretical balancesoundly integrates statistical practice with physical theory
— often preferred by scientists over purely empirically drivenapproaches
CMAR-Hobart Science Seminar, March 6, 2009 36
Conclusion
So why statistical inference for food webs?statistical models are more honest:
properly acknowledge uncertaintystatistical inference-based techniques are tractableproper prediction inference is possible
(straightforward within Bayesian framework)
ENA with Bayesian melding additionallyovercomes empirical imbalance under theoretical balancesoundly integrates statistical practice with physical theory
— often preferred by scientists over purely empirically drivenapproaches
CMAR-Hobart Science Seminar, March 6, 2009 36
Conclusion
So why statistical inference for food webs?statistical models are more honest:
properly acknowledge uncertaintystatistical inference-based techniques are tractableproper prediction inference is possible
(straightforward within Bayesian framework)ENA with Bayesian melding additionally
overcomes empirical imbalance under theoretical balancesoundly integrates statistical practice with physical theory
— often preferred by scientists over purely empirically drivenapproaches
CMAR-Hobart Science Seminar, March 6, 2009 36
Thank you!
This presentation is available from:http://www.stats.uwaterloo.ca/˜gchiu/Talks/csiro-hobart09.pdf
Articles:Chiu & Gould (submitted).Chiu & Gould (2008),U of Waterloo Working Paper #2008-07.
www.stats.uwaterloo.ca/stats navigation/techreports/08WorkingPapers/08-07.pdf
CMAR-Hobart Science Seminar, March 6, 2009 37