Duality in a maximum

Shinto Eguchi

MaxEnt, 201421-26 September 2014Chateau Clos Luce, Amboise

generalized entropy model

Osamu KomoriAtsumi Ohara

MaxEnt in ecology

Presence data obtained by an ecological study

Log-entropy (Boltzmann-Gibbs-Shannon)

Equal mean space

2

Maximum entropy distribution

Problem : find a distribution f to maximize H( f under the constraints:

Solution is Gibbs distribution

3

MAP estimator

Loss function

Cf. adaptive Lasso, Zou (2006, JASA)

4

Sequential L1-regularization (Phillip, Anderson, Schapire 2006)

GIS (Geographic Information System)

GBIF (Global Biodiversity Inventory Facility) in R

5http://www.seascapemodeling.org/

Habitat map of right whales

Area Under ROC Curve

6

Problems in MaxEnt

7

The duality between the model and estimation is not robust(model misspecification, over-fitting to data, cf. Bayes robust)

MaxEnt jointly suggests the statistical model and estimation,(exponential model and maximum likelihood)

MaxEnt is useful in presence-only-data in ecological studies

U-entropy leads to extension from MaxEnt to MaxU-Ent(The duality is extended in functional degree of freedom)

Model selection and model validation have simple forms

Hill’s diversity numbers(effective number of species) (1973)

8

richness Rare species are highly weighted

exp (log-entropy) All species are fairly weighted

Simpson index Dominant species are highly weighted

Several measures of entropy

Tsallis entropy (1988)

Hill’s diversity (1973)

Simpson (1949)

9

U-entropy (Eguchi, 2006)

Generalized entropy

10

Generator

U-entropy

Example

Convex conjugate

log entropy

Renyi-Tsallis entropy

Generalized divergence

11

U-cross entropy

U-divergence

MaxU-Ent

Problem : find a distribution f to maximize HU ( f under the constraints:

12

13

Minimum divergence geometry

Riemannian metric

Linear connections

14

Dually flat

Riemann metric

Linear connections

Conjugate convexity:

U-model

15

Pythagorean theorem for U-divergence

Theorem

m-geodesicU-geodesic

Max-Entropy model

MaxEnt and MinDiv

Duality of totally geodesic

17

MaxU-Ent= MaxU-model + MinU-divergence

18

Project

19

Which -MaxEnt is the best ?

Can we select the best based on presence data?

Test AUC comparison may suggest better Max -Ent.

Hill’s diversity numbers associate with

one parameter family of MaxEnt methods, { Max -Ent : in R }

Can the family improve the classical Maxent?

Folivora( Three-toed sloth )

20

Max-Ent

21MaxEnt)

22MaxEnt)

Max-Ent

U-MaxEnt

U-entropy

U-Gibbs distribution

U-estimator

U-divergence

U-MaxEnt

U-entropy

U-Gibbs distribution

U-estimator

U-divergence

A class of MaxU-Ent

23

MaxEnt

Log-entropy

Gibbs distribution

KL-divergence

MLE

U-MaxEnt

U-entropy

U-Gibbs distribution

U-estimator

U-divergence

MaxU-Ent

U-entropy

U-Gibbs distribution

U-divergence

MinU-estimator

Thank you!

24

Model validation

25

A brief tutorial on Maxent(Phillips, 2006)

model

loss function

U-loss

exp-model U-model

( U-estimator , U-model)

- log-likelihoodMLE

Robust estimator

Robust model

26

U-estimate

MaxU-Ent

Problem : find a distribution f to maximize HU ( f under the constraints:

27

Related problems

28

Maxent is equivalent to Poisson point process model

( Renner-Warton, 2013, Loyle et all. 2012,

Aarts et al. 2012, Fithian-Hastie, 2012)

-Maxent is equivalent to -Poisson point process model?

-Poisson distribution

5 10 15 20

0.05

0.10

0.15

0.20

0.25

0.30

0.35

5 10 15 20

0.05

0.10

0.15

0.20

0.25

0.30

0.35

5 10 15 20

0.05

0.10

0.15

0.20

0.25

0.30

0.35

5 10 15 20

0.05

0.10

0.15

0.20

0.25

0.30

0.35

29

Habitat map

30

= 0 = 0.5