A TOUR IN OPTIMAL TRANSPORT Michiel Stock

@michielstockKERMIT

DESSERTS MADE BY TINNE

PORTIONS PER KERMIT MEMBER

PREFERENCES FOR DESSERTS

FORMAL PROBLEM DESCRIPTION

r: vector containing portions of dessert per person (general: n-dim.)

c: vector containing portions of each dessert (general: m-dim.)

M: a cost matrix (negative preference)

U(r, c) = {P 2 Rn⇥m>0 | P1m = r, P |1n = c}

Polyhedral set containing all valid partitions:

Solve the following problem:

dM (r, c) = minP2U(r,c)

X

i,j

PijMij

Minimizer is the optimal distribution!P ?ij

d�M (r, c) = minP2U(r,c)

hP,MiF � 1

�h(P )

OPTIMAL TRANSPORT WITH ENTROPIC REGULARIZATION

Cost:

Entropic regularization:

Tuning parameter:

hP,MiF =X

i,j

PijMij

h(P ) = �X

i,j

Pij logPij

�

Constrain solution to possess a minimal ‘evenness’

GEOMETRY OF THE OPTIMAL TRANSPORT PROBLEM

M

P ?

P ?�

rc|

U(r, c)

DERIVATION OF THE SOLUTION

@L(P,�, {a1, . . . , an})@Pij

|P⇤ij= 0

Lagrangian of the problem:

Choose constants to satisfy constraints!P ?ij = e�ai�bj�1e��Mij

= ↵i�je��Mij

L(P,�, {a1, . . . , an}, {b1, . . . , bm}) =X

ij

PijMij +1

�

X

ij

Pij logPij

+

nX

i=1

ai(ri �X

j

Pij) +

mX

j=1

bj(cj �X

i

Pij)

@L(P,�, {a1, . . . , an})@Pij

= Mij +logPij

�+

1

�� ai � bj

THE SINKHORN-KNOPP ALGORITHM

Init

Until convergedScale rowsScale columns

P = e��M

SOLUTION (HIGH LAMBDA)

Solution is very good approximation of unregularized OT problem!

total average

preference:

36

SOLUTION (LOW(ER) LAMBDA)

Every person has to try a bit of everything!

total average

preference:

29.6

APPLICATIONS

➤ Matching distributions

➤ Interpolation

➤ Domain adaptation

➤ Color transfer

➤ Comparing distributions

➤ Modelling complex systems

MATCHING AND INTERPOLATING DISTRIBUTIONS

DOMAIN ADAPTATION WHEN DISTR, TRAIN AND TEST DIFFER

IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS

IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS

COMPARING DISTRIBUTIONS (WITH METRIC/COST)

➤ Comparing two distributions with cost

➤ Comparing two sets of objects with pairwise similarity

No equal number of

bins required!

d�M (r, c) = minP2U(r,c)

hP,MiF � 1

�h(P )

OPTIMAL TRANSPORT AS ENERGY MINIMISATION

OT can be seen as a physical system of interacting parts

Energy of the system

Physical constrains (i.e. mass balance)

Inverse temperature

Entropy of system

Interacting systems with competition.

COMPUTATIONAL FLUID DYNAMICS

Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv

LEARNING EPIGENETIC LANDSCAPES

Reconstruction of developmental landscapes by optimal-transport analysis of single-cell gene expression sheds light on cellular reprogramming. doi: https://doi.org/10.1101/191056

IN SUMMARY

➤ OT is a simple framework for thinking about distributions

➤ Powerful tool for modelling complex systems (constraints + competition)

➤ Efficient solvers: O(n^2) (when using entropic regularization)

REFERENCES

Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv

Courty, N., Flamary, R., Tuia, D. and Rakotomamonjy, A. (2016). Optimal transport for domain adaptation

Cuturi, M. (2013) Sinkhorn distances: lightspeed computation of optimal transportation distances