Post on 28-Jan-2018
transcript
A TOUR IN OPTIMAL TRANSPORT Michiel Stock
@michielstockKERMIT
DESSERTS MADE BY TINNE
PORTIONS PER KERMIT MEMBER
PREFERENCES FOR DESSERTS
FORMAL PROBLEM DESCRIPTION
r: vector containing portions of dessert per person (general: n-dim.)
c: vector containing portions of each dessert (general: m-dim.)
M: a cost matrix (negative preference)
U(r, c) = {P 2 Rn⇥m>0 | P1m = r, P |1n = c}
Polyhedral set containing all valid partitions:
Solve the following problem:
dM (r, c) = minP2U(r,c)
X
i,j
PijMij
Minimizer is the optimal distribution!P ?ij
d�M (r, c) = minP2U(r,c)
hP,MiF � 1
�h(P )
OPTIMAL TRANSPORT WITH ENTROPIC REGULARIZATION
Cost:
Entropic regularization:
Tuning parameter:
hP,MiF =X
i,j
PijMij
h(P ) = �X
i,j
Pij logPij
�
Constrain solution to possess a minimal ‘evenness’
GEOMETRY OF THE OPTIMAL TRANSPORT PROBLEM
M
P ?
P ?�
rc|
U(r, c)
DERIVATION OF THE SOLUTION
@L(P,�, {a1, . . . , an})@Pij
|P⇤ij= 0
Lagrangian of the problem:
Choose constants to satisfy constraints!P ?ij = e�ai�bj�1e��Mij
= ↵i�je��Mij
L(P,�, {a1, . . . , an}, {b1, . . . , bm}) =X
ij
PijMij +1
�
X
ij
Pij logPij
+
nX
i=1
ai(ri �X
j
Pij) +
mX
j=1
bj(cj �X
i
Pij)
@L(P,�, {a1, . . . , an})@Pij
= Mij +logPij
�+
1
�� ai � bj
THE SINKHORN-KNOPP ALGORITHM
Init
Until convergedScale rowsScale columns
P = e��M
SOLUTION (HIGH LAMBDA)
Solution is very good approximation of unregularized OT problem!
total average
preference:
36
SOLUTION (LOW(ER) LAMBDA)
Every person has to try a bit of everything!
total average
preference:
29.6
APPLICATIONS
➤ Matching distributions
➤ Interpolation
➤ Domain adaptation
➤ Color transfer
➤ Comparing distributions
➤ Modelling complex systems
MATCHING AND INTERPOLATING DISTRIBUTIONS
DOMAIN ADAPTATION WHEN DISTR, TRAIN AND TEST DIFFER
IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS
IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS
COMPARING DISTRIBUTIONS (WITH METRIC/COST)
➤ Comparing two distributions with cost
➤ Comparing two sets of objects with pairwise similarity
No equal number of
bins required!
d�M (r, c) = minP2U(r,c)
hP,MiF � 1
�h(P )
OPTIMAL TRANSPORT AS ENERGY MINIMISATION
OT can be seen as a physical system of interacting parts
Energy of the system
Physical constrains (i.e. mass balance)
Inverse temperature
Entropy of system
Interacting systems with competition.
COMPUTATIONAL FLUID DYNAMICS
Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv
LEARNING EPIGENETIC LANDSCAPES
Reconstruction of developmental landscapes by optimal-transport analysis of single-cell gene expression sheds light on cellular reprogramming. doi: https://doi.org/10.1101/191056
IN SUMMARY
➤ OT is a simple framework for thinking about distributions
➤ Powerful tool for modelling complex systems (constraints + competition)
➤ Efficient solvers: O(n^2) (when using entropic regularization)
REFERENCES
Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv
Courty, N., Flamary, R., Tuia, D. and Rakotomamonjy, A. (2016). Optimal transport for domain adaptation
Cuturi, M. (2013) Sinkhorn distances: lightspeed computation of optimal transportation distances