Post on 23-Jun-2020
transcript
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
No man is an island
No man is an Iland, intire of itselfe;every man is a peece of the Continent, apart of the maine . . .
MEDITATION XVII Devotions uponEmergent Occasions John Donne
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Who?
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Robust Mean Field Games
Fabio & Dario H. Tembine2 Q. Zhu3 T. Basar3
2Supelec
3University of Illinois Urbana-Champaign
Padua, 22 March 2013
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Outline
Mean field gamesIntroduction
Examples, (O. Gueant et al. 2011)Mexican waveLarge populationOil production
Robust mean field gamesOil production with uncertainty...generalizing
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Advection
I N →∞ homogeneous agents withdynamics
x(t) = u(x(t), t), x0 ∈ Rn
I u(x, t) is vector field in Rn
I density m(x, t) in x evolvesaccording to advection equation
∂tm+div(m·u(x, t)) = 0, in Rn × [0, T ]
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Mean field games
I Agents wish to minimize∫ T0
[ 1
2|u(x(t), t)|2︸ ︷︷ ︸
penalty on control
+ g(x(t),m(·, t))︸ ︷︷ ︸...on state & distribution
]dt+G(x(T ),m(·, T ))︸ ︷︷ ︸
...on final state
I opt. control u(x(t), t) = −∇xJ(x(t), t), [J(., .) is opt. cost]
I coupled partial differential equations in Rn × [0, T ]:
−∂tJ + 12 |∇xJ |2 = g(x,m)
u
��
(HJB) - backward
∂tm+ div(m · u(x)) = 0
m
TT
(advection) - forward
I boundary conditionsm(·, 0) = m0, J(x, T ) = G(x,m(·, T ))
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Hamilton Jacobi Bellman
I From Bellman
J(x0, t0)︸ ︷︷ ︸today’s cost
= minu [1
2|u0|2 + g(x0,m0)]dt︸ ︷︷ ︸
stage cost
+ J(x0 + dx, t0 + dt)︸ ︷︷ ︸future cost
I Taylor expanding future cost
J(x0 + dx, t0 + dt) = J(x0, t0) + ∂tJdt+∇xJxdt
I minu [1
2|u|2 + g(x,m) + ∂tJ +∇xJ
u︷︸︸︷x ]︸ ︷︷ ︸
Hamiltonian
= 0 (drop index 0)
I optimal control u = −∇xJ yields
−∂tJ +1
2|∇xJ |2 = g(x,m) HJB
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Stochastic differential game
I stochastic dynamics is
dx = udt+ σdBt
I dBt infinitesimal Brownian motion
I Mean field games (∆ =∑n
i=1∂2
∂x2i
Laplacian)
−∂tJ + 12 |∇xJ |2 − σ2
2 ∆J = g(x,m)
u��
(HJB)-backward
∂tm+ div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)-forward
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Hamilton Jacobi Bellman (with dx = udt+ σdBt)I From Bellman
J(x0, t0)︸ ︷︷ ︸today’s cost
= minu [1
2|u|2 + g(x,m)]dt︸ ︷︷ ︸
stage cost
+ EJ(x0 + dx, t0 + dt)︸ ︷︷ ︸exp. future cost
I Taylor expanding future cost (EdBt = 0, EdB2t → dt)
J(x0+dx, t0+dt) = J(x0, t0)+∂tJdt+E∇xJdx︸ ︷︷ ︸∇xJudt
+ E1
2dx′∇2
xJdx︸ ︷︷ ︸σ2
2∆JEdB2
t
I minu [1
2|u|2 + g(x,m) + ∂tJ +∇xJu+
σ2
2∆J ]︸ ︷︷ ︸
Hamiltonian
= 0
I optimal control u(t) = −∇xJ yields
−∂tJ +1
2|∇xJ |2 −
σ2
2∆J = g(x,m) HJB
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Average cost (ergodic)
I J = E lim supT→∞1T
∫ T0
[12 |u(t)|2 + g(x(t),m(·, t))
]dt
I Mean field games (∆ =∑n
i=1∂2
∂x2i
Laplacian), solve in Rn
λ+ 12 |∇xJ |2 − σ2
2 ∆J = g(x, m)
u��
(HJB)
div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field gamesIntroduction
Discounted cost
I J = E∫∞
0 e−ρt[
12 |u(t)|2 + g(x(t),m(·, t))
]dt
I Mean field games (∆ =∑n
i=1∂2
∂x2i
Laplacian), solve in
Rn × [0, T ]
−∂tJ + 12 |∇xJ |2 − σ2
2 ∆J + ρJ = g(x,m)
u��
(HJB)
∂tm+ div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Mexican waveLarge populationOil production
Mexican wave (mimicry & fashion)
1z
yL
m(y, t)
0
I state x = [y, z], y ∈ [0, L) coordinate, z position:
z =
{1 standing0 seated
, z ∈ (0, 1) intermediate
I dynamics dz = udt (u control)I penalty on state and distribution g(x,m) =
Kzα(1− z)β︸ ︷︷ ︸comfort
+1
ε2
∫(z − z)2m(y; t, z)
1
εs(y − yε
)dzdy︸ ︷︷ ︸mimicry
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Mexican waveLarge populationOil production
Meeting starting time (coordination with externality)
−xmaxx0
m(x, t)
I dynamics dxi = uidt+ σdBt
I τi = mins(xi(s) = 0) arrival time, ts scheduled time,t actual starting time
I penalty on final state and distributionG(x(τi),m(·, τi)) = c1[τi − ts]+︸ ︷︷ ︸
reputation
+ c2[τi − t]+︸ ︷︷ ︸inconvenience
+ c3[t− τi]+︸ ︷︷ ︸waiting
I people arrived up to time s: F (s) = −∫ s
0 ∂xm(0, v)dv
I starting time t = F−1(θ), (θ is quorum)
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Mexican waveLarge populationOil production
Large population (herd behaviour)
I behaviour dynamicsdxi = uidt+ σdBt
I penalty
g(x,m) = β(x−∫ym(y, t)dy︸ ︷︷ ︸average
)2
I discounted cost J = E∫∞
0 e−ρt[
12 |u(t)|2 + g(x(t),m(·, t))
]dt
I mean field game with discounted cost
−∂tJ + 12 |∇xJ |2 − σ2
2 ∆J + ρJ = g(x,m)
u��
(HJB)
∂tm+ div(m · u(x))− σ2
2 ∆m = 0
m
TT
(Kolmogorov)
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Mexican waveLarge populationOil production
Oil production
I stock market model,
dx = [αtx+ βtu] dt+ σtxdBt
I βtu produced quantity
I penalty (- total income + production costs)
g(x, u,m) = −h(m)u+ [a
2u2 + bu]
I h(m) is sale price of oil (decreasing in m)
I penalty on final state accounts for unexploited reserve:
G(x(T )) = φ|x(T )|2, φ > 0
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Uncertain inflation or taxation
I stock market model,
dx = [αtx+ βtu+ σtζ] dt+ σtxdBt
I σtζ taxation or inflation on theproduction
I penalty (- total income + production costs):
g(x, u,m, ζ) = −h(m, ζ)u+ [a
2u2 + bu]
I cost under worst disturbance [Basar, Bernhard, 1995]:
inf{u}t
sup{ζ}t
E(G(x(T )) +
∫ T
0g(x, u,m, ζ)dt−γ2
∫ T
0|ζ|2dt
).
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
coupled PDEs
∂tJ +Ht(x, ∂xJ,m) +
(σt2γ
)2
|∂xJ |2 +1
2σ2t x
2∂2xxJ = 0,
ζ∗ =σt
2γ2∂xJ, u∗ =
1
βt
[∂pHt(xt,
2γ2
σtζ∗t ,m)− αtxt
]
J,u∗,ζ∗
��m
XX
∂tm+ ∂x (m∂pHt(x, ∂xJ,m)) +σ2t
2γ2∂x(m∂xJ)− 1
2σ2t ∂
2xx
[x2m
]= 0
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
a new equilibrium concept
I worst-case disturbance feedback Nash equilibrium(H∞ literature)
I feedback mean field equilibrium (mean field gamesliterature)
⇓
I worst-case disturbance feedback mean fieldequilibrium
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Simulations 1/3
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Simulations 2/3
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Simulations 3/3
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Conclusions
I Mean field games require solving coupled partialdifferential equations (HJB-Kolmogorov)
I bring robustness within the picture and solve HJB underworst disturbance (robust mean field games)
I worst-case disturbance feedback mean field equilibrium
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Main references
I J.-M. Lasry and P.-L. Lions. Mean field games. JapaneseJournal of Mathematics, 2(1), Mar. 2007.
I O. Gueant, J.-M. Lasry and P.-L. Lions. Mean field gamesand applications. Lect. notes in math., Springer 2011.
I T. Basar, P. Bernhard, H∞-Optimal Control and RelatedMinimax Design Problems: A Dynamic Game Approach.Birkhauser, Boston, MA, 1995
I H. Tembine, Q. Zhu, and T. Basar, Risk-sensitive meanfield stochastic games, IFAC WC 2011, Milano, Italy, 2011.
I M. Bardi, Explicit solutions of some Linear-QuadraticMean Field Games, Network and Heterog. Media, 2012.
I D. Bauso, H. Tembine, T. Basar, Robust Mean FieldGames with Application to Production of ExhaustibleResource. ROCOND ’12
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Links
I Workshop on MFG and related topics (Rome, 2011)http://www.mat.uniroma1.it/ricerca/convegni/2011/mfg/
I Short course on MFG (videos and notes)http://www.ima.umn.edu/2012-2013/SW11.12-13.12/
I MFG Labhttp://mfglabs.com/
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games
Mean field gamesExamples, (O. Gueant et al. 2011)
Robust mean field games
Oil production with uncertainty...generalizing
Questions?
Thank you!
Fabio & Dario, H. Tembine, Q. Zhu, T. Basar Robust mean field games