From mean field games to the best reply strategy in a stochastic framework

Matt Barker

doi:10.3934/jdg.2019020

Article Contents

2019, Volume 6, Issue 4: 291-314. Doi: 10.3934/jdg.2019020

This issue Previous Article Games for Pucci's maximal operators Next Article Evolutionary, mean-field and pressure-resistance game modelling of networks security

From mean field games to the best reply strategy in a stochastic framework

Matt Barker^,

Imperial College London, London, SW7 2AZ, UK

^* Corresponding author: Matt Barker
^* Corresponding author: Matt Barker

Received: October 31, 2018

Revised: June 30, 2019

Published: October 2019

Matt Barker is funded by NERC through the SSCP DTP at the Grantham Institute.

Abstract Full Text(HTML) Figure(1) Related Papers Cited by

Abstract

This paper builds on the work of Degond, Herty and Liu in [16] by considering $ N $-player stochastic differential games. The control corresponding to a Nash equilibrium of such a game is approximated through model predictive control (MPC) techniques. In the case of a linear quadratic running-cost, considered here, the MPC method is shown to approximate the solution to the control problem by the best reply strategy (BRS) for the running cost. We then compare the MPC approach when taking the mean field limit with the popular mean field game (MFG) strategy. We find that our MPC approach reduces the two coupled PDEs to a single PDE, greatly increasing the simplicity and tractability of the original problem. We give two examples of applications of this approach to previous literature and conclude with future perspectives for this research.

Keywords:

Mathematics Subject Classification: Primary: 49N70, 35Q93, 91A13; Secondary: 93C20.

Citation:

Full Text(HTML)

Figure 1. Schematic diagram describing the links between $ N $-player games, mean field games and model predictive control

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	Y. Achdou, F. Camilli and I. Capuzzo-Dolcetta, Mean field games: Convergence of a finite difference method, SIAM J. Numer. Anal., 51 (2013), 2585-2612. doi: 10.1137/120882421.
[2]	Y. Achdou, F. Camilli and I. Capuzzo-Dolcetta, Mean field games: Numerical methods for the planning problem, SIAM J. Numer. Anal., 50 (2012), 77-109. doi: 10.1137/100790069.
[3]	Y. Achdou and A. Porretta, Convergence of a finite difference scheme to weak solutions of the system of partial differential equations arising in mean field games, SIAM J. Numer. Anal., 54 (2016), 161-186. doi: 10.1137/15M1015455.
[4]	G. Albi, M. Herty and L. Pareschi, Kinetic description of optimal control problems and applications to opinion consensus, Commun. Math. Sci., 13 (2015), 1407-1429. doi: 10.4310/CMS.2015.v13.n6.a3.
[5]	L. Ambrosio, N. Gigli and G. Savaré, Gradient Flows in Metric Spaces and in the Space of Probability Measures, 2$^{nd}$ edition, Birkhaüser, Basel, 2008. doi: 10.1007/978-3-7643-8722-8.
[6]	R. J. Aumann, Markets with a Continuum of Traders, Econometrica, 32 (1964), 39-50. doi: 10.2307/1913732.
[7]	M. Bardi and F. S. Priuli, Linear-quadratic n-person and mean-field games with ergodic cost, SIAM J. Control Optim., 52 (2014), 3022-3052. doi: 10.1137/140951795.
[8]	A. Blanchet and G. Carlier, From Nash to Cournot-Nash equilibria via the Monge-Kantorovich problem, Philos. T. R. Soc. A, 372 (2014), 20130398, 11 pp. doi: 10.1098/rsta.2013.0398.
[9]	J. P. Bouchaud and M. Mézard, Wealth condensation in a simple model of economy, Physica A, 282 (2000), 536-545. doi: 10.1016/S0378-4371(00)00205-3.
[10]	P. Cardaliaguet, Notes on mean field games, Preprint.
[11]	P. Cardaliaguet, F. Delarue, J.-M. Lasry and P.-L. Lions, The Master Equation and the Convergence Problem in Mean Field Games, Annals of Mathematics Studies, 201. Princeton University Press, Princeton, NJ, 2019, arXiv: 1509.02505. doi: 10.2307/j.ctvckq7qf.
[12]	P. Cardaliaguet, P. J. Graber, A. Porretta and D. Tonon, Second order mean field games with degenerate diffusion and local coupling, NODEA-Nonlinear Diff., 22 (2015), 1287-1317. doi: 10.1007/s00030-015-0323-4.
[13]	R. Carmona and F. Delarue, Mean field forward-backward stochastic differential equations, Electron. Commun. Prob., 18 (2013), 15pp. doi: 10.1214/ECP.v18-2446.
[14]	R. Carmona and F. Delarue, Probabilistic analysis of mean-field games, SIAM J. Control Optim., 51 (2013), 2705-2734. doi: 10.1137/120883499.
[15]	R. Carmona, F. Delarue and D. Lacker, Mean field games with common noise, Ann. Probab., 44 (2016), 3740-3803. doi: 10.1214/15-AOP1060.
[16]	P. Degond, M. Herty and J.-G. Liu, Meanfield games and model predictive control, Commun. Math. Sci., 15 (2017), 1403-1422. doi: 10.4310/CMS.2017.v15.n5.a9.
[17]	P. Degond, J.-G. Liu and C. Ringhofer, Evolution of the distribution of wealth in an economic environment driven by local Nash equilibria, J. Stat. Phys., 154 (2014), 751-780. doi: 10.1007/s10955-013-0888-4.
[18]	P. Degond, J.-G. Liu and C. Ringhofer, Evolution of wealth in a non-conservative economy driven by local Nash equilibria, Philos. T. R. Soc. A, 372 (2014), 20130394, 15 pp. doi: 10.1098/rsta.2013.0394.
[19]	P. Degond, J.-G. Liu and C. Ringhofer, Large-scale dynamics of mean-field games driven by local Nash equilibria, J. Nonlinear Sci., 24 (2014), 93-115. doi: 10.1007/s00332-013-9185-2.
[20]	F. Delarue and R. Carmona, Probabilistic Theory of Mean Field Games with Applications I, Springer, Cham, 2018. doi: 10.1007/978-3-319-58920-6.
[21]	B. Düring and G. Toscani, Hydrodynamics from kinetic models of conservative economies, Physica A, 384 (2007), 493-506. doi: 10.1016/j.physa.2007.05.062.
[22]	E. Feleqi, The derivation of ergodic mean field game equations for several populations of players, Dyn. Games Appl., 3 (2013), 523-536. doi: 10.1007/s13235-013-0088-5.
[23]	A. Friedman, Stochastic differential games, J. Differ. Equations, 11 (1972), 79-108. doi: 10.1016/0022-0396(72)90082-4.
[24]	M. Herty and M. Zanella, Performance bounds for the mean-field limit of constrained dynamics, Discrete Cont. Dyn.–A, 37 (2017), 2023-2043. doi: 10.3934/dcds.2017086.
[25]	M. Huang, P. E. Caines and R. P. Malhamé, An invariance principle in large population stochastic dynamic games, J. Syst. Sci. Complex., 20 (2007), 162-172. doi: 10.1007/s11424-007-9015-4.
[26]	M. Huang, P. E. Caines and R. P. Malhamé, Distributed multi-agent decision-making with partial observations: Asymptotic Nash equilibria, in Proceedings of the 17th International Symposium on Mathematical Theory of Networks and Systems, (2006), 2725–2730.
[27]	M. Huang, P. E. Caines and R. P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized $ \varepsilon$-nash equilibria, IEEE T. Automat. Contr., 52 (2007), 1560-1571. doi: 10.1109/TAC.2007.904450.
[28]	M. Huang, P. E. Caines and R. P. Malhamé, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Communications in Information and Systems, 6 (2006), 221-251. doi: 10.4310/CIS.2006.v6.n3.a5.
[29]	M. Huang, P. E. Caines and R. P. Malhamé, Nash certainty equivalence in large population stochastic dynamic games: Connections with the physics of interacting particle systems, in Proceedings of the 45th IEEE Conference on Decision and Control, (2006), 4921–4926.
[30]	M. Huang and S. L. Nguyen, Mean field games for stochastic growth with relative consumption, Appl. Math. Opt., 74 (2016), 643-668. doi: 10.1007/s00245-016-9395-8.
[31]	B. Jourdain, S. Méléard and W. Woyczynski, Nonlinear SDEs driven by Lévy processes and related PDEs, ALEA–Lat. Am. J. Probab., 4 (2008), 1-29.
[32]	A. C. Kizilkale and R. P. Malhamé, Collective target tracking mean field control for markovian jump-driven models of electric water heating loads,, in Control of Complex Systems: Theory and Applications (eds. K. G. Vamvoudakis and S. Jagannathan) Elsevier, (2016), 559–584. doi: 10.1016/B978-0-12-805246-4.00020-3.
[33]	P. E. Kloeden and E. Platen, Numerical Solution of Stochastic Differential Equations, 2$^{nd}$ edition Springer-Verlag, Berlin Heidelberg, 1992. doi: 10.1007/978-3-662-12616-5.
[34]	A. Lachapelle and M. -T. Wolfram, On a mean field game approach modeling congestion and aversion in pedestrian crowds, Transport. Res. B–Meth., 45 (2011), 1572-1589. doi: 10.1016/j.trb.2011.07.011.
[35]	J. M. Lasry and P. L. Lions, Jeux à champ moyen. Ⅰ — Le cas stationnaire, C. r. math., 343 (2006), 619-625. doi: 10.1016/j.crma.2006.09.019.
[36]	J. M. Lasry and P. L. Lions, Jeux à champ moyen. Ⅱ — Horizon fini et contrôle optimal, C. R. Math., 343 (2006), 679-684. doi: 10.1016/j.crma.2006.09.018.
[37]	J. M. Lasry and P. L. Lions, Mean field games, Jpn. J. Math., 2 (2007), 229-260. doi: 10.1007/s11537-007-0657-8.
[38]	A. Mas-Colell, On a theorem of Schmeidler, J. Math. Econ., 13 (1984), 201-206. doi: 10.1016/0304-4068(84)90029-6.
[39]	D. Q. Mayne and H. Michalska, Receding horizon control of nonlinear systems, IEEE T. Automat. Contr., 35 (1990), 814-824. doi: 10.1109/9.57020.
[40]	B. Oksendal, Stochastic Differential Equations: An Introduction with Applications, Springer-Verlag, Berlin Heidelberg, 2003. doi: 10.1007/978-3-642-14394-6.
[41]	D. Schmeidler, Equilibrium points of nonatomic games, J. Stat. Phys., 7 (1973), 295-300. doi: 10.1007/BF01014905.
[42]	A.-S. Sznitman, Topics in propagation of chaos, in Ecole d'Eté de Probabilités de Saint-Flour XIX–1989 (Ed. P.-L. Hennequin), Springer, Berlin Heidelberg, 1464 (1991), 165–251. doi: 10.1007/BFb0085169.