# American Institute of Mathematical Sciences

• Previous Article
A general maximum principle for partially observed mean-field stochastic system with random jumps in progressive structure
• MCRF Home
• This Issue
• Next Article
Barrier Lyapunov functions-based adaptive neural tracking control for non-strict feedback stochastic nonlinear systems with full-state constraints: A command filter approach
doi: 10.3934/mcrf.2022029
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

## Control theory approach to continuous-time finite state mean field games

 Higher School of Economics, Moscow, Russia

Received  March 2021 Revised  January 2022 Early access July 2022

Fund Project: The article was prepared in the framework of a research grant funded by the Ministry of Science and Higher Education of the Russian Federation (grant ID: 075-15-2020-928)

In the paper, we study the dependence of solutions of the continuous-time finite state mean field game on initial distribution of players. Our approach relies on the concept of value multifunction of the mean field game that is a mapping assigning to an initial time and an initial distribution a set of expected outcomes of the representative player corresponding to solutions of the mean field game. Using the reformulation of the finite state mean field game as a control problem with mixed constraints, we give the sufficient condition on a given multifunction to be a value multifunction in the terms of the viability theory. The maximal multifunction (i.e., the mapping assigning to an initial time and an initial distribution the whole set of values corresponding to solutions of the mean field game) is characterized via the backward attainability set for the certain control system.

Citation: Yurii Averboukh. Control theory approach to continuous-time finite state mean field games. Mathematical Control and Related Fields, doi: 10.3934/mcrf.2022029
##### References:
 [1] C. D. Aliprantis and K. C. Border, Infinite Dimensional Analysis: A Hitchhiker's Guide, Springer, Berlin, Heidelberg, 2006. [2] L. Ambrosio, N. Gigli and G. Savaré, Gradient Flows in Metric Spaces and in the Space of Probability Measures, Lectures in Mathematics ETH Zurich, Birkhäuser, Basel, 2005. [3] J.-P. Aubin, Viability Theory, Systems & Control: Foundations & Applications. Birkhäuser Boston, Inc., Boston, MA, 1991. [4] R. Basna, A. Hilbert and V. N. Kolokoltsov, An approximate Nash equilibrium for pure jump Markov games of mean-field-type on continuous state space, Stochastics, 89 (2017), 967-993.  doi: 10.1080/17442508.2017.1297812. [5] E. Bayraktar, A. Cecchin, A. Cohen and F. Delarue, Finite state mean field games with Wright-Fisher common noise, J. Math. Pures Appl., 147 (2021), 98-162.  doi: 10.1016/j.matpur.2021.01.003. [6] E. Bayraktar, A. Cecchin, A. Cohen and F. Delarue, Finite state mean field games with Wright-Fisher common noise as limits of ${N}$-player weighted games, Math. Oper. Res., 147 (2021), 98-162.  doi: 10.1016/j.matpur.2021.01.003. [7] E. Bayraktar and A. Cohen, Analysis of a finite state many player game using its master equation, SIAM J. Control. Optim., 56 (2018), 3538-3568.  doi: 10.1137/17M113887X. [8] E. Bayraktar and X. Zhang, On non-uniqueness in mean field games, Proc. Amer. Math. Soc., 148 (2020), 4091-4106.  doi: 10.1090/proc/15046. [9] C. Belak, D. Hoffmann and F. T. Seifried, Continuous-time mean field games with finite state space and common noise, Appl. Math. Optim., 84 (2021), 3173-3216.  doi: 10.1007/s00245-020-09743-7. [10] P. Cardaliaguet, F. Delarue, J.-M. Lasry and P.-L. Lions, The Master Equation and the Convergence Problem in Mean Field Games, Annals of Mathematics Studies, 201. Princeton University Press, Princeton, 2019. doi: 10.2307/j.ctvckq7qf. [11] A. Cecchin and M. Fischer, Probabilistic approach to finite state mean field games, Appl. Math. Opt., 81 (2020), 253-300.  doi: 10.1007/s00245-018-9488-7. [12] A. Cecchin and G. Pelino, Convergence, fluctuations and large deviations for finite state mean field games via the master equation, Stochastic Process. Appl., 129 (2019), 4510-4555.  doi: 10.1016/j.spa.2018.12.002. [13] C. Dellacherie and P.-A. Meyer, Probabilities and Potential, North-Holland Mathematics Studies, 29. North-Holland Publishing Co., Amsterdam-New York, 1978. [14] L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, Textbooks in Mathematics, CRC Press, Boca Raton, FL, 2015. [15] W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006. [16] D. A. Gomes, J. Mohr and R. R. Souza, Continuous time finite state mean field games, Appl. Math. Opt., 68 (2013), 99-143.  doi: 10.1007/s00245-013-9202-8. [17] X. Guo and O. Hernández-Lerma, Continuous-Time Markov Decision Processes, Stochastic Modelling and Applied Probability, 62. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-642-02547-1. [18] M. Huang, P. E. Caines and R. P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized Nash equilibria, IEEE Trans. Automat. Control, 52 (2007), 1560-1571.  doi: 10.1109/TAC.2007.904450. [19] M. Huang, R. P. Malhamé and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Commun. Inf. Syst., 6 (2006), 221-251.  doi: 10.4310/CIS.2006.v6.n3.a5. [20] S. Katsikas and V. Kolokoltsov, Evolutionary, mean-field and pressure-resistance game modelling of networks security, J. Dyn. Games, 6 (2019), 315-335.  doi: 10.3934/jdg.2019021. [21] V. Kolokoltsov and W. Yang, Inspection games in a mean field setting, (2015), arXiv: 1507.08339. [22] V. N. Kolokoltsov, Nonlinear Markov Process and Kinetic Equations, Cambridge Tracts in Mathematics, 182. Cambridge University Press, Cambridge, 2010. doi: 10.1017/CBO9780511760303. [23] V. N. Kolokoltsov and A. Bensoussan, Mean-field-game model for botnet defense in cyber-security, Appl. Math. Opt., 74 (2016), 669-692.  doi: 10.1007/s00245-016-9389-6. [24] V. N. Kolokoltsov, J. J. Li and W. Yang, Mean field games and nonlinear Markov processes, (2011), arXiv: 1112.3744v2. [25] V. N. Kolokoltsov and O. A. Malafeyev, Many Agent Games in Socio-Economic Systems: Corruption, Inspection, Coalition Building, Network Growth, Security, Springer Series in Operations Research and Financial Engineering, Springer, Cham, 2019. doi: 10.1007/978-3-030-12371-0. [26] V. N. Kolokoltsov and O. A. Malafeyev, Corruption and botnet defense: A mean field game approach, Int. J. Game Theory, 47 (2018), 977-999.  doi: 10.1007/s00182-018-0614-1. [27] J.-M. Lasry and P.-L. Lions, Jeux à champ moyen. Ⅰ. Le cas stationnaire, C. R. Math. Acad. Sci. Paris, 343 (2006), 619-625.  doi: 10.1016/j.crma.2006.09.019. [28] J.-M. Lasry and P.-L. Lions, Jeux à champ moyen. Ⅱ. Horizon fini et contrôle optimal, C. R. Math. Acad. Sci. Paris, 343 (2006), 679-684.  doi: 10.1016/j.crma.2006.09.018. [29] P.-L. Lions, College de France Course on Mean-Field Games, College de France, 2007–2011.

show all references

##### References:
 [1] C. D. Aliprantis and K. C. Border, Infinite Dimensional Analysis: A Hitchhiker's Guide, Springer, Berlin, Heidelberg, 2006. [2] L. Ambrosio, N. Gigli and G. Savaré, Gradient Flows in Metric Spaces and in the Space of Probability Measures, Lectures in Mathematics ETH Zurich, Birkhäuser, Basel, 2005. [3] J.-P. Aubin, Viability Theory, Systems & Control: Foundations & Applications. Birkhäuser Boston, Inc., Boston, MA, 1991. [4] R. Basna, A. Hilbert and V. N. Kolokoltsov, An approximate Nash equilibrium for pure jump Markov games of mean-field-type on continuous state space, Stochastics, 89 (2017), 967-993.  doi: 10.1080/17442508.2017.1297812. [5] E. Bayraktar, A. Cecchin, A. Cohen and F. Delarue, Finite state mean field games with Wright-Fisher common noise, J. Math. Pures Appl., 147 (2021), 98-162.  doi: 10.1016/j.matpur.2021.01.003. [6] E. Bayraktar, A. Cecchin, A. Cohen and F. Delarue, Finite state mean field games with Wright-Fisher common noise as limits of ${N}$-player weighted games, Math. Oper. Res., 147 (2021), 98-162.  doi: 10.1016/j.matpur.2021.01.003. [7] E. Bayraktar and A. Cohen, Analysis of a finite state many player game using its master equation, SIAM J. Control. Optim., 56 (2018), 3538-3568.  doi: 10.1137/17M113887X. [8] E. Bayraktar and X. Zhang, On non-uniqueness in mean field games, Proc. Amer. Math. Soc., 148 (2020), 4091-4106.  doi: 10.1090/proc/15046. [9] C. Belak, D. Hoffmann and F. T. Seifried, Continuous-time mean field games with finite state space and common noise, Appl. Math. Optim., 84 (2021), 3173-3216.  doi: 10.1007/s00245-020-09743-7. [10] P. Cardaliaguet, F. Delarue, J.-M. Lasry and P.-L. Lions, The Master Equation and the Convergence Problem in Mean Field Games, Annals of Mathematics Studies, 201. Princeton University Press, Princeton, 2019. doi: 10.2307/j.ctvckq7qf. [11] A. Cecchin and M. Fischer, Probabilistic approach to finite state mean field games, Appl. Math. Opt., 81 (2020), 253-300.  doi: 10.1007/s00245-018-9488-7. [12] A. Cecchin and G. Pelino, Convergence, fluctuations and large deviations for finite state mean field games via the master equation, Stochastic Process. Appl., 129 (2019), 4510-4555.  doi: 10.1016/j.spa.2018.12.002. [13] C. Dellacherie and P.-A. Meyer, Probabilities and Potential, North-Holland Mathematics Studies, 29. North-Holland Publishing Co., Amsterdam-New York, 1978. [14] L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, Textbooks in Mathematics, CRC Press, Boca Raton, FL, 2015. [15] W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006. [16] D. A. Gomes, J. Mohr and R. R. Souza, Continuous time finite state mean field games, Appl. Math. Opt., 68 (2013), 99-143.  doi: 10.1007/s00245-013-9202-8. [17] X. Guo and O. Hernández-Lerma, Continuous-Time Markov Decision Processes, Stochastic Modelling and Applied Probability, 62. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-642-02547-1. [18] M. Huang, P. E. Caines and R. P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized Nash equilibria, IEEE Trans. Automat. Control, 52 (2007), 1560-1571.  doi: 10.1109/TAC.2007.904450. [19] M. Huang, R. P. Malhamé and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Commun. Inf. Syst., 6 (2006), 221-251.  doi: 10.4310/CIS.2006.v6.n3.a5. [20] S. Katsikas and V. Kolokoltsov, Evolutionary, mean-field and pressure-resistance game modelling of networks security, J. Dyn. Games, 6 (2019), 315-335.  doi: 10.3934/jdg.2019021. [21] V. Kolokoltsov and W. Yang, Inspection games in a mean field setting, (2015), arXiv: 1507.08339. [22] V. N. Kolokoltsov, Nonlinear Markov Process and Kinetic Equations, Cambridge Tracts in Mathematics, 182. Cambridge University Press, Cambridge, 2010. doi: 10.1017/CBO9780511760303. [23] V. N. Kolokoltsov and A. Bensoussan, Mean-field-game model for botnet defense in cyber-security, Appl. Math. Opt., 74 (2016), 669-692.  doi: 10.1007/s00245-016-9389-6. [24] V. N. Kolokoltsov, J. J. Li and W. Yang, Mean field games and nonlinear Markov processes, (2011), arXiv: 1112.3744v2. [25] V. N. Kolokoltsov and O. A. Malafeyev, Many Agent Games in Socio-Economic Systems: Corruption, Inspection, Coalition Building, Network Growth, Security, Springer Series in Operations Research and Financial Engineering, Springer, Cham, 2019. doi: 10.1007/978-3-030-12371-0. [26] V. N. Kolokoltsov and O. A. Malafeyev, Corruption and botnet defense: A mean field game approach, Int. J. Game Theory, 47 (2018), 977-999.  doi: 10.1007/s00182-018-0614-1. [27] J.-M. Lasry and P.-L. Lions, Jeux à champ moyen. Ⅰ. Le cas stationnaire, C. R. Math. Acad. Sci. Paris, 343 (2006), 619-625.  doi: 10.1016/j.crma.2006.09.019. [28] J.-M. Lasry and P.-L. Lions, Jeux à champ moyen. Ⅱ. Horizon fini et contrôle optimal, C. R. Math. Acad. Sci. Paris, 343 (2006), 679-684.  doi: 10.1016/j.crma.2006.09.018. [29] P.-L. Lions, College de France Course on Mean-Field Games, College de France, 2007–2011.
 [1] Huijuan Li, Junxia Wang. Input-to-state stability of continuous-time systems via finite-time Lyapunov functions. Discrete and Continuous Dynamical Systems - B, 2020, 25 (3) : 841-857. doi: 10.3934/dcdsb.2019192 [2] Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control and Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028 [3] Kazuhisa Ichikawa, Mahemauti Rouzimaimaiti, Takashi Suzuki. Reaction diffusion equation with non-local term arises as a mean field limit of the master equation. Discrete and Continuous Dynamical Systems - S, 2012, 5 (1) : 115-126. doi: 10.3934/dcdss.2012.5.115 [4] Ping Chen, Haixiang Yao. Continuous-time mean-variance portfolio selection with no-shorting constraints and regime-switching. Journal of Industrial and Management Optimization, 2020, 16 (2) : 531-551. doi: 10.3934/jimo.2018166 [5] Huai-Nian Zhu, Cheng-Ke Zhang, Zhuo Jin. Continuous-time mean-variance asset-liability management with stochastic interest rates and inflation risks. Journal of Industrial and Management Optimization, 2020, 16 (2) : 813-834. doi: 10.3934/jimo.2018180 [6] Theresa Lange, Wilhelm Stannat. Mean field limit of Ensemble Square Root filters - discrete and continuous time. Foundations of Data Science, 2021, 3 (3) : 563-588. doi: 10.3934/fods.2021003 [7] Juan Li, Wenqiang Li. Controlled reflected mean-field backward stochastic differential equations coupled with value function and related PDEs. Mathematical Control and Related Fields, 2015, 5 (3) : 501-516. doi: 10.3934/mcrf.2015.5.501 [8] Joon Kwon, Panayotis Mertikopoulos. A continuous-time approach to online optimization. Journal of Dynamics and Games, 2017, 4 (2) : 125-148. doi: 10.3934/jdg.2017008 [9] Hanqing Jin, Xun Yu Zhou. Continuous-time portfolio selection under ambiguity. Mathematical Control and Related Fields, 2015, 5 (3) : 475-488. doi: 10.3934/mcrf.2015.5.475 [10] Fabio Bagagiolo, Luciano Marzufero. A time-dependent switching mean-field game on networks motivated by optimal visiting problems. Journal of Dynamics and Games, 2022  doi: 10.3934/jdg.2022019 [11] César Barilla, Guillaume Carlier, Jean-Michel Lasry. A mean field game model for the evolution of cities. Journal of Dynamics and Games, 2021, 8 (3) : 299-329. doi: 10.3934/jdg.2021017 [12] René Aïd, Roxana Dumitrescu, Peter Tankov. The entry and exit game in the electricity markets: A mean-field game approach. Journal of Dynamics and Games, 2021, 8 (4) : 331-358. doi: 10.3934/jdg.2021012 [13] Laurent Bourgeois, Dmitry Ponomarev, Jérémi Dardé. An inverse obstacle problem for the wave equation in a finite time domain. Inverse Problems and Imaging, 2019, 13 (2) : 377-400. doi: 10.3934/ipi.2019019 [14] Fritz Colonius, Guilherme Mazanti. Decay rates for stabilization of linear continuous-time systems with random switching. Mathematical Control and Related Fields, 2019, 9 (1) : 39-58. doi: 10.3934/mcrf.2019002 [15] Shui-Nee Chow, Xiaojing Ye, Hongyuan Zha, Haomin Zhou. Influence prediction for continuous-time information propagation on networks. Networks and Heterogeneous Media, 2018, 13 (4) : 567-583. doi: 10.3934/nhm.2018026 [16] J. C. Dallon, Lynnae C. Despain, Emily J. Evans, Christopher P. Grant. A continuous-time stochastic model of cell motion in the presence of a chemoattractant. Discrete and Continuous Dynamical Systems - B, 2020, 25 (12) : 4839-4852. doi: 10.3934/dcdsb.2020129 [17] Hui Meng, Fei Lung Yuen, Tak Kuen Siu, Hailiang Yang. Optimal portfolio in a continuous-time self-exciting threshold model. Journal of Industrial and Management Optimization, 2013, 9 (2) : 487-504. doi: 10.3934/jimo.2013.9.487 [18] Wenpin Tang, Xun Yu Zhou. Tail probability estimates of continuous-time simulated annealing processes. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022015 [19] Andy Hammerlindl, Bernd Krauskopf, Gemma Mason, Hinke M. Osinga. Determining the global manifold structure of a continuous-time heterodimensional cycle. Journal of Computational Dynamics, 2022, 9 (3) : 393-419. doi: 10.3934/jcd.2022008 [20] Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Mean-field backward stochastic Volterra integral equations. Discrete and Continuous Dynamical Systems - B, 2013, 18 (7) : 1929-1967. doi: 10.3934/dcdsb.2013.18.1929

2021 Impact Factor: 1.141