-
Previous Article
Discretized best-response dynamics for the Rock-Paper-Scissors game
- JDG Home
- This Issue
-
Next Article
Price of anarchy for graph coloring games with concave payoff
Control systems of interacting objects modeled as a game against nature under a mean field approach
1. | Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, Col. Centro, 83000, Hermosillo, Sonora, México |
2. | Departamento de Matemáticas, CINVESTAV-IPN, Apartado Postal 14-740, México D.F., 07000, México |
This paper deals with discrete-time stochastic systems composed of a large number of N interacting objects (a.k.a. agents or particles). There is a central controller whose decisions, at each stage, affect the system behavior. Each object evolves randomly among a finite set of classes, according to a transition law which depends on an unknown parameter. Such a parameter is possibly non observable and may change from stage to stage. Due to the lack of information and to the large number of agents, the control problem under study is rewritten as a game against nature according to the mean field theory; that is, we introduce a game model associated to the proportions of the objects in each class, whereas the values of the unknown parameter are now considered as "actions" selected by an opponent to the controller (the nature). Then, letting $N \to \infty $ (the mean field limit) and considering a discounted optimality criterion, the objective for the controller is to minimize the maximum cost, where the maximum is taken over all possible strategies of the nature.
References:
[1] |
I. Achdou and I. Capuzzo-Dolcetta,
Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.
doi: 10.1137/090758477. |
[2] |
M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877. Google Scholar |
[3] |
A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013.
doi: 10.1007/978-1-4614-8508-7. |
[4] |
D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987. |
[5] |
A. Budhiraja, P. Dupuis and M. Fischer,
Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.
doi: 10.1214/10-AOP616. |
[6] |
R. Carmona, J. P. Fouque and D. Vestal,
Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.
doi: 10.1007/s00780-009-0098-8. |
[7] |
S. P. Coraluppi and S. I. Marcus,
Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.
doi: 10.1109/9.847737. |
[8] |
E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.
![]() |
[9] |
N. Gast and B. Gaujal,
A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.
doi: 10.1007/s10626-010-0094-3. |
[10] |
N. Gast, N. B. Gaujal and J. Y. Le Boudec,
Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.
doi: 10.1109/TAC.2012.2186176. |
[11] |
D. A. Gomes, J. Mohr and R. R. Souza,
Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.
doi: 10.1016/j.matpur.2009.10.010. |
[12] |
T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes,
Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.
doi: 10.1137/S0363012901383837. |
[13] |
C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa,
Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.
doi: 10.1007/s00245-015-9312-6. |
[14] |
M. Huang,
Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.
doi: 10.1137/080735370. |
[15] |
H. Jasso-Fuentes and J. D. López-Barrientos,
On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.
doi: 10.1080/00207179.2014.984764. |
[16] |
V. N. Kolokoltsov, M. Troeva and W. Yang,
On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.
doi: 10.1007/s13235-013-0095-6. |
[17] |
M. Kurano,
Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247.
|
[18] |
A. Lachapelle, J. Salomon and G. Turinici,
Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.
doi: 10.1142/S0218202510004349. |
[19] |
J. M. Lasry and P. L. Lions,
Mean field games, Jap. J.Math., 2 (2007), 229-260.
doi: 10.1007/s11537-007-0657-8. |
[20] |
J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007).
doi: 10.1109/QEST.2007.8. |
[21] |
J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo,
Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.
doi: 10.1007/s11750-014-0323-2. |
[22] |
F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas,
Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.
doi: 10.1007/s10440-011-9605-y. |
[23] |
N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam. Google Scholar |
[24] |
A. Schield,
Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.
doi: 10.1007/s00186-007-0172-y. |
show all references
References:
[1] |
I. Achdou and I. Capuzzo-Dolcetta,
Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.
doi: 10.1137/090758477. |
[2] |
M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877. Google Scholar |
[3] |
A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013.
doi: 10.1007/978-1-4614-8508-7. |
[4] |
D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987. |
[5] |
A. Budhiraja, P. Dupuis and M. Fischer,
Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.
doi: 10.1214/10-AOP616. |
[6] |
R. Carmona, J. P. Fouque and D. Vestal,
Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.
doi: 10.1007/s00780-009-0098-8. |
[7] |
S. P. Coraluppi and S. I. Marcus,
Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.
doi: 10.1109/9.847737. |
[8] |
E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.
![]() |
[9] |
N. Gast and B. Gaujal,
A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.
doi: 10.1007/s10626-010-0094-3. |
[10] |
N. Gast, N. B. Gaujal and J. Y. Le Boudec,
Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.
doi: 10.1109/TAC.2012.2186176. |
[11] |
D. A. Gomes, J. Mohr and R. R. Souza,
Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.
doi: 10.1016/j.matpur.2009.10.010. |
[12] |
T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes,
Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.
doi: 10.1137/S0363012901383837. |
[13] |
C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa,
Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.
doi: 10.1007/s00245-015-9312-6. |
[14] |
M. Huang,
Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.
doi: 10.1137/080735370. |
[15] |
H. Jasso-Fuentes and J. D. López-Barrientos,
On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.
doi: 10.1080/00207179.2014.984764. |
[16] |
V. N. Kolokoltsov, M. Troeva and W. Yang,
On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.
doi: 10.1007/s13235-013-0095-6. |
[17] |
M. Kurano,
Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247.
|
[18] |
A. Lachapelle, J. Salomon and G. Turinici,
Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.
doi: 10.1142/S0218202510004349. |
[19] |
J. M. Lasry and P. L. Lions,
Mean field games, Jap. J.Math., 2 (2007), 229-260.
doi: 10.1007/s11537-007-0657-8. |
[20] |
J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007).
doi: 10.1109/QEST.2007.8. |
[21] |
J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo,
Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.
doi: 10.1007/s11750-014-0323-2. |
[22] |
F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas,
Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.
doi: 10.1007/s10440-011-9605-y. |
[23] |
N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam. Google Scholar |
[24] |
A. Schield,
Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.
doi: 10.1007/s00186-007-0172-y. |
[1] |
Laura Aquilanti, Simone Cacace, Fabio Camilli, Raul De Maio. A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions. Journal of Dynamics & Games, 2020 doi: 10.3934/jdg.2020033 |
[2] |
Xuefeng Zhang, Yingbo Zhang. Fault-tolerant control against actuator failures for uncertain singular fractional order systems. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 1-12. doi: 10.3934/naco.2020011 |
[3] |
Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026 |
[4] |
Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020 doi: 10.3934/mcrf.2020046 |
[5] |
Illés Horváth, Kristóf Attila Horváth, Péter Kovács, Miklós Telek. Mean-field analysis of a scaling MAC radio protocol. Journal of Industrial & Management Optimization, 2021, 17 (1) : 279-297. doi: 10.3934/jimo.2019111 |
[6] |
Theresa Lange, Wilhelm Stannat. Mean field limit of ensemble square root filters - discrete and continuous time. Foundations of Data Science, 2021 doi: 10.3934/fods.2021003 |
[7] |
Felix Finster, Jürg Fröhlich, Marco Oppio, Claudio F. Paganini. Causal fermion systems and the ETH approach to quantum theory. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020451 |
[8] |
Shin-Ichiro Ei, Hiroshi Ishii. The motion of weakly interacting localized patterns for reaction-diffusion systems with nonlocal effect. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 173-190. doi: 10.3934/dcdsb.2020329 |
[9] |
Daniele Bartolucci, Changfeng Gui, Yeyao Hu, Aleks Jevnikar, Wen Yang. Mean field equations on tori: Existence and uniqueness of evenly symmetric blow-up solutions. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3093-3116. doi: 10.3934/dcds.2020039 |
[10] |
Alain Bensoussan, Xinwei Feng, Jianhui Huang. Linear-quadratic-Gaussian mean-field-game with partial observation and common noise. Mathematical Control & Related Fields, 2021, 11 (1) : 23-46. doi: 10.3934/mcrf.2020025 |
[11] |
Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, 2021, 20 (1) : 449-465. doi: 10.3934/cpaa.2020276 |
[12] |
Jann-Long Chern, Sze-Guang Yang, Zhi-You Chen, Chih-Her Chen. On the family of non-topological solutions for the elliptic system arising from a product Abelian gauge field theory. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3291-3304. doi: 10.3934/dcds.2020127 |
[13] |
Jian Zhang, Tony T. Lee, Tong Ye, Liang Huang. An approximate mean queue length formula for queueing systems with varying service rate. Journal of Industrial & Management Optimization, 2021, 17 (1) : 185-204. doi: 10.3934/jimo.2019106 |
[14] |
Sergey Rashkovskiy. Hamilton-Jacobi theory for Hamiltonian and non-Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 563-583. doi: 10.3934/jgm.2020024 |
[15] |
Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020 doi: 10.3934/dcdss.2020444 |
[16] |
Duy Phan, Lassi Paunonen. Finite-dimensional controllers for robust regulation of boundary control systems. Mathematical Control & Related Fields, 2021, 11 (1) : 95-117. doi: 10.3934/mcrf.2020029 |
[17] |
Hui Lv, Xing'an Wang. Dissipative control for uncertain singular markovian jump systems via hybrid impulsive control. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 127-142. doi: 10.3934/naco.2020020 |
[18] |
Guillaume Wafo-Tapa, Slim Bettaieb, Loïc Bidoux, Philippe Gaborit, Etienne Marcatel. A practicable timing attack against HQC and its countermeasure. Advances in Mathematics of Communications, 2020 doi: 10.3934/amc.2020126 |
[19] |
Pierre Baras. A generalization of a criterion for the existence of solutions to semilinear elliptic equations. Discrete & Continuous Dynamical Systems - S, 2021, 14 (2) : 465-504. doi: 10.3934/dcdss.2020439 |
[20] |
Pierluigi Colli, Gianni Gilardi, Jürgen Sprekels. Deep quench approximation and optimal control of general Cahn–Hilliard systems with fractional operators and double obstacle potentials. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 243-271. doi: 10.3934/dcdss.2020213 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]