American Institute of Mathematical Sciences

Advanced
January  2017, 4(1): 59-74. doi: 10.3934/jdg.2017004

Control systems of interacting objects modeled as a game against nature under a mean field approach

Carmen G. Higuera-Chan 1, , Héctor Jasso-Fuentes 2, and  J. Adolfo Minjárez-Sosa 1,,

1. 

Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, Col. Centro, 83000, Hermosillo, Sonora, México
2. 

Departamento de Matemáticas, CINVESTAV-IPN, Apartado Postal 14-740, México D.F., 07000, México

* Corresponding author: aminjare@gauss.mat.uson.mx

Received  December 2015 Revised  December 2016 Published  December 2016

Fund Project: Work partially supported by Consejo Nacional de Ciencia y Tecnologia (CONACyT) under grants CB2015/254306 and CB2015/238045.

This paper deals with discrete-time stochastic systems composed of a large number of N interacting objects (a.k.a. agents or particles). There is a central controller whose decisions, at each stage, affect the system behavior. Each object evolves randomly among a finite set of classes, according to a transition law which depends on an unknown parameter. Such a parameter is possibly non observable and may change from stage to stage. Due to the lack of information and to the large number of agents, the control problem under study is rewritten as a game against nature according to the mean field theory; that is, we introduce a game model associated to the proportions of the objects in each class, whereas the values of the unknown parameter are now considered as "actions" selected by an opponent to the controller (the nature). Then, letting $N \to \infty $ (the mean field limit) and considering a discounted optimality criterion, the objective for the controller is to minimize the maximum cost, where the maximum is taken over all possible strategies of the nature.

Keywords: Systems of interacting objects, mean field theory, discounted criterion, games against nature, minimax control.
Mathematics Subject Classification: Primary: 93E20; Secondary: 60K35.
Citation: Carmen G. Higuera-Chan, Héctor Jasso-Fuentes, J. Adolfo Minjárez-Sosa. Control systems of interacting objects modeled as a game against nature under a mean field approach. Journal of Dynamics and Games, 2017, 4 (1) : 59-74. doi: 10.3934/jdg.2017004
References:
[1]

I. Achdou and I. Capuzzo-Dolcetta, Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.  doi: 10.1137/090758477.

[2]

M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877. 

[3]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013. doi: 10.1007/978-1-4614-8508-7.

[4]

D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987.

[5]

A. Budhiraja, P. Dupuis and M. Fischer, Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.  doi: 10.1214/10-AOP616.

[6]

R. Carmona, J. P. Fouque and D. Vestal, Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.  doi: 10.1007/s00780-009-0098-8.

[7]

S. P. Coraluppi and S. I. Marcus, Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.  doi: 10.1109/9.847737.

[8] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979. 
[9]

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.  doi: 10.1007/s10626-010-0094-3.

[10]

N. Gast, N. B. Gaujal and J. Y. Le Boudec, Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.  doi: 10.1109/TAC.2012.2186176.

[11]

D. A. Gomes, J. Mohr and R. R. Souza, Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.  doi: 10.1016/j.matpur.2009.10.010.

[12]

T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.  doi: 10.1137/S0363012901383837.

[13]

C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa, Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.  doi: 10.1007/s00245-015-9312-6.

[14]

M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.  doi: 10.1137/080735370.

[15]

H. Jasso-Fuentes and J. D. López-Barrientos, On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.  doi: 10.1080/00207179.2014.984764.

[16]

V. N. Kolokoltsov, M. Troeva and W. Yang, On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.  doi: 10.1007/s13235-013-0095-6.

[17]

M. Kurano, Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247. 

[18]

A. Lachapelle, J. Salomon and G. Turinici, Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.  doi: 10.1142/S0218202510004349.

[19]

J. M. Lasry and P. L. Lions, Mean field games, Jap. J.Math., 2 (2007), 229-260.  doi: 10.1007/s11537-007-0657-8.

[20]

J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007). doi: 10.1109/QEST.2007.8.

[21]

J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo, Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.  doi: 10.1007/s11750-014-0323-2.

[22]

F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas, Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.  doi: 10.1007/s10440-011-9605-y.

[23]

N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam.

[24]

A. Schield, Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.  doi: 10.1007/s00186-007-0172-y.

show all references

References:
[1]

I. Achdou and I. Capuzzo-Dolcetta, Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.  doi: 10.1137/090758477.

[2]

M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877. 

[3]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013. doi: 10.1007/978-1-4614-8508-7.

[4]

D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987.

[5]

A. Budhiraja, P. Dupuis and M. Fischer, Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.  doi: 10.1214/10-AOP616.

[6]

R. Carmona, J. P. Fouque and D. Vestal, Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.  doi: 10.1007/s00780-009-0098-8.

[7]

S. P. Coraluppi and S. I. Marcus, Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.  doi: 10.1109/9.847737.

[8] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979. 
[9]

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.  doi: 10.1007/s10626-010-0094-3.

[10]

N. Gast, N. B. Gaujal and J. Y. Le Boudec, Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.  doi: 10.1109/TAC.2012.2186176.

[11]

D. A. Gomes, J. Mohr and R. R. Souza, Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.  doi: 10.1016/j.matpur.2009.10.010.

[12]

T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.  doi: 10.1137/S0363012901383837.

[13]

C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa, Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.  doi: 10.1007/s00245-015-9312-6.

[14]

M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.  doi: 10.1137/080735370.

[15]

H. Jasso-Fuentes and J. D. López-Barrientos, On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.  doi: 10.1080/00207179.2014.984764.

[16]

V. N. Kolokoltsov, M. Troeva and W. Yang, On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.  doi: 10.1007/s13235-013-0095-6.

[17]

M. Kurano, Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247. 

[18]

A. Lachapelle, J. Salomon and G. Turinici, Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.  doi: 10.1142/S0218202510004349.

[19]

J. M. Lasry and P. L. Lions, Mean field games, Jap. J.Math., 2 (2007), 229-260.  doi: 10.1007/s11537-007-0657-8.

[20]

J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007). doi: 10.1109/QEST.2007.8.

[21]

J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo, Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.  doi: 10.1007/s11750-014-0323-2.

[22]

F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas, Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.  doi: 10.1007/s10440-011-9605-y.

[23]

N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam.

[24]

A. Schield, Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.  doi: 10.1007/s00186-007-0172-y.

[1]

Lucio Boccardo, Luigi Orsina. The duality method for mean field games systems. Communications on Pure and Applied Analysis, 2022, 21 (4) : 1343-1360. doi: 10.3934/cpaa.2022021
[2]

Yves Achdou, Victor Perez. Iterative strategies for solving linearized discrete mean field games systems. Networks and Heterogeneous Media, 2012, 7 (2) : 197-217. doi: 10.3934/nhm.2012.7.197
[3]

Kuang Huang, Xuan Di, Qiang Du, Xi Chen. A game-theoretic framework for autonomous vehicles velocity control: Bridging microscopic differential games and macroscopic mean field games. Discrete and Continuous Dynamical Systems - B, 2020, 25 (12) : 4869-4903. doi: 10.3934/dcdsb.2020131
[4]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control and Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026
[5]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control and Related Fields, 2022, 12 (2) : 371-404. doi: 10.3934/mcrf.2021026
[6]

Pierre Cardaliaguet, Jean-Michel Lasry, Pierre-Louis Lions, Alessio Porretta. Long time average of mean field games. Networks and Heterogeneous Media, 2012, 7 (2) : 279-301. doi: 10.3934/nhm.2012.7.279
[7]

Josu Doncel, Nicolas Gast, Bruno Gaujal. Discrete mean field games: Existence of equilibria and convergence. Journal of Dynamics and Games, 2019, 6 (3) : 221-239. doi: 10.3934/jdg.2019016
[8]

Yves Achdou, Manh-Khang Dao, Olivier Ley, Nicoletta Tchou. A class of infinite horizon mean field games on networks. Networks and Heterogeneous Media, 2019, 14 (3) : 537-566. doi: 10.3934/nhm.2019021
[9]

Fabio Camilli, Elisabetta Carlini, Claudio Marchi. A model problem for Mean Field Games on networks. Discrete and Continuous Dynamical Systems, 2015, 35 (9) : 4173-4192. doi: 10.3934/dcds.2015.35.4173
[10]

Martin Burger, Marco Di Francesco, Peter A. Markowich, Marie-Therese Wolfram. Mean field games with nonlinear mobilities in pedestrian dynamics. Discrete and Continuous Dynamical Systems - B, 2014, 19 (5) : 1311-1333. doi: 10.3934/dcdsb.2014.19.1311
[11]

Adriano Festa, Diogo Gomes, Francisco J. Silva, Daniela Tonon. Preface: Mean field games: New trends and applications. Journal of Dynamics and Games, 2021, 8 (4) : i-ii. doi: 10.3934/jdg.2021025
[12]

Marco Cirant, Diogo A. Gomes, Edgard A. Pimentel, Héctor Sánchez-Morgado. On some singular mean-field games. Journal of Dynamics and Games, 2021, 8 (4) : 445-465. doi: 10.3934/jdg.2021006
[13]

Martino Bardi. Explicit solutions of some linear-quadratic mean field games. Networks and Heterogeneous Media, 2012, 7 (2) : 243-261. doi: 10.3934/nhm.2012.7.243
[14]

Diogo A. Gomes, Gabriel E. Pires, Héctor Sánchez-Morgado. A-priori estimates for stationary mean-field games. Networks and Heterogeneous Media, 2012, 7 (2) : 303-314. doi: 10.3934/nhm.2012.7.303
[15]

Matt Barker. From mean field games to the best reply strategy in a stochastic framework. Journal of Dynamics and Games, 2019, 6 (4) : 291-314. doi: 10.3934/jdg.2019020
[16]

Olivier Guéant. New numerical methods for mean field games with quadratic costs. Networks and Heterogeneous Media, 2012, 7 (2) : 315-336. doi: 10.3934/nhm.2012.7.315
[17]

Juan Pablo Maldonado López. Discrete time mean field games: The short-stage limit. Journal of Dynamics and Games, 2015, 2 (1) : 89-101. doi: 10.3934/jdg.2015.2.89
[18]

Laura Aquilanti, Simone Cacace, Fabio Camilli, Raul De Maio. A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions. Journal of Dynamics and Games, 2021, 8 (1) : 35-59. doi: 10.3934/jdg.2020033
[19]

Siting Liu, Levon Nurbekyan. Splitting methods for a class of non-potential mean field games. Journal of Dynamics and Games, 2021, 8 (4) : 467-486. doi: 10.3934/jdg.2021014
[20]

Tigran Bakaryan, Rita Ferreira, Diogo Gomes. A potential approach for planning mean-field games in one dimension. Communications on Pure and Applied Analysis, 2022, 21 (6) : 2147-2187. doi: 10.3934/cpaa.2022054

 Impact Factor: 

Tools

Metrics

  • PDF downloads (268)
  • HTML views (139)
  • Cited by (6)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy