American Institute of Mathematical Sciences

Advanced
January  2017, 4(1): 59-74. doi: 10.3934/jdg.2017004

Control systems of interacting objects modeled as a game against nature under a mean field approach

Carmen G. Higuera-Chan 1, , Héctor Jasso-Fuentes 2, and  J. Adolfo Minjárez-Sosa 1,,

1. 

Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, Col. Centro, 83000, Hermosillo, Sonora, México
2. 

Departamento de Matemáticas, CINVESTAV-IPN, Apartado Postal 14-740, México D.F., 07000, México

* Corresponding author: aminjare@gauss.mat.uson.mx

Received  December 2015 Revised  December 2016 Published  December 2016

Fund Project: Work partially supported by Consejo Nacional de Ciencia y Tecnologia (CONACyT) under grants CB2015/254306 and CB2015/238045.

This paper deals with discrete-time stochastic systems composed of a large number of N interacting objects (a.k.a. agents or particles). There is a central controller whose decisions, at each stage, affect the system behavior. Each object evolves randomly among a finite set of classes, according to a transition law which depends on an unknown parameter. Such a parameter is possibly non observable and may change from stage to stage. Due to the lack of information and to the large number of agents, the control problem under study is rewritten as a game against nature according to the mean field theory; that is, we introduce a game model associated to the proportions of the objects in each class, whereas the values of the unknown parameter are now considered as "actions" selected by an opponent to the controller (the nature). Then, letting $N \to \infty $ (the mean field limit) and considering a discounted optimality criterion, the objective for the controller is to minimize the maximum cost, where the maximum is taken over all possible strategies of the nature.

Keywords: Systems of interacting objects, mean field theory, discounted criterion, games against nature, minimax control.
Mathematics Subject Classification: Primary: 93E20; Secondary: 60K35.
Citation: Carmen G. Higuera-Chan, Héctor Jasso-Fuentes, J. Adolfo Minjárez-Sosa. Control systems of interacting objects modeled as a game against nature under a mean field approach. Journal of Dynamics & Games, 2017, 4 (1) : 59-74. doi: 10.3934/jdg.2017004
References:
[1]

I. Achdou and I. Capuzzo-Dolcetta, Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.  doi: 10.1137/090758477.  Google Scholar

[2]

M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877.   Google Scholar

[3]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[4]

D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987.  Google Scholar

[5]

A. Budhiraja, P. Dupuis and M. Fischer, Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.  doi: 10.1214/10-AOP616.  Google Scholar

[6]

R. Carmona, J. P. Fouque and D. Vestal, Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.  doi: 10.1007/s00780-009-0098-8.  Google Scholar

[7]

S. P. Coraluppi and S. I. Marcus, Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.  doi: 10.1109/9.847737.  Google Scholar

[8] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.   Google Scholar
[9]

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.  doi: 10.1007/s10626-010-0094-3.  Google Scholar

[10]

N. Gast, N. B. Gaujal and J. Y. Le Boudec, Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.  doi: 10.1109/TAC.2012.2186176.  Google Scholar

[11]

D. A. Gomes, J. Mohr and R. R. Souza, Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.  doi: 10.1016/j.matpur.2009.10.010.  Google Scholar

[12]

T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.  doi: 10.1137/S0363012901383837.  Google Scholar

[13]

C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa, Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.  doi: 10.1007/s00245-015-9312-6.  Google Scholar

[14]

M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.  doi: 10.1137/080735370.  Google Scholar

[15]

H. Jasso-Fuentes and J. D. López-Barrientos, On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.  doi: 10.1080/00207179.2014.984764.  Google Scholar

[16]

V. N. Kolokoltsov, M. Troeva and W. Yang, On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.  doi: 10.1007/s13235-013-0095-6.  Google Scholar

[17]

M. Kurano, Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247.   Google Scholar

[18]

A. Lachapelle, J. Salomon and G. Turinici, Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.  doi: 10.1142/S0218202510004349.  Google Scholar

[19]

J. M. Lasry and P. L. Lions, Mean field games, Jap. J.Math., 2 (2007), 229-260.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[20]

J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007). doi: 10.1109/QEST.2007.8.  Google Scholar

[21]

J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo, Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.  doi: 10.1007/s11750-014-0323-2.  Google Scholar

[22]

F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas, Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.  doi: 10.1007/s10440-011-9605-y.  Google Scholar

[23]

N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam. Google Scholar

[24]

A. Schield, Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.  doi: 10.1007/s00186-007-0172-y.  Google Scholar

show all references

References:
[1]

I. Achdou and I. Capuzzo-Dolcetta, Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.  doi: 10.1137/090758477.  Google Scholar

[2]

M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877.   Google Scholar

[3]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[4]

D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987.  Google Scholar

[5]

A. Budhiraja, P. Dupuis and M. Fischer, Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.  doi: 10.1214/10-AOP616.  Google Scholar

[6]

R. Carmona, J. P. Fouque and D. Vestal, Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.  doi: 10.1007/s00780-009-0098-8.  Google Scholar

[7]

S. P. Coraluppi and S. I. Marcus, Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.  doi: 10.1109/9.847737.  Google Scholar

[8] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.   Google Scholar
[9]

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.  doi: 10.1007/s10626-010-0094-3.  Google Scholar

[10]

N. Gast, N. B. Gaujal and J. Y. Le Boudec, Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.  doi: 10.1109/TAC.2012.2186176.  Google Scholar

[11]

D. A. Gomes, J. Mohr and R. R. Souza, Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.  doi: 10.1016/j.matpur.2009.10.010.  Google Scholar

[12]

T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.  doi: 10.1137/S0363012901383837.  Google Scholar

[13]

C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa, Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.  doi: 10.1007/s00245-015-9312-6.  Google Scholar

[14]

M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.  doi: 10.1137/080735370.  Google Scholar

[15]

H. Jasso-Fuentes and J. D. López-Barrientos, On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.  doi: 10.1080/00207179.2014.984764.  Google Scholar

[16]

V. N. Kolokoltsov, M. Troeva and W. Yang, On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.  doi: 10.1007/s13235-013-0095-6.  Google Scholar

[17]

M. Kurano, Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247.   Google Scholar

[18]

A. Lachapelle, J. Salomon and G. Turinici, Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.  doi: 10.1142/S0218202510004349.  Google Scholar

[19]

J. M. Lasry and P. L. Lions, Mean field games, Jap. J.Math., 2 (2007), 229-260.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[20]

J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007). doi: 10.1109/QEST.2007.8.  Google Scholar

[21]

J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo, Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.  doi: 10.1007/s11750-014-0323-2.  Google Scholar

[22]

F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas, Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.  doi: 10.1007/s10440-011-9605-y.  Google Scholar

[23]

N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam. Google Scholar

[24]

A. Schield, Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.  doi: 10.1007/s00186-007-0172-y.  Google Scholar

[1]

Yves Achdou, Victor Perez. Iterative strategies for solving linearized discrete mean field games systems. Networks & Heterogeneous Media, 2012, 7 (2) : 197-217. doi: 10.3934/nhm.2012.7.197
[2]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control & Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026
[3]

Pierre Cardaliaguet, Jean-Michel Lasry, Pierre-Louis Lions, Alessio Porretta. Long time average of mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 279-301. doi: 10.3934/nhm.2012.7.279
[4]

Fabio Camilli, Elisabetta Carlini, Claudio Marchi. A model problem for Mean Field Games on networks. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 4173-4192. doi: 10.3934/dcds.2015.35.4173
[5]

Martin Burger, Marco Di Francesco, Peter A. Markowich, Marie-Therese Wolfram. Mean field games with nonlinear mobilities in pedestrian dynamics. Discrete & Continuous Dynamical Systems - B, 2014, 19 (5) : 1311-1333. doi: 10.3934/dcdsb.2014.19.1311
[6]

Yves Achdou, Manh-Khang Dao, Olivier Ley, Nicoletta Tchou. A class of infinite horizon mean field games on networks. Networks & Heterogeneous Media, 2019, 14 (3) : 537-566. doi: 10.3934/nhm.2019021
[7]

Josu Doncel, Nicolas Gast, Bruno Gaujal. Discrete mean field games: Existence of equilibria and convergence. Journal of Dynamics & Games, 2019, 6 (3) : 221-239. doi: 10.3934/jdg.2019016
[8]

Martino Bardi. Explicit solutions of some linear-quadratic mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 243-261. doi: 10.3934/nhm.2012.7.243
[9]

Diogo A. Gomes, Gabriel E. Pires, Héctor Sánchez-Morgado. A-priori estimates for stationary mean-field games. Networks & Heterogeneous Media, 2012, 7 (2) : 303-314. doi: 10.3934/nhm.2012.7.303
[10]

Olivier Guéant. New numerical methods for mean field games with quadratic costs. Networks & Heterogeneous Media, 2012, 7 (2) : 315-336. doi: 10.3934/nhm.2012.7.315
[11]

Juan Pablo Maldonado López. Discrete time mean field games: The short-stage limit. Journal of Dynamics & Games, 2015, 2 (1) : 89-101. doi: 10.3934/jdg.2015.2.89
[12]

Matt Barker. From mean field games to the best reply strategy in a stochastic framework. Journal of Dynamics & Games, 2019, 6 (4) : 291-314. doi: 10.3934/jdg.2019020
[13]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks & Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699
[14]

Dung Tien Nguyen, Son Luu Nguyen, Nguyen Huu Du. On mean field systems with multi-classes. Discrete & Continuous Dynamical Systems - A, 2020, 40 (2) : 683-707. doi: 10.3934/dcds.2020057
[15]

Xiaoyu Zheng, Peter Palffy-Muhoray. One order parameter tensor mean field theory for biaxial liquid crystals. Discrete & Continuous Dynamical Systems - B, 2011, 15 (2) : 475-490. doi: 10.3934/dcdsb.2011.15.475
[16]

Santiago Capriotti. Dirac constraints in field theory and exterior differential systems. Journal of Geometric Mechanics, 2010, 2 (1) : 1-50. doi: 10.3934/jgm.2010.2.1
[17]

Diogo Gomes, Marc Sedjro. One-dimensional, forward-forward mean-field games with congestion. Discrete & Continuous Dynamical Systems - S, 2018, 11 (5) : 901-914. doi: 10.3934/dcdss.2018054
[18]

Yves Achdou, Mathieu Laurière. On the system of partial differential equations arising in mean field type control. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3879-3900. doi: 10.3934/dcds.2015.35.3879
[19]

Sylvain Sorin, Guillaume Vigeral. Reversibility and oscillations in zero-sum discounted stochastic games. Journal of Dynamics & Games, 2015, 2 (1) : 103-115. doi: 10.3934/jdg.2015.2.103
[20]

Victor Kozyakin. Minimax joint spectral radius and stabilizability of discrete-time linear switching control systems. Discrete & Continuous Dynamical Systems - B, 2019, 24 (8) : 3537-3556. doi: 10.3934/dcdsb.2018277

 Impact Factor: 

Tools

Metrics

  • PDF downloads (28)
  • HTML views (90)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2019 American Institute of Mathematical Sciences