American Institute of Mathematical Sciences

Advanced
January  2017, 4(1): 59-74. doi: 10.3934/jdg.2017004

Control systems of interacting objects modeled as a game against nature under a mean field approach

Carmen G. Higuera-Chan 1, , Héctor Jasso-Fuentes 2, and  J. Adolfo Minjárez-Sosa 1,,

1. 

Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, Col. Centro, 83000, Hermosillo, Sonora, México
2. 

Departamento de Matemáticas, CINVESTAV-IPN, Apartado Postal 14-740, México D.F., 07000, México

* Corresponding author: aminjare@gauss.mat.uson.mx

Received  December 2015 Revised  December 2016 Published  December 2016

Fund Project: Work partially supported by Consejo Nacional de Ciencia y Tecnologia (CONACyT) under grants CB2015/254306 and CB2015/238045.

This paper deals with discrete-time stochastic systems composed of a large number of N interacting objects (a.k.a. agents or particles). There is a central controller whose decisions, at each stage, affect the system behavior. Each object evolves randomly among a finite set of classes, according to a transition law which depends on an unknown parameter. Such a parameter is possibly non observable and may change from stage to stage. Due to the lack of information and to the large number of agents, the control problem under study is rewritten as a game against nature according to the mean field theory; that is, we introduce a game model associated to the proportions of the objects in each class, whereas the values of the unknown parameter are now considered as "actions" selected by an opponent to the controller (the nature). Then, letting $N \to \infty $ (the mean field limit) and considering a discounted optimality criterion, the objective for the controller is to minimize the maximum cost, where the maximum is taken over all possible strategies of the nature.

Keywords: Systems of interacting objects, mean field theory, discounted criterion, games against nature, minimax control.
Mathematics Subject Classification: Primary: 93E20; Secondary: 60K35.
Citation: Carmen G. Higuera-Chan, Héctor Jasso-Fuentes, J. Adolfo Minjárez-Sosa. Control systems of interacting objects modeled as a game against nature under a mean field approach. Journal of Dynamics & Games, 2017, 4 (1) : 59-74. doi: 10.3934/jdg.2017004
References:
[1]

I. Achdou and I. Capuzzo-Dolcetta, Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.  doi: 10.1137/090758477.  Google Scholar

[2]

M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877.   Google Scholar

[3]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[4]

D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987.  Google Scholar

[5]

A. Budhiraja, P. Dupuis and M. Fischer, Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.  doi: 10.1214/10-AOP616.  Google Scholar

[6]

R. Carmona, J. P. Fouque and D. Vestal, Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.  doi: 10.1007/s00780-009-0098-8.  Google Scholar

[7]

S. P. Coraluppi and S. I. Marcus, Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.  doi: 10.1109/9.847737.  Google Scholar

[8] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.   Google Scholar
[9]

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.  doi: 10.1007/s10626-010-0094-3.  Google Scholar

[10]

N. Gast, N. B. Gaujal and J. Y. Le Boudec, Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.  doi: 10.1109/TAC.2012.2186176.  Google Scholar

[11]

D. A. Gomes, J. Mohr and R. R. Souza, Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.  doi: 10.1016/j.matpur.2009.10.010.  Google Scholar

[12]

T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.  doi: 10.1137/S0363012901383837.  Google Scholar

[13]

C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa, Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.  doi: 10.1007/s00245-015-9312-6.  Google Scholar

[14]

M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.  doi: 10.1137/080735370.  Google Scholar

[15]

H. Jasso-Fuentes and J. D. López-Barrientos, On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.  doi: 10.1080/00207179.2014.984764.  Google Scholar

[16]

V. N. Kolokoltsov, M. Troeva and W. Yang, On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.  doi: 10.1007/s13235-013-0095-6.  Google Scholar

[17]

M. Kurano, Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247.   Google Scholar

[18]

A. Lachapelle, J. Salomon and G. Turinici, Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.  doi: 10.1142/S0218202510004349.  Google Scholar

[19]

J. M. Lasry and P. L. Lions, Mean field games, Jap. J.Math., 2 (2007), 229-260.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[20]

J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007). doi: 10.1109/QEST.2007.8.  Google Scholar

[21]

J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo, Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.  doi: 10.1007/s11750-014-0323-2.  Google Scholar

[22]

F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas, Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.  doi: 10.1007/s10440-011-9605-y.  Google Scholar

[23]

N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam. Google Scholar

[24]

A. Schield, Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.  doi: 10.1007/s00186-007-0172-y.  Google Scholar

show all references

References:
[1]

I. Achdou and I. Capuzzo-Dolcetta, Mean field games: Numerical methods, SIAM J. Numer. Anal., 48 (2010), 1136-1162.  doi: 10.1137/090758477.  Google Scholar

[2]

M. Aoki, New macroeconomic modeling approaches.Hierarchical dynamics and mean field approximation, J. Econ. Dyn. Control, 18 (1994), 865-877.   Google Scholar

[3]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Control Theory, Springer Briefs in Mathematics, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[4]

D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N. J., 1987.  Google Scholar

[5]

A. Budhiraja, P. Dupuis and M. Fischer, Large deviation properties of weakly interacting processes via weak convergence methods, Annals Probab., 40 (2012), 74-102.  doi: 10.1214/10-AOP616.  Google Scholar

[6]

R. Carmona, J. P. Fouque and D. Vestal, Interacting particle systems for the computation of rare credit portfolio losses, Finance Stoch., 13 (2009), 613-633.  doi: 10.1007/s00780-009-0098-8.  Google Scholar

[7]

S. P. Coraluppi and S. I. Marcus, Mixed risk-neutral/minimax control of discrete-time finite state Markov decision processes, IEEE Trans. Autom. Control, 45 (2000), 528-532.  doi: 10.1109/9.847737.  Google Scholar

[8] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer-Verlag, New York, 1979.   Google Scholar
[9]

N. Gast and B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dyn. Syst., 21 (2011), 63-101.  doi: 10.1007/s10626-010-0094-3.  Google Scholar

[10]

N. Gast, N. B. Gaujal and J. Y. Le Boudec, Mean field for Markov decision processes: From discrete to continuous optimization, IEEE Trans. Autom. Control, 57 (2012), 2266-2280.  doi: 10.1109/TAC.2012.2186176.  Google Scholar

[11]

D. A. Gomes, J. Mohr and R. R. Souza, Discrete time, finite state space mean field games, J. Math. Pures Appl., 93 (2010), 308-328.  doi: 10.1016/j.matpur.2009.10.010.  Google Scholar

[12]

T. J. González-Trejo, O. Hernández-Lerma and L. F. Hoyos-Reyes, Minimax control of discrete-time stochastic systems, SIAM J. Control Optim., 41 (2002), 1626-1659.  doi: 10.1137/S0363012901383837.  Google Scholar

[13]

C. G. Higuera-Chan, H. Jasso-Fuentes and J. A. Minjárez-Sosa, Discrete-time control for systems of interacting objects with unknown random disturbance distributions: a mean field approach, Appl. Math. Optim., 74 (2016), 197-227.  doi: 10.1007/s00245-015-9312-6.  Google Scholar

[14]

M. Huang, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM J. Control Optim., 48 (2010), 3318-3353.  doi: 10.1137/080735370.  Google Scholar

[15]

H. Jasso-Fuentes and J. D. López-Barrientos, On the use of stochastic differential games against nature to ergodic control problems with unknown parameters, Internat. J. Control, 88 (2015), 897-909.  doi: 10.1080/00207179.2014.984764.  Google Scholar

[16]

V. N. Kolokoltsov, M. Troeva and W. Yang, On the rate of convergence for the mean-field approximation of controlled difusions with large number of players, Dyn. Games Appl., 4 (2014), 208-230.  doi: 10.1007/s13235-013-0095-6.  Google Scholar

[17]

M. Kurano, Minimax strategies for averange cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Jpn., 30 (1987), 232-247.   Google Scholar

[18]

A. Lachapelle, J. Salomon and G. Turinici, Computation of mean field equilibria in economics, Math. Models Meth. Appl. Sci., 20 (2010), 567-588.  doi: 10.1142/S0218202510004349.  Google Scholar

[19]

J. M. Lasry and P. L. Lions, Mean field games, Jap. J.Math., 2 (2007), 229-260.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[20]

J. Y. Le Boudec, D. McDonald and J. Mundinger. A generic mean field convergence result for systems of interacting objects, 4th Int. Conf. Quantitative Evaluation of Systems, (2007). doi: 10.1109/QEST.2007.8.  Google Scholar

[21]

J. D. López-Barrientos, H. Jasso-Fuentes and B. A. Escobedo Trujillo, Discounted robust control for Markov diffusion processes, TOP., 23 (2015), 53-76.  doi: 10.1007/s11750-014-0323-2.  Google Scholar

[22]

F. Luque-Vásquez, J. A. Minjárez-Sosa and L. C. Rosas-Rosas, Semi-Markov control models with partially known holding times distribution: Discounted and Average criteria, Appl. Math., 114 (2011), 135-156.  doi: 10.1007/s10440-011-9605-y.  Google Scholar

[23]

N. Peyrard and R. Sabbbadin, Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes, Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds. ) Frontiers in Artificial Intelligence and Applications, (2006), 595-599. IOS Press, Amsterdam. Google Scholar

[24]

A. Schield, Robust optimal control for a consumption-investment problem, Math. Methods Oper. Res., 67 (2008), 1-20.  doi: 10.1007/s00186-007-0172-y.  Google Scholar

[1]

Laura Aquilanti, Simone Cacace, Fabio Camilli, Raul De Maio. A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions. Journal of Dynamics & Games, 2020  doi: 10.3934/jdg.2020033
[2]

Xuefeng Zhang, Yingbo Zhang. Fault-tolerant control against actuator failures for uncertain singular fractional order systems. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 1-12. doi: 10.3934/naco.2020011
[3]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[4]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046
[5]

Illés Horváth, Kristóf Attila Horváth, Péter Kovács, Miklós Telek. Mean-field analysis of a scaling MAC radio protocol. Journal of Industrial & Management Optimization, 2021, 17 (1) : 279-297. doi: 10.3934/jimo.2019111
[6]

Theresa Lange, Wilhelm Stannat. Mean field limit of ensemble square root filters - discrete and continuous time. Foundations of Data Science, 2021  doi: 10.3934/fods.2021003
[7]

Felix Finster, Jürg Fröhlich, Marco Oppio, Claudio F. Paganini. Causal fermion systems and the ETH approach to quantum theory. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020451
[8]

Shin-Ichiro Ei, Hiroshi Ishii. The motion of weakly interacting localized patterns for reaction-diffusion systems with nonlocal effect. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 173-190. doi: 10.3934/dcdsb.2020329
[9]

Daniele Bartolucci, Changfeng Gui, Yeyao Hu, Aleks Jevnikar, Wen Yang. Mean field equations on tori: Existence and uniqueness of evenly symmetric blow-up solutions. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3093-3116. doi: 10.3934/dcds.2020039
[10]

Alain Bensoussan, Xinwei Feng, Jianhui Huang. Linear-quadratic-Gaussian mean-field-game with partial observation and common noise. Mathematical Control & Related Fields, 2021, 11 (1) : 23-46. doi: 10.3934/mcrf.2020025
[11]

Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, 2021, 20 (1) : 449-465. doi: 10.3934/cpaa.2020276
[12]

Jann-Long Chern, Sze-Guang Yang, Zhi-You Chen, Chih-Her Chen. On the family of non-topological solutions for the elliptic system arising from a product Abelian gauge field theory. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3291-3304. doi: 10.3934/dcds.2020127
[13]

Jian Zhang, Tony T. Lee, Tong Ye, Liang Huang. An approximate mean queue length formula for queueing systems with varying service rate. Journal of Industrial & Management Optimization, 2021, 17 (1) : 185-204. doi: 10.3934/jimo.2019106
[14]

Sergey Rashkovskiy. Hamilton-Jacobi theory for Hamiltonian and non-Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 563-583. doi: 10.3934/jgm.2020024
[15]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444
[16]

Duy Phan, Lassi Paunonen. Finite-dimensional controllers for robust regulation of boundary control systems. Mathematical Control & Related Fields, 2021, 11 (1) : 95-117. doi: 10.3934/mcrf.2020029
[17]

Hui Lv, Xing'an Wang. Dissipative control for uncertain singular markovian jump systems via hybrid impulsive control. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 127-142. doi: 10.3934/naco.2020020
[18]

Guillaume Wafo-Tapa, Slim Bettaieb, Loïc Bidoux, Philippe Gaborit, Etienne Marcatel. A practicable timing attack against HQC and its countermeasure. Advances in Mathematics of Communications, 2020  doi: 10.3934/amc.2020126
[19]

Pierre Baras. A generalization of a criterion for the existence of solutions to semilinear elliptic equations. Discrete & Continuous Dynamical Systems - S, 2021, 14 (2) : 465-504. doi: 10.3934/dcdss.2020439
[20]

Pierluigi Colli, Gianni Gilardi, Jürgen Sprekels. Deep quench approximation and optimal control of general Cahn–Hilliard systems with fractional operators and double obstacle potentials. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 243-271. doi: 10.3934/dcdss.2020213

 Impact Factor: 

Tools

Metrics

  • PDF downloads (90)
  • HTML views (134)
  • Cited by (4)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences