American Institute of Mathematical Sciences

Advanced
January  2016, 3(1): 25-50. doi: 10.3934/jdg.2016002

Discount-sensitive equilibria in zero-sum stochastic differential games

Beatris A. Escobedo-Trujillo 1,

1. 

Engineering Faculty, Universidad Veracruzana, Coatzacoalcos, Ver. 96538, Mexico

Received  April 2015 Revised  November 2015 Published  March 2016

We consider infinite-horizon zero-sum stochastic differential games with average payoff criteria, discount -sensitive criteria and, infinite-horizon undiscounted reward criteria which are sensitive to the growth rate of finite-horizon payoffs. These criteria include, average reward optimality, strong 0-discount optimality, strong -1-discount optimality, 0-discount optimality, bias optimality, F-strong average optimality and overtaking optimality. The main objective is to give conditions under which these criteria are interrelated.
Keywords: strong average., Additive structure, sensitive equilibria, Nash equlibrium, $m-$ discount optimality.
Mathematics Subject Classification: Primary: 91A10, 91A15, 91A2.
Citation: Beatris A. Escobedo-Trujillo. Discount-sensitive equilibria in zero-sum stochastic differential games. Journal of Dynamics & Games, 2016, 3 (1) : 25-50. doi: 10.3934/jdg.2016002
References:
[1]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control,, SIAM J. Control Optim., 48 (2010), 4181.  doi: 10.1137/090762464.  Google Scholar

[2]

A. Arapostathis, M. Ghosh and V. Borkar, Ergodic Control of Diffusion Processes,, Vol. 143, (2012).   Google Scholar

[3]

A. Arapostathis, V. Borkar and K. Surech, Relative value iteration for stochastic differential games,, , 13 (2013), 3.  doi: 10.1007/978-3-319-02690-9_1.  Google Scholar

[4]

M. Bardi, Explicit solutions of some linear-quadratic mean field games. Networks and heterogeneous media,, American Institute of Mathematical Sciences, 7 (2012), 243.  doi: 10.3934/nhm.2012.7.243.  Google Scholar

[5]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach,, J. Optim. Theory Appl., 73 (1992), 359.  doi: 10.1007/BF00940187.  Google Scholar

[6]

R. Cavazos-Cadena and J. B. Lasserre, Strong 1-optimal stationary policies in denumerable Markov decision processes,, Syst. Control Lett., 11 (1988), 65.  doi: 10.1016/0167-6911(88)90113-2.  Google Scholar

[7]

B. Escobedo-Trujillo, D. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games,, J. Optim. Theory Appl., 153 (2012), 662.  doi: 10.1007/s10957-011-9974-4.  Google Scholar

[8]

B. Escobedo-Trujillo and J. López-Barrientos, Nonzero-sum stochastic differential games with additive structure and average payoffs,, Journal of Dynamics and Games, 1 (2014), 555.  doi: 10.3934/jdg.2014.1.555.  Google Scholar

[9]

K. Fan, Fixed-point and minimax theorems in locally convex linear spaces,, Proc. N.A.S.U.S.A., 38 (1952), 121.  doi: 10.1073/pnas.38.2.121.  Google Scholar

[10]

A. Friedman, Stochastic Differential Equations and Applications,, Vol. 1, (1975).   Google Scholar

[11]

J. Flynn, On optimality criteria for dynamic programs with long finite horizons,, J. Math. Anal. Appl., 76 (1980), 202.  doi: 10.1016/0022-247X(80)90072-4.  Google Scholar

[12]

M. Ghosh, A. Arapostathis and S. Marcus, Ergodic control of switching diffusions,, SIAM J. Control Optim., 35 (1997), 1952.  doi: 10.1137/S0363012996299302.  Google Scholar

[13]

O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality,, Appl. Math. (Warsaw), 25 (1998), 153.   Google Scholar

[14]

O. Hernández-Lerma and J. Lasserre, Further Topics on Discrete-Time Markov Control Processes,, Springer, (1999).  doi: 10.1007/978-1-4612-0561-6.  Google Scholar

[15]

N. Hilgert and O. Hernández-Lerma, Bias optimality versus strong 0-discount optimality in Markov control processes with unbounded costs,, Acta Applicandae Mathematicae, 77 (2003), 215.  doi: 10.1023/A:1024996308133.  Google Scholar

[16]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes,, Appl. Math. Optim., 57 (2008), 349.  doi: 10.1007/s00245-007-9025-6.  Google Scholar

[17]

H. Jasso-Fuentes and O. Hernández-Lerma, Ergodic control, bias, and sensitive discount optimality for Markov diffusion processes,, Stochatic Analysis and Applications, 27 (2009), 363.  doi: 10.1080/07362990802679034.  Google Scholar

[18]

H. Jasso-Fuentes, J. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure,, IMA J. Math. Control Inform. doi: 10.1093/imamci/dnv045, (2015).   Google Scholar

[19]

S. Meyn and R. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time precesses,, Adv. Appl. Prob., 25 (1993), 518.  doi: 10.2307/1427522.  Google Scholar

[20]

H. Morimoto and M. Ohashi, On linear stochastic differential games with average cost criterions,, J. Optim. Theory Appl., 64 (1990), 127.  doi: 10.1007/BF00940027.  Google Scholar

[21]

A. Nowak, Sensitive equilibria for ergodic stochastic games with countable state spaces,, Math. Meth. Oper. Res., 50 (1999), 65.   Google Scholar

[22]

A. Nowak, Optimal strategies in a class of zero-sum ergodic stochastic games,, Math. Meth. Oper. Res., 50 (1999), 399.  doi: 10.1007/s001860050078.  Google Scholar

[23]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure,, Math. Meth. Oper. Res., 64 (2006), 481.  doi: 10.1007/s00186-006-0090-4.  Google Scholar

[24]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games,, Math. Meth. Oper. Res., 61 (2005), 437.  doi: 10.1007/s001860400392.  Google Scholar

[25]

T. Prieto-Rumeau and O. Hernández-Lerma, The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains,, Math. Meth. Oper. Res., 61 (2005), 123.  doi: 10.1007/s001860400393.  Google Scholar

[26]

M. Puterman, Sensitive discount optimality in controlled one-dimensional diffusions,, Annals of Probability, 2 (1974), 408.  doi: 10.1214/aop/1176996656.  Google Scholar

[27]

W. Qingda and C. Xian, Strong average optimality criterion for continuos-time Markov decision processes,, Kybernetika, 50 (2014), 950.   Google Scholar

[28]

M. Sion, On general minimax theorems,, Pacific J. Math., 8 (1958), 171.  doi: 10.2140/pjm.1958.8.171.  Google Scholar

[29]

W. Schmitendorf, Differential games without pure strategy saddle-point solutions,, J. Optim. Theory Appl., 18 (1976), 81.  doi: 10.1007/BF00933796.  Google Scholar

[30]

Q. Zhu and X. Guo, Another set of conditions for strong $n$ ($n=-1,0$) discount optimality in Markov decision processes,, Stochastic Analysis and Applications, 23 (2005), 953.  doi: 10.1080/07362990500184865.  Google Scholar

[31]

Q. Zhu, Bias optimality and strong $n$ ($n=-1,0$) discount optimality for Markov decision processes,, J. Math. Anal. Appl., 334 (2007), 576.  doi: 10.1016/j.jmaa.2007.01.002.  Google Scholar

show all references

References:
[1]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control,, SIAM J. Control Optim., 48 (2010), 4181.  doi: 10.1137/090762464.  Google Scholar

[2]

A. Arapostathis, M. Ghosh and V. Borkar, Ergodic Control of Diffusion Processes,, Vol. 143, (2012).   Google Scholar

[3]

A. Arapostathis, V. Borkar and K. Surech, Relative value iteration for stochastic differential games,, , 13 (2013), 3.  doi: 10.1007/978-3-319-02690-9_1.  Google Scholar

[4]

M. Bardi, Explicit solutions of some linear-quadratic mean field games. Networks and heterogeneous media,, American Institute of Mathematical Sciences, 7 (2012), 243.  doi: 10.3934/nhm.2012.7.243.  Google Scholar

[5]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach,, J. Optim. Theory Appl., 73 (1992), 359.  doi: 10.1007/BF00940187.  Google Scholar

[6]

R. Cavazos-Cadena and J. B. Lasserre, Strong 1-optimal stationary policies in denumerable Markov decision processes,, Syst. Control Lett., 11 (1988), 65.  doi: 10.1016/0167-6911(88)90113-2.  Google Scholar

[7]

B. Escobedo-Trujillo, D. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games,, J. Optim. Theory Appl., 153 (2012), 662.  doi: 10.1007/s10957-011-9974-4.  Google Scholar

[8]

B. Escobedo-Trujillo and J. López-Barrientos, Nonzero-sum stochastic differential games with additive structure and average payoffs,, Journal of Dynamics and Games, 1 (2014), 555.  doi: 10.3934/jdg.2014.1.555.  Google Scholar

[9]

K. Fan, Fixed-point and minimax theorems in locally convex linear spaces,, Proc. N.A.S.U.S.A., 38 (1952), 121.  doi: 10.1073/pnas.38.2.121.  Google Scholar

[10]

A. Friedman, Stochastic Differential Equations and Applications,, Vol. 1, (1975).   Google Scholar

[11]

J. Flynn, On optimality criteria for dynamic programs with long finite horizons,, J. Math. Anal. Appl., 76 (1980), 202.  doi: 10.1016/0022-247X(80)90072-4.  Google Scholar

[12]

M. Ghosh, A. Arapostathis and S. Marcus, Ergodic control of switching diffusions,, SIAM J. Control Optim., 35 (1997), 1952.  doi: 10.1137/S0363012996299302.  Google Scholar

[13]

O. Hernández-Lerma and O. Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality,, Appl. Math. (Warsaw), 25 (1998), 153.   Google Scholar

[14]

O. Hernández-Lerma and J. Lasserre, Further Topics on Discrete-Time Markov Control Processes,, Springer, (1999).  doi: 10.1007/978-1-4612-0561-6.  Google Scholar

[15]

N. Hilgert and O. Hernández-Lerma, Bias optimality versus strong 0-discount optimality in Markov control processes with unbounded costs,, Acta Applicandae Mathematicae, 77 (2003), 215.  doi: 10.1023/A:1024996308133.  Google Scholar

[16]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes,, Appl. Math. Optim., 57 (2008), 349.  doi: 10.1007/s00245-007-9025-6.  Google Scholar

[17]

H. Jasso-Fuentes and O. Hernández-Lerma, Ergodic control, bias, and sensitive discount optimality for Markov diffusion processes,, Stochatic Analysis and Applications, 27 (2009), 363.  doi: 10.1080/07362990802679034.  Google Scholar

[18]

H. Jasso-Fuentes, J. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure,, IMA J. Math. Control Inform. doi: 10.1093/imamci/dnv045, (2015).   Google Scholar

[19]

S. Meyn and R. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time precesses,, Adv. Appl. Prob., 25 (1993), 518.  doi: 10.2307/1427522.  Google Scholar

[20]

H. Morimoto and M. Ohashi, On linear stochastic differential games with average cost criterions,, J. Optim. Theory Appl., 64 (1990), 127.  doi: 10.1007/BF00940027.  Google Scholar

[21]

A. Nowak, Sensitive equilibria for ergodic stochastic games with countable state spaces,, Math. Meth. Oper. Res., 50 (1999), 65.   Google Scholar

[22]

A. Nowak, Optimal strategies in a class of zero-sum ergodic stochastic games,, Math. Meth. Oper. Res., 50 (1999), 399.  doi: 10.1007/s001860050078.  Google Scholar

[23]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure,, Math. Meth. Oper. Res., 64 (2006), 481.  doi: 10.1007/s00186-006-0090-4.  Google Scholar

[24]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games,, Math. Meth. Oper. Res., 61 (2005), 437.  doi: 10.1007/s001860400392.  Google Scholar

[25]

T. Prieto-Rumeau and O. Hernández-Lerma, The Laurent series, sensitive discount and Blackwell optimality for continuous-time controlled Markov chains,, Math. Meth. Oper. Res., 61 (2005), 123.  doi: 10.1007/s001860400393.  Google Scholar

[26]

M. Puterman, Sensitive discount optimality in controlled one-dimensional diffusions,, Annals of Probability, 2 (1974), 408.  doi: 10.1214/aop/1176996656.  Google Scholar

[27]

W. Qingda and C. Xian, Strong average optimality criterion for continuos-time Markov decision processes,, Kybernetika, 50 (2014), 950.   Google Scholar

[28]

M. Sion, On general minimax theorems,, Pacific J. Math., 8 (1958), 171.  doi: 10.2140/pjm.1958.8.171.  Google Scholar

[29]

W. Schmitendorf, Differential games without pure strategy saddle-point solutions,, J. Optim. Theory Appl., 18 (1976), 81.  doi: 10.1007/BF00933796.  Google Scholar

[30]

Q. Zhu and X. Guo, Another set of conditions for strong $n$ ($n=-1,0$) discount optimality in Markov decision processes,, Stochastic Analysis and Applications, 23 (2005), 953.  doi: 10.1080/07362990500184865.  Google Scholar

[31]

Q. Zhu, Bias optimality and strong $n$ ($n=-1,0$) discount optimality for Markov decision processes,, J. Math. Anal. Appl., 334 (2007), 576.  doi: 10.1016/j.jmaa.2007.01.002.  Google Scholar

[1]

Feimin Zhong, Jinxing Xie, Yuwei Shen. Bargaining in a multi-echelon supply chain with power structure: KS solution vs. Nash solution. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020172
[2]

Shengxin Zhu, Tongxiang Gu, Xingping Liu. AIMS: Average information matrix splitting. Mathematical Foundations of Computing, 2020, 3 (4) : 301-308. doi: 10.3934/mfc.2020012
[3]

Elvio Accinelli, Humberto Muñiz. A dynamic for production economies with multiple equilibria. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021002
[4]

Thierry Horsin, Mohamed Ali Jendoubi. On the convergence to equilibria of a sequence defined by an implicit scheme. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020465
[5]

Manil T. Mohan. First order necessary conditions of optimality for the two dimensional tidal dynamics system. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020045
[6]

Vaibhav Mehandiratta, Mani Mehra, Günter Leugering. Fractional optimal control problems on a star graph: Optimality system and numerical solution. Mathematical Control & Related Fields, 2021, 11 (1) : 189-209. doi: 10.3934/mcrf.2020033
[7]

Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020, 2 (4) : 351-390. doi: 10.3934/fods.2020017
[8]

Wen Huang, Jianya Liu, Ke Wang. Möbius disjointness for skew products on a circle and a nilmanifold. Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021006
[9]

Xiyou Cheng, Zhitao Zhang. Structure of positive solutions to a class of Schrödinger systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020461
[10]

José Luiz Boldrini, Jonathan Bravo-Olivares, Eduardo Notte-Cuello, Marko A. Rojas-Medar. Asymptotic behavior of weak and strong solutions of the magnetohydrodynamic equations. Electronic Research Archive, 2021, 29 (1) : 1783-1801. doi: 10.3934/era.2020091
[11]

Philipp Harms. Strong convergence rates for markovian representations of fractional processes. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020367
[12]

Biyue Chen, Chunxiang Zhao, Chengkui Zhong. The global attractor for the wave equation with nonlocal strong damping. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021015
[13]

Yu Zhou, Xinfeng Dong, Yongzhuang Wei, Fengrong Zhang. A note on the Signal-to-noise ratio of $ (n, m) $-functions. Advances in Mathematics of Communications, 2020  doi: 10.3934/amc.2020117
[14]

Hongwei Liu, Jingge Liu. On $ \sigma $-self-orthogonal constacyclic codes over $ \mathbb F_{p^m}+u\mathbb F_{p^m} $. Advances in Mathematics of Communications, 2020  doi: 10.3934/amc.2020127
[15]

Martin Kalousek, Joshua Kortum, Anja Schlömerkemper. Mathematical analysis of weak and strong solutions to an evolutionary model for magnetoviscoelasticity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 17-39. doi: 10.3934/dcdss.2020331
[16]

Toshiko Ogiwara, Danielle Hilhorst, Hiroshi Matano. Convergence and structure theorems for order-preserving dynamical systems with mass conservation. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3883-3907. doi: 10.3934/dcds.2020129
[17]

Adrian Viorel, Cristian D. Alecsa, Titus O. Pinţa. Asymptotic analysis of a structure-preserving integrator for damped Hamiltonian systems. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020407
[18]

Pablo Neme, Jorge Oviedo. A note on the lattice structure for matching markets via linear programming. Journal of Dynamics & Games, 2020  doi: 10.3934/jdg.2021001
[19]

Gheorghe Craciun, Jiaxin Jin, Casian Pantea, Adrian Tudorascu. Convergence to the complex balanced equilibrium for some chemical reaction-diffusion systems with boundary equilibria. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1305-1335. doi: 10.3934/dcdsb.2020164
[20]

Jan Březina, Eduard Feireisl, Antonín Novotný. On convergence to equilibria of flows of compressible viscous fluids under in/out–flux boundary conditions. Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021009

 Impact Factor: 

Tools

Metrics

  • PDF downloads (53)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences