Nonzero-sum stochastic differential games with additive structure and average payoffs

Beatris Adriana Escobedo-Trujillo; José Daniel López-Barrientos

doi:10.3934/jdg.2014.1.555

Article Contents

2014, Volume 1, Issue 4: 555-578. Doi: 10.3934/jdg.2014.1.555

This issue Previous Article Next Article Investment under uncertainty, competition and regulation

Nonzero-sum stochastic differential games with additive structure and average payoffs

1.
Engineering Faculty, Universidad Veracruzana, Coatzacoalcos, Ver. 96538
2.
Faculty of Actuarial Sciences, Universidad Anáhuac, Av. Universidad Anáhuac 46. Col. Lomas Anáhuac, Huixquilucan, Edo. de México

Received: May 31, 2014

Revised: August 31, 2014

Published: September 2014

Abstract Related Papers Cited by

Abstract

This paper deals with nonzero-sum stochastic differential games with an additive structure and long-run average payoffs. Our main objective is to give conditions for the existence of Nash equilibria in the set of relaxed stationary strategies. Such conditions also ensure the existence of a Nash equilibrium within the set of stationary Markov (deterministic) strategies, and that the values of the average payoffs for these equilibria coincide almost everywhere with respect to Lebesgue's measure. This fact generalizes the results in the controlled (single player game) case found by Raghavan [47] and Rosenblueth [48]. We use relaxation theory and standard dynamic programming techniques to achieve our goals. We illustrate our results with an example motivated by a manufacturing system.

Keywords:

Additive structure,
average (or ergodic) payoff criterion,
dynamic programming,
relaxation theory,
nonzero-sum stochastic differential games.

Mathematics Subject Classification: Primary: 91A10, 91A15, 91A25, 60H10.

Citation:

Related Papers

Cited by

References

[1]	R. Adams, Sobolev Spaces, Academic Press, New York, 1975.
[2]	R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control, 31 (1986), 116-126.doi: 10.1109/TAC.1986.1104206.
[3]	A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim, 48 (2010), 4181-4223.doi: 10.1137/090762464.
[4]	E. Balder, Lectures on Young Measures, Cahier de Mathématiques de la Décision, CEREMADE, Université Paris IX-Dauphine, Paris 1995.
[5]	M. Bardi, Explicit solutions of some linear-quadratic mean field games, Netw. Heterog. Media., 7 (2012), 243-261.doi: 10.3934/nhm.2012.7.243.
[6]	M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493-2498.doi: 10.1109/CDC.2013.6760255.
[7]	A. Bhatt and R. Karandikar, Invariant measures and evolution equations for Markov processes, Ann. Probab., 21 (1993), 2246-2268.doi: 10.1214/aop/1176989019.
[8]	P. Billingsley, Probability and Measure, $3^{rd}$ edition, Wiley Ser. Probab. Math. Statist., John Willey & Sons, Inc., New York, 1995.
[9]	V. Bogachev, N. Krylov and M. Röckner, On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions, Comm. PartialDifferential Equations, 26 (2001), 2037-2080.doi: 10.1081/PDE-100107815.
[10]	V. Bogachev, N. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processes, Dokl. Akad. Nauk, 405 (2005), 583-587.
[11]	V. Bogachev, M. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976.doi: 10.1070/SM2002v193n07ABEH000665.
[12]	V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62.doi: 10.1007/BF01447645.
[13]	V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. Correction: 88 (1996), 251-252.doi: 10.1007/BF00940187.
[14]	R. Buckdahn, P. Cardaliaguet and C. Rainer, Nash equilibrium payoffs for nonzero-sum stochastic diferential games, SIAM J. Control Optim., 43 (2004), 624-642.doi: 10.1137/S0363012902411556.
[15]	A. Calderón and J. Rosenblueth, Minimizing Approximate Original Solutions for Commensurate Delayed Controls, Appl. Math. Lett., 7 (1994), 5-10.doi: 10.1016/0893-9659(94)90063-9.
[16]	R. Durrett, Stochastic Calculus: A Practical Introduction, CRC Press, Boca Raton, FLA., 1996.
[17]	A. Dvoretzky, A. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21.doi: 10.1214/aoms/1177729689.
[18]	B. Escobedo-Trujillo, J. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687.doi: 10.1007/s10957-011-9974-4.
[19]	K. Fan, Fixed-point and minimax theorems in locally convex linear spaces, Proc. N.A.S.U.S.A., 38 (1952), 121-126.doi: 10.1073/pnas.38.2.121.
[20]	K. Fan, Minimax theorems, Proc. N.A.S.U.S.A., 39 (1953), 32-47.doi: 10.1073/pnas.39.1.42.
[21]	G. Folland, Real Analysis. Modern Techniques and Their Applications, $2^{nd}$ edition, John Wiley and Sons, New York, 1999.
[22]	A. Friedman, Stochastic Differential Equations and Applications, Vol. 1, Academic Press, New York, 1975.
[23]	M. Ghosh, A. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988.doi: 10.1137/S0363012996299302.
[24]	M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301.doi: 10.1007/s002459900092.
[25]	D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order, Reprinted version, Springer, Heidelberg, 1998.
[26]	P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931.doi: 10.1214/aop/1039639370.
[27]	J. González-Hernández and O. Hernández-Lerma, Extreme points of sets of randomized strategies in constrained optimization and control problems, SIAM J. Optim., 15 (2005), 1085-1104.doi: 10.1137/040605345.
[28]	H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65.
[29]	O. Hernández-Lerma and J. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York, 1996.doi: 10.1007/978-1-4612-0729-0.
[30]	O. Hernández-Lerma and J. Lasserre, Further topics on Discrete-Time Markov Control Processes, Springer, New York, 1999.doi: 10.1007/978-1-4612-0561-6.
[31]	H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369.doi: 10.1007/s00245-007-9025-6.
[32]	F. Klebaner, Introduction to Stochastic Calculus with Applications, $2^{nd}$ edition, Imperial College Press, London, 2005.doi: 10.1142/p386.
[33]	A. Knapp, Advanced Real Analysis, $2^{nd}$ edition, Birkäuser, New York, 2005.
[34]	A. Kolmogorov and S. Fomin, Elements of the Theory of Functions and Functional Analysis, Dover Publications, 1999.
[35]	N. Krylov, Controlled diffusion processes. Applications of Mathematics, Springer-Verlag, New York, 1980.
[36]	H. Kushner, Numerical approximations for nonzero-sum stochastic differential games, SIAM J.Control Optim., 46 (2007), 1942-1971.doi: 10.1137/050647931.
[37]	H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147.doi: 10.1142/S0219198999000098.
[38]	S. Meyn and R. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548.doi: 10.2307/1427522.
[39]	H. Morimoto and M. Ohashi, On linear stochastic differential games with average cost criterions, J. Optimization Theory and Applications, 64 (1990), 127-140.doi: 10.1007/BF00940027.
[40]	J. Munkres, Topology, $2^{nd}$ edition, Prentice Hall Inc., 2000.
[41]	A. Nowak, Stationary equilibria for nonzero-sum average payoff ergodic stochastic games with general state space, Advances in Dynamic Games and Applications (T. Basar and A. Haurie, eds.), Birkhäuser, 1 (1994), 231-246.
[42]	A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494.doi: 10.1007/s00186-006-0090-4.
[43]	P. Pedregal, Optimization, relaxation and Young measures, Bull. Amer. Math. Soc. (N.S.), 36 (1999), 27-58.doi: 10.1090/S0273-0979-99-00774-0.
[44]	B. Oksendal, Stochastic Differential Equations: An Introduction with Applications, $4^{th}$ edition, Springer, New York, 1995.doi: 10.1007/978-3-662-03185-8.
[45]	K. Parthasarathy and T. Steerneman, A tool in establishing total variation convergence, Proceedings of the American Mathematical Society, 95 (1985), 626-630.doi: 10.1090/S0002-9939-1985-0810175-X.
[46]	T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454.doi: 10.1007/s001860400392.
[47]	T. Raghavan, S. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464.doi: 10.1007/BF00942191.
[48]	J. Rosenblueth, Proper relaxation of optimal control problem, J. Optim. Theory Appl., 74 (1992), 509-526.doi: 10.1007/BF00940324.
[49]	N. Saldi, T. Linder and S. Yüksel, Asymptotic optimality of quantized control in Markov decision processes. Submitted to IEEE Conference on Decision and Control.
[50]	M. Schäl, Conditions for optimality and for the limit of $n-$stage optimal policies to be optimal, Z. Wahrs. Verw. Gerb., 32 (1975), 179-196.doi: 10.1007/BF00532612.
[51]	L. Young, Lectures on the Calculus of Variations and Optimal Control Theory, W. B. Saunders, Philadelphia, PA, 1969.
[52]	J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972.
[53]	E. Zeidler, Nonlinear Functional Analysis and its Applications II/A, Springer-Verlag, New York, 1990.doi: 10.1007/978-1-4612-0985-0.