April  2016, 36(4): 2113-2132. doi: 10.3934/dcds.2016.36.2113

Limit value for optimal control with general means

1. 

Sorbonne Universités, UPMC Univ Paris 06, IMJ-PRG, UMR 7586, CNRS, Univ Paris Diderot, Sorbonne Paris Cité, 4 Place Jussieu, 75005 Paris, France

2. 

Laboratoire de Mathématiques de Bretagne Atlantique, UMR 6205, Université de Brest, 6 Avenue Victor Le Gorgeu, 29200 Brest, France

3. 

TSE (GREMAQ, Université Toulouse 1 Capitole and GDR 2932 Théorie des Jeux), 21 allée de Brienne, 31000 Toulouse

Received  March 2015 Revised  July 2015 Published  September 2015

We consider optimal control problems where the running cost of the trajectory is evaluated by a probability measure on $\mathbb{R}_+$. As a particular case, we take the Cesàro average of the running cost over a fixed horizon. The limit of the value with Cesàro average when the horizon tends to infinity is widely studied in the literature. We address the more general question of the existence of a limit for values defined by general evaluations satisfying certain long-term condition.
    For this aim, we introduce an asymptotic regularity condition for a sequence of probability measures on $\mathbb{R}_+$. Our main result is that, for any sequence of probability measures on $\mathbb{R}_+$ satisfying this condition, the associated value functions converge uniformly if and only if this family is totally bounded for the uniform norm.
    As a byproduct, we obtain the existence of a limit value (for general evaluations) for control systems defined on a compact invariant domain and satisfying suitable nonexpansive property.
Citation: Xiaoxi Li, Marc Quincampoix, Jérôme Renault. Limit value for optimal control with general means. Discrete & Continuous Dynamical Systems - A, 2016, 36 (4) : 2113-2132. doi: 10.3934/dcds.2016.36.2113
References:
[1]

O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations,, Mem. Amer. Math. Soc., 204 (2010). doi: 10.1090/S0065-9266-09-00588-2.

[2]

M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations,, Ann. Henri Poincaré, 14 (1997), 415. doi: 10.1016/S0294-1449(97)80134-5.

[3]

M. Arisawa and P. L. Lions, On ergodic stochastic control,, Comm. Partial Differential Equations, 23 (1998), 2187. doi: 10.1080/03605309808821413.

[4]

M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost,, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493. doi: 10.1109/CDC.2013.6760255.

[5]

A. Bensoussan, Perturbation Methods in Optimal Control,, Wiley/Gauthiers-Villas, (1988).

[6]

R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems,, Applied Mathematics and Optimization, 70 (2014), 1. doi: 10.1007/s00245-013-9230-4.

[7]

V. Gaitsgory, On the use of the averaging method in control problems,, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876.

[8]

D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics,, preprint, ().

[9]

R. Z. Khasminskii, On the averaging principle for Itô stochastic equations,, Kybernetika, 4 (1968), 260.

[10]

D. Khlopin, On uniform Tauberian theorems for dynamic games,, preprint, ().

[11]

M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control,, in Advances in Dynamic Games, 12 (2012), 199.

[12]

M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems,, SIAM Journal on Control and Optimization, 49 (2011), 2118. doi: 10.1137/090756818.

[13]

J. Renault, Uniform value in dynamic programming,, J. Eur. Math. Soc. (JEMS), 13 (2011), 309. doi: 10.4171/JEMS/254.

[14]

J. Renault, General long-term values in dynamic programming,, Journal of Dynamics and Games, 1 (2014), 471.

[15]

J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games,, preprint, ().

[16]

S. Sorin, A First Course on Zero-sum Repeated Games,, Springer, (2002).

[17]

B. Ziliotto, General limit value in stochastic games,, preprint, ().

show all references

References:
[1]

O. Alvarez and M. Bardi, Ergodicity, stabilization, and singular pertubations for Bellman-Isaacs equations,, Mem. Amer. Math. Soc., 204 (2010). doi: 10.1090/S0065-9266-09-00588-2.

[2]

M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Belmann equations,, Ann. Henri Poincaré, 14 (1997), 415. doi: 10.1016/S0294-1449(97)80134-5.

[3]

M. Arisawa and P. L. Lions, On ergodic stochastic control,, Comm. Partial Differential Equations, 23 (1998), 2187. doi: 10.1080/03605309808821413.

[4]

M. Bardi and F. Priuli, LQG Mean-Field Games with ergodic cost,, Proc. 52nd IEEE Conference on Decision and Control, (2013), 2493. doi: 10.1109/CDC.2013.6760255.

[5]

A. Bensoussan, Perturbation Methods in Optimal Control,, Wiley/Gauthiers-Villas, (1988).

[6]

R. Buckdahn, D. Goreac and M. Quincampoix, Existence of asymptotic values for nonexpansive stochastic control systems,, Applied Mathematics and Optimization, 70 (2014), 1. doi: 10.1007/s00245-013-9230-4.

[7]

V. Gaitsgory, On the use of the averaging method in control problems,, (Russian) Differentsialnye Uravneniya, 22 (1986), 1876.

[8]

D. Goreac, A note on general Tauberian-type results for controlled stochastic dynamics,, preprint, ().

[9]

R. Z. Khasminskii, On the averaging principle for Itô stochastic equations,, Kybernetika, 4 (1968), 260.

[10]

D. Khlopin, On uniform Tauberian theorems for dynamic games,, preprint, ().

[11]

M. Oliu-Barton and G. Vigeral, A uniform Tauberian theorem in optimal control,, in Advances in Dynamic Games, 12 (2012), 199.

[12]

M. Quincampoix and J. Renault, On the existence of a limit value in some nonexpansive optimal control problems,, SIAM Journal on Control and Optimization, 49 (2011), 2118. doi: 10.1137/090756818.

[13]

J. Renault, Uniform value in dynamic programming,, J. Eur. Math. Soc. (JEMS), 13 (2011), 309. doi: 10.4171/JEMS/254.

[14]

J. Renault, General long-term values in dynamic programming,, Journal of Dynamics and Games, 1 (2014), 471.

[15]

J. Renault and X. Venel, A distance for probability spaces, and long-term values in Markov decision processes and repeated games,, preprint, ().

[16]

S. Sorin, A First Course on Zero-sum Repeated Games,, Springer, (2002).

[17]

B. Ziliotto, General limit value in stochastic games,, preprint, ().

[1]

Jérôme Renault. General limit value in dynamic programming. Journal of Dynamics & Games, 2014, 1 (3) : 471-484. doi: 10.3934/jdg.2014.1.471

[2]

Nguyen Huy Chieu, Jen-Chih Yao. Subgradients of the optimal value function in a parametric discrete optimal control problem. Journal of Industrial & Management Optimization, 2010, 6 (2) : 401-410. doi: 10.3934/jimo.2010.6.401

[3]

Pierre Cardaliaguet, Jean-Michel Lasry, Pierre-Louis Lions, Alessio Porretta. Long time average of mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 279-301. doi: 10.3934/nhm.2012.7.279

[4]

K. F. Cedric Yiu, S. Y. Wang, K. L. Mak. Optimal portfolios under a value-at-risk constraint with applications to inventory control in supply chains. Journal of Industrial & Management Optimization, 2008, 4 (1) : 81-94. doi: 10.3934/jimo.2008.4.81

[5]

Brahim El Asri. The value of a minimax problem involving impulse control. Journal of Dynamics & Games, 2019, 6 (1) : 1-17. doi: 10.3934/jdg.2019001

[6]

Jingzhen Liu, Lihua Bai, Ka-Fai Cedric Yiu. Optimal investment with a value-at-risk constraint. Journal of Industrial & Management Optimization, 2012, 8 (3) : 531-547. doi: 10.3934/jimo.2012.8.531

[7]

K. Q. Lan, G. C. Yang. Optimal constants for two point boundary value problems. Conference Publications, 2007, 2007 (Special) : 624-633. doi: 10.3934/proc.2007.2007.624

[8]

M.J. Lopez-Herrero. The existence of weak solutions for a general class of mixed boundary value problems. Conference Publications, 2011, 2011 (Special) : 1015-1024. doi: 10.3934/proc.2011.2011.1015

[9]

Andrei V. Dmitruk, Nikolai P. Osmolovski. Necessary conditions for a weak minimum in a general optimal control problem with integral equations on a variable time interval. Mathematical Control & Related Fields, 2017, 7 (4) : 507-535. doi: 10.3934/mcrf.2017019

[10]

Mark I. Vishik, Sergey Zelik. Attractors for the nonlinear elliptic boundary value problems and their parabolic singular limit. Communications on Pure & Applied Analysis, 2014, 13 (5) : 2059-2093. doi: 10.3934/cpaa.2014.13.2059

[11]

Francis Clarke. A general theorem on necessary conditions in optimal control. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 485-503. doi: 10.3934/dcds.2011.29.485

[12]

Yihong Du, Yoshio Yamada. On the long-time limit of positive solutions to the degenerate logistic equation. Discrete & Continuous Dynamical Systems - A, 2009, 25 (1) : 123-132. doi: 10.3934/dcds.2009.25.123

[13]

Ariela Briani, Hasnaa Zidani. Characterization of the value function of final state constrained control problems with BV trajectories. Communications on Pure & Applied Analysis, 2011, 10 (6) : 1567-1587. doi: 10.3934/cpaa.2011.10.1567

[14]

Gongpin Cheng, Lin Xu. Optimal size of business and dividend strategy in a nonlinear model with refinancing and liquidation value. Mathematical Control & Related Fields, 2017, 7 (1) : 1-19. doi: 10.3934/mcrf.2017001

[15]

Carlo Sinestrari. Semiconcavity of the value function for exit time problems with nonsmooth target. Communications on Pure & Applied Analysis, 2004, 3 (4) : 757-774. doi: 10.3934/cpaa.2004.3.757

[16]

Joachim Escher, Christina Lienstromberg. A survey on second order free boundary value problems modelling MEMS with general permittivity profile. Discrete & Continuous Dynamical Systems - S, 2017, 10 (4) : 745-771. doi: 10.3934/dcdss.2017038

[17]

Roberto A. Capistrano-Filho, Shuming Sun, Bing-Yu Zhang. General boundary value problems of the Korteweg-de Vries equation on a bounded domain. Mathematical Control & Related Fields, 2018, 8 (3&4) : 583-605. doi: 10.3934/mcrf.2018024

[18]

Elena Rossi. Well-posedness of general 1D initial boundary value problems for scalar balance laws. Discrete & Continuous Dynamical Systems - A, 2019, 39 (6) : 3577-3608. doi: 10.3934/dcds.2019147

[19]

Liran Rotem. Banach limit in convexity and geometric means for convex bodies. Electronic Research Announcements, 2016, 23: 41-51. doi: 10.3934/era.2016.23.005

[20]

María Teresa V. Martínez-Palacios, Adrián Hernández-Del-Valle, Ambrosio Ortiz-Ramírez. On the pricing of Asian options with geometric average of American type with stochastic interest rate: A stochastic optimal control approach. Journal of Dynamics & Games, 2019, 6 (1) : 53-64. doi: 10.3934/jdg.2019004

2018 Impact Factor: 1.143

Metrics

  • PDF downloads (5)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]