Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Previous Article
Local wellposedness for the critical nonlinear Schrödinger equation on $ \mathbb{T}^3 $
DCDS Home
This Issue
Next Article
Existence and stability of one-peak symmetric stationary solutions for the Schnakenberg model with heterogeneity

May 2019, 39(5): 2785-2805. doi: 10.3934/dcds.2019117

Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Hanxiao Wang ^1, , Jingrui Sun ^2,, and Jiongmin Yong ^3,

1.	School of Mathematical Sciences, Fudan University, Shanghai 200433, China
2.	Department of Mathematics, Southern University of Science and Technology, Shenzhen, Guangdong 518055, China
3.	Department of Mathematics, University of Central Florida, Orlando FL 32816, USA

* Corresponding author: Jingrui Sun

Received June 2018 Published January 2019

Fund Project: The first author is supported in part by the China Scholarship Council, while visiting University of Central Florida. The third author is supported in part by NSF DMS-1812921

Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.

Keywords: Stochastic linear-quadratic optimal control, Riccati equation, open-loop solvability, weak closed-loop solvability, state feedback.

Mathematics Subject Classification: Primary: 93E20; Secondary: 49N10, 49N35.

Citation: Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems - A, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117

References:

[1]	M. Ait Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), 1296-1311. doi: 10.1137/S0363012900371083.
[2]	R. Bellman, I. Glicksberg and O. Gross, Some Aspects of the Mathematical Theory of Control Processes, RAND Corporation, Santa Monica, CA, 1958.
[3]	A. Bensoussan, Lectures on stochstic control, part Ⅰ, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math., Springer-Verlag, Berlin, 972 (1982), 1–62.
[4]	J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), 419-444. doi: 10.1137/0314028.
[5]	S. Chen, X. Li and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), 1685-1702. doi: 10.1137/S0363012996310478.
[6]	S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chin. Ann. Math., 21 B (2000), 323-338. doi: 10.1142/S0252959900000339.
[7]	S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), 21-45. doi: 10.1007/s002450010016.
[8]	S. Chen and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. Ⅱ, SIAM J. Control Optim., 39 (2000), 1065-1081. doi: 10.1137/S0363012998346578.
[9]	M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, 1977.
[10]	R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc., Mat. Mexicana, 5 (1960), 102-119.
[11]	A. M. Letov, The analytical design of control systems, Automat. Remote Control, 22 (1961), 363-372.
[12]	A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Trans. Automat. Control, 44 (1999), 1359-1369. doi: 10.1109/9.774108.
[13]	J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308. doi: 10.1137/15M103532X.
[14]	J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121. doi: 10.1137/140953642.
[15]	S. Tang, General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations, SIAM J. Control Optim., 42 (2003), 53-75. doi: 10.1137/S0363012901387550.
[16]	S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 53 (2015), 1082-1106. doi: 10.1137/140979940.
[17]	W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697. doi: 10.1137/0306044.
[18]	J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

show all references

References:

[1]	M. Ait Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), 1296-1311. doi: 10.1137/S0363012900371083.
[2]	R. Bellman, I. Glicksberg and O. Gross, Some Aspects of the Mathematical Theory of Control Processes, RAND Corporation, Santa Monica, CA, 1958.
[3]	A. Bensoussan, Lectures on stochstic control, part Ⅰ, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math., Springer-Verlag, Berlin, 972 (1982), 1–62.
[4]	J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), 419-444. doi: 10.1137/0314028.
[5]	S. Chen, X. Li and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), 1685-1702. doi: 10.1137/S0363012996310478.
[6]	S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chin. Ann. Math., 21 B (2000), 323-338. doi: 10.1142/S0252959900000339.
[7]	S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), 21-45. doi: 10.1007/s002450010016.
[8]	S. Chen and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. Ⅱ, SIAM J. Control Optim., 39 (2000), 1065-1081. doi: 10.1137/S0363012998346578.
[9]	M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, 1977.
[10]	R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc., Mat. Mexicana, 5 (1960), 102-119.
[11]	A. M. Letov, The analytical design of control systems, Automat. Remote Control, 22 (1961), 363-372.
[12]	A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Trans. Automat. Control, 44 (1999), 1359-1369. doi: 10.1109/9.774108.
[13]	J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308. doi: 10.1137/15M103532X.
[14]	J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121. doi: 10.1137/140953642.
[15]	S. Tang, General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations, SIAM J. Control Optim., 42 (2003), 53-75. doi: 10.1137/S0363012901387550.
[16]	S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 53 (2015), 1082-1106. doi: 10.1137/140979940.
[17]	W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697. doi: 10.1137/0306044.
[18]	J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[1]	Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[2]	Dean A. Carlson. Finding open-loop Nash equilibrium for variational games. Conference Publications, 2005, 2005 (Special) : 153-163. doi: 10.3934/proc.2005.2005.153
[3]	Pasquale Palumbo, Pierdomenico Pepe, Simona Panunzi, Andrea De Gaetano. Robust closed-loop control of plasma glycemia: A discrete-delay model approach. Discrete & Continuous Dynamical Systems - B, 2009, 12 (2) : 455-468. doi: 10.3934/dcdsb.2009.12.455
[4]	Filippo Cacace, Valerio Cusimano, Alfredo Germani, Pasquale Palumbo, Federico Papa. Closed-loop control of tumor growth by means of anti-angiogenic administration. Mathematical Biosciences & Engineering, 2018, 15 (4) : 827-839. doi: 10.3934/mbe.2018037
[5]	Xiaochen Sun, Fei Hu, Yancong Zhou, Cheng-Chew Lim. Optimal acquisition, inventory and production decisions for a closed-loop manufacturing system with legislation constraint. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1355-1373. doi: 10.3934/jimo.2015.11.1355
[6]	Justine Yasappan, Ángela Jiménez-Casas, Mario Castro. Stabilizing interplay between thermodiffusion and viscoelasticity in a closed-loop thermosyphon. Discrete & Continuous Dynamical Systems - B, 2015, 20 (9) : 3267-3299. doi: 10.3934/dcdsb.2015.20.3267
[7]	Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[8]	Robert S. Anderssen, Martin Kružík. Modelling of wheat-flour dough mixing as an open-loop hysteretic process. Discrete & Continuous Dynamical Systems - B, 2013, 18 (2) : 283-293. doi: 10.3934/dcdsb.2013.18.283
[9]	Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764
[10]	Yi Jing, Wenchuan Li. Integrated recycling-integrated production - distribution planning for decentralized closed-loop supply chain. Journal of Industrial & Management Optimization, 2018, 14 (2) : 511-539. doi: 10.3934/jimo.2017058
[11]	Wenbin Wang, Peng Zhang, Junfei Ding, Jian Li, Hao Sun, Lingyun He. Closed-loop supply chain network equilibrium model with retailer-collection under legislation. Journal of Industrial & Management Optimization, 2019, 15 (1) : 199-219. doi: 10.3934/jimo.2018039
[12]	Masoud Mohammadzadeh, Alireza Arshadi Khamseh, Mohammad Mohammadi. A multi-objective integrated model for closed-loop supply chain configuration and supplier selection considering uncertain demand and different performance levels. Journal of Industrial & Management Optimization, 2017, 13 (2) : 1041-1064. doi: 10.3934/jimo.2016061
[13]	Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete & Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261
[14]	Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889
[15]	Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465
[16]	Xiaohong Chen, Kui Li, Fuqiang Wang, Xihua Li. Optimal production, pricing and government subsidy policies for a closed loop supply chain with uncertain returns. Journal of Industrial & Management Optimization, 2017, 13 (5) : 1-26. doi: 10.3934/jimo.2019008
[17]	Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete & Continuous Dynamical Systems - A, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193
[18]	Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[19]	Marcus A. Khuri. On the local solvability of Darboux's equation. Conference Publications, 2009, 2009 (Special) : 451-456. doi: 10.3934/proc.2009.2009.451
[20]	Victor Zvyagin, Vladimir Orlov. Weak solvability of fractional Voigt model of viscoelasticity. Discrete & Continuous Dynamical Systems - A, 2018, 38 (12) : 6327-6350. doi: 10.3934/dcds.2018270

American Institute of Mathematical Sciences