American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Existence and stability of one-peak symmetric stationary solutions for the Schnakenberg model with heterogeneity
  • DCDS Home
  • This Issue
  • Next Article
    Local wellposedness for the critical nonlinear Schrödinger equation on $ \mathbb{T}^3 $
May  2019, 39(5): 2785-2805. doi: 10.3934/dcds.2019117

Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Hanxiao Wang 1, , Jingrui Sun 2,, and  Jiongmin Yong 3,

1. 

School of Mathematical Sciences, Fudan University, Shanghai 200433, China
2. 

Department of Mathematics, Southern University of Science and Technology, Shenzhen, Guangdong 518055, China
3. 

Department of Mathematics, University of Central Florida, Orlando FL 32816, USA

* Corresponding author: Jingrui Sun

Received  June 2018 Published  January 2019

Fund Project: The first author is supported in part by the China Scholarship Council, while visiting University of Central Florida. The third author is supported in part by NSF DMS-1812921

Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.

Keywords: Stochastic linear-quadratic optimal control, Riccati equation, open-loop solvability, weak closed-loop solvability, state feedback.
Mathematics Subject Classification: Primary: 93E20; Secondary: 49N10, 49N35.
Citation: Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems - A, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
References:
[1]

M. Ait Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), 1296-1311.  doi: 10.1137/S0363012900371083.  Google Scholar

[2]

R. Bellman, I. Glicksberg and O. Gross, Some Aspects of the Mathematical Theory of Control Processes, RAND Corporation, Santa Monica, CA, 1958.  Google Scholar

[3]

A. Bensoussan, Lectures on stochstic control, part Ⅰ, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math., Springer-Verlag, Berlin, 972 (1982), 1–62.  Google Scholar

[4]

J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), 419-444.  doi: 10.1137/0314028.  Google Scholar

[5]

S. Chen, X. Li and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), 1685-1702.  doi: 10.1137/S0363012996310478.  Google Scholar

[6]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chin. Ann. Math., 21 B (2000), 323-338.  doi: 10.1142/S0252959900000339.  Google Scholar

[7]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), 21-45.  doi: 10.1007/s002450010016.  Google Scholar

[8]

S. Chen and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. Ⅱ, SIAM J. Control Optim., 39 (2000), 1065-1081.  doi: 10.1137/S0363012998346578.  Google Scholar

[9]

M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, 1977.  Google Scholar

[10]

R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc., Mat. Mexicana, 5 (1960), 102-119.   Google Scholar

[11]

A. M. Letov, The analytical design of control systems, Automat. Remote Control, 22 (1961), 363-372.   Google Scholar

[12]

A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Trans. Automat. Control, 44 (1999), 1359-1369.  doi: 10.1109/9.774108.  Google Scholar

[13]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.  doi: 10.1137/15M103532X.  Google Scholar

[14]

J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[15]

S. Tang, General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations, SIAM J. Control Optim., 42 (2003), 53-75.  doi: 10.1137/S0363012901387550.  Google Scholar

[16]

S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 53 (2015), 1082-1106.  doi: 10.1137/140979940.  Google Scholar

[17]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar

[18]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

M. Ait Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), 1296-1311.  doi: 10.1137/S0363012900371083.  Google Scholar

[2]

R. Bellman, I. Glicksberg and O. Gross, Some Aspects of the Mathematical Theory of Control Processes, RAND Corporation, Santa Monica, CA, 1958.  Google Scholar

[3]

A. Bensoussan, Lectures on stochstic control, part Ⅰ, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math., Springer-Verlag, Berlin, 972 (1982), 1–62.  Google Scholar

[4]

J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), 419-444.  doi: 10.1137/0314028.  Google Scholar

[5]

S. Chen, X. Li and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), 1685-1702.  doi: 10.1137/S0363012996310478.  Google Scholar

[6]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chin. Ann. Math., 21 B (2000), 323-338.  doi: 10.1142/S0252959900000339.  Google Scholar

[7]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), 21-45.  doi: 10.1007/s002450010016.  Google Scholar

[8]

S. Chen and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. Ⅱ, SIAM J. Control Optim., 39 (2000), 1065-1081.  doi: 10.1137/S0363012998346578.  Google Scholar

[9]

M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, 1977.  Google Scholar

[10]

R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc., Mat. Mexicana, 5 (1960), 102-119.   Google Scholar

[11]

A. M. Letov, The analytical design of control systems, Automat. Remote Control, 22 (1961), 363-372.   Google Scholar

[12]

A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Trans. Automat. Control, 44 (1999), 1359-1369.  doi: 10.1109/9.774108.  Google Scholar

[13]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.  doi: 10.1137/15M103532X.  Google Scholar

[14]

J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[15]

S. Tang, General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations, SIAM J. Control Optim., 42 (2003), 53-75.  doi: 10.1137/S0363012901387550.  Google Scholar

[16]

S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 53 (2015), 1082-1106.  doi: 10.1137/140979940.  Google Scholar

[17]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar

[18]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[1]

Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[2]

Pasquale Palumbo, Pierdomenico Pepe, Simona Panunzi, Andrea De Gaetano. Robust closed-loop control of plasma glycemia: A discrete-delay model approach. Discrete & Continuous Dynamical Systems - B, 2009, 12 (2) : 455-468. doi: 10.3934/dcdsb.2009.12.455
[3]

Filippo Cacace, Valerio Cusimano, Alfredo Germani, Pasquale Palumbo, Federico Papa. Closed-loop control of tumor growth by means of anti-angiogenic administration. Mathematical Biosciences & Engineering, 2018, 15 (4) : 827-839. doi: 10.3934/mbe.2018037
[4]

Xiaochen Sun, Fei Hu, Yancong Zhou, Cheng-Chew Lim. Optimal acquisition, inventory and production decisions for a closed-loop manufacturing system with legislation constraint. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1355-1373. doi: 10.3934/jimo.2015.11.1355
[5]

Dean A. Carlson. Finding open-loop Nash equilibrium for variational games. Conference Publications, 2005, 2005 (Special) : 153-163. doi: 10.3934/proc.2005.2005.153
[6]

Justine Yasappan, Ángela Jiménez-Casas, Mario Castro. Stabilizing interplay between thermodiffusion and viscoelasticity in a closed-loop thermosyphon. Discrete & Continuous Dynamical Systems - B, 2015, 20 (9) : 3267-3299. doi: 10.3934/dcdsb.2015.20.3267
[7]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[8]

Robert S. Anderssen, Martin Kružík. Modelling of wheat-flour dough mixing as an open-loop hysteretic process. Discrete & Continuous Dynamical Systems - B, 2013, 18 (2) : 283-293. doi: 10.3934/dcdsb.2013.18.283
[9]

Jiannan Zhang, Ping Chen, Zhuo Jin, Shuanming Li. Open-loop equilibrium strategy for mean-variance portfolio selection: A log-return model. Journal of Industrial & Management Optimization, 2017, 13 (5) : 0-0. doi: 10.3934/jimo.2019133
[10]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764
[11]

Yi Jing, Wenchuan Li. Integrated recycling-integrated production - distribution planning for decentralized closed-loop supply chain. Journal of Industrial & Management Optimization, 2018, 14 (2) : 511-539. doi: 10.3934/jimo.2017058
[12]

Wenbin Wang, Peng Zhang, Junfei Ding, Jian Li, Hao Sun, Lingyun He. Closed-loop supply chain network equilibrium model with retailer-collection under legislation. Journal of Industrial & Management Optimization, 2019, 15 (1) : 199-219. doi: 10.3934/jimo.2018039
[13]

Masoud Mohammadzadeh, Alireza Arshadi Khamseh, Mohammad Mohammadi. A multi-objective integrated model for closed-loop supply chain configuration and supplier selection considering uncertain demand and different performance levels. Journal of Industrial & Management Optimization, 2017, 13 (2) : 1041-1064. doi: 10.3934/jimo.2016061
[14]

Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete & Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261
[15]

Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889
[16]

Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465
[17]

Xiaohong Chen, Kui Li, Fuqiang Wang, Xihua Li. Optimal production, pricing and government subsidy policies for a closed loop supply chain with uncertain returns. Journal of Industrial & Management Optimization, 2017, 13 (5) : 1-26. doi: 10.3934/jimo.2019008
[18]

Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete & Continuous Dynamical Systems - A, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193
[19]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[20]

Marcus A. Khuri. On the local solvability of Darboux's equation. Conference Publications, 2009, 2009 (Special) : 451-456. doi: 10.3934/proc.2009.2009.451

2018 Impact Factor: 1.143

Tools

Metrics

  • PDF downloads (52)
  • HTML views (118)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2019 American Institute of Mathematical Sciences