American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Existence and stability of one-peak symmetric stationary solutions for the Schnakenberg model with heterogeneity
  • DCDS Home
  • This Issue
  • Next Article
    Local wellposedness for the critical nonlinear Schrödinger equation on $ \mathbb{T}^3 $
May  2019, 39(5): 2785-2805. doi: 10.3934/dcds.2019117

Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Hanxiao Wang 1, , Jingrui Sun 2,, and  Jiongmin Yong 3,

1. 

School of Mathematical Sciences, Fudan University, Shanghai 200433, China
2. 

Department of Mathematics, Southern University of Science and Technology, Shenzhen, Guangdong 518055, China
3. 

Department of Mathematics, University of Central Florida, Orlando FL 32816, USA

* Corresponding author: Jingrui Sun

Received  June 2018 Published  January 2019

Fund Project: The first author is supported in part by the China Scholarship Council, while visiting University of Central Florida. The third author is supported in part by NSF DMS-1812921.

Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.

Keywords: Stochastic linear-quadratic optimal control, Riccati equation, open-loop solvability, weak closed-loop solvability, state feedback.
Mathematics Subject Classification: Primary: 93E20; Secondary: 49N10, 49N35.
Citation: Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
References:
[1]

M. Ait Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), 1296-1311.  doi: 10.1137/S0363012900371083.  Google Scholar

[2]

R. Bellman, I. Glicksberg and O. Gross, Some Aspects of the Mathematical Theory of Control Processes, RAND Corporation, Santa Monica, CA, 1958.  Google Scholar

[3]

A. Bensoussan, Lectures on stochstic control, part Ⅰ, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math., Springer-Verlag, Berlin, 972 (1982), 1–62.  Google Scholar

[4]

J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), 419-444.  doi: 10.1137/0314028.  Google Scholar

[5]

S. Chen, X. Li and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), 1685-1702.  doi: 10.1137/S0363012996310478.  Google Scholar

[6]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chin. Ann. Math., 21 B (2000), 323-338.  doi: 10.1142/S0252959900000339.  Google Scholar

[7]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), 21-45.  doi: 10.1007/s002450010016.  Google Scholar

[8]

S. Chen and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. Ⅱ, SIAM J. Control Optim., 39 (2000), 1065-1081.  doi: 10.1137/S0363012998346578.  Google Scholar

[9]

M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, 1977.  Google Scholar

[10]

R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc., Mat. Mexicana, 5 (1960), 102-119.   Google Scholar

[11]

A. M. Letov, The analytical design of control systems, Automat. Remote Control, 22 (1961), 363-372.   Google Scholar

[12]

A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Trans. Automat. Control, 44 (1999), 1359-1369.  doi: 10.1109/9.774108.  Google Scholar

[13]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.  doi: 10.1137/15M103532X.  Google Scholar

[14]

J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[15]

S. Tang, General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations, SIAM J. Control Optim., 42 (2003), 53-75.  doi: 10.1137/S0363012901387550.  Google Scholar

[16]

S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 53 (2015), 1082-1106.  doi: 10.1137/140979940.  Google Scholar

[17]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar

[18]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

M. Ait Rami, J. B. Moore and X. Y. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), 1296-1311.  doi: 10.1137/S0363012900371083.  Google Scholar

[2]

R. Bellman, I. Glicksberg and O. Gross, Some Aspects of the Mathematical Theory of Control Processes, RAND Corporation, Santa Monica, CA, 1958.  Google Scholar

[3]

A. Bensoussan, Lectures on stochstic control, part Ⅰ, in Nonlinear Filtering and Stochastic Control, Lecture Notes in Math., Springer-Verlag, Berlin, 972 (1982), 1–62.  Google Scholar

[4]

J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), 419-444.  doi: 10.1137/0314028.  Google Scholar

[5]

S. Chen, X. Li and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), 1685-1702.  doi: 10.1137/S0363012996310478.  Google Scholar

[6]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems with random coefficients, Chin. Ann. Math., 21 B (2000), 323-338.  doi: 10.1142/S0252959900000339.  Google Scholar

[7]

S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), 21-45.  doi: 10.1007/s002450010016.  Google Scholar

[8]

S. Chen and X. Y. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. Ⅱ, SIAM J. Control Optim., 39 (2000), 1065-1081.  doi: 10.1137/S0363012998346578.  Google Scholar

[9]

M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, 1977.  Google Scholar

[10]

R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc., Mat. Mexicana, 5 (1960), 102-119.   Google Scholar

[11]

A. M. Letov, The analytical design of control systems, Automat. Remote Control, 22 (1961), 363-372.   Google Scholar

[12]

A. E. B. Lim and X. Y. Zhou, Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights, IEEE Trans. Automat. Control, 44 (1999), 1359-1369.  doi: 10.1109/9.774108.  Google Scholar

[13]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), 2274-2308.  doi: 10.1137/15M103532X.  Google Scholar

[14]

J. Sun and J. Yong, Linear quadratic stochastic differential games: Open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[15]

S. Tang, General linear quadratic optimal stochastic control problems with random coefficients: linear stochastic Hamilton systems and backward stochastic Riccati equations, SIAM J. Control Optim., 42 (2003), 53-75.  doi: 10.1137/S0363012901387550.  Google Scholar

[16]

S. Tang, Dynamic programming for general linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 53 (2015), 1082-1106.  doi: 10.1137/140979940.  Google Scholar

[17]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J. Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar

[18]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[1]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[2]

Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3
[3]

Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021074
[4]

Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[5]

Simon Hochgerner. Symmetry actuated closed-loop Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 641-669. doi: 10.3934/jgm.2020030
[6]

Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control & Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040
[7]

Dean A. Carlson. Finding open-loop Nash equilibrium for variational games. Conference Publications, 2005, 2005 (Special) : 153-163. doi: 10.3934/proc.2005.2005.153
[8]

Michael Schönlein. Computation of open-loop inputs for uniformly ensemble controllable systems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021046
[9]

Pasquale Palumbo, Pierdomenico Pepe, Simona Panunzi, Andrea De Gaetano. Robust closed-loop control of plasma glycemia: A discrete-delay model approach. Discrete & Continuous Dynamical Systems - B, 2009, 12 (2) : 455-468. doi: 10.3934/dcdsb.2009.12.455
[10]

Filippo Cacace, Valerio Cusimano, Alfredo Germani, Pasquale Palumbo, Federico Papa. Closed-loop control of tumor growth by means of anti-angiogenic administration. Mathematical Biosciences & Engineering, 2018, 15 (4) : 827-839. doi: 10.3934/mbe.2018037
[11]

Xiaochen Sun, Fei Hu, Yancong Zhou, Cheng-Chew Lim. Optimal acquisition, inventory and production decisions for a closed-loop manufacturing system with legislation constraint. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1355-1373. doi: 10.3934/jimo.2015.11.1355
[12]

Justine Yasappan, Ángela Jiménez-Casas, Mario Castro. Stabilizing interplay between thermodiffusion and viscoelasticity in a closed-loop thermosyphon. Discrete & Continuous Dynamical Systems - B, 2015, 20 (9) : 3267-3299. doi: 10.3934/dcdsb.2015.20.3267
[13]

Qi Lü, Tianxiao Wang, Xu Zhang. Characterization of optimal feedback for stochastic linear quadratic control problems. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 11-. doi: 10.1186/s41546-017-0022-7
[14]

Yadong Shu, Bo Li. Linear-quadratic optimal control for discrete-time stochastic descriptor systems. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021034
[15]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[16]

Jiannan Zhang, Ping Chen, Zhuo Jin, Shuanming Li. Open-loop equilibrium strategy for mean-variance portfolio selection: A log-return model. Journal of Industrial & Management Optimization, 2021, 17 (2) : 765-777. doi: 10.3934/jimo.2019133
[17]

Robert S. Anderssen, Martin Kružík. Modelling of wheat-flour dough mixing as an open-loop hysteretic process. Discrete & Continuous Dynamical Systems - B, 2013, 18 (2) : 283-293. doi: 10.3934/dcdsb.2013.18.283
[18]

Liming Zhang, Rongming Wang, Jiaqin Wei. Open-loop equilibrium mean-variance reinsurance, new business and investment strategies with constraints. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021140
[19]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764
[20]

Mohamed Aliane, Mohand Bentobache, Nacima Moussouni, Philippe Marthon. Direct method to solve linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2021, 11 (4) : 645-663. doi: 10.3934/naco.2021002

2020 Impact Factor: 1.392

Tools

Metrics

  • PDF downloads (155)
  • HTML views (169)
  • Cited by (2)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences