American Institute of Mathematical Sciences

Advanced
January  2017, 2: 11 doi: 10.1186/s41546-017-0022-7

Characterization of optimal feedback for stochastic linear quadratic control problems

Qi Lü , Tianxiao Wang and  Xu Zhang

School of Mathematics, Sichuan University, Chengdu 610064, Sichuan Province, China

Received  June 21, 2016 Revised  September 2017

Fund Project: This work is supported by the NSF of China under grants 11471231, 11221101, 11231007, 11301298 and 11401404, the PCSIRT under grant IRT 16R53 and the Chang Jiang Scholars Program from Chinese Education Ministry, the Fundamental Research Funds for the Central Universities in China under grant 2015SCU04A02, and the NSFC-CNRS Joint Research Project under grant 11711530142.

One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the study of deterministic linear quadratic control problems is exactly to construct the desired feedbacks. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a suitable sense. We also give a counterexample showing the nonexistence of feedback controls to a solvable stochastic linear quadratic control problem. This is a new phenomenon in the stochastic setting, significantly different from its deterministic counterpart.
Keywords: Stochastic linear quadratic problem, Feedback control, Backward stochastic Riccati equation, Backward stochastic differential equation.
Mathematics Subject Classification: 93E20;93B52;93C05;60H10.
Citation: Qi Lü, Tianxiao Wang, Xu Zhang. Characterization of optimal feedback for stochastic linear quadratic control problems. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 11-. doi: 10.1186/s41546-017-0022-7
References:
[1]

Ait Rami, M, Moore, JB, Zhou, X:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001),

[2]

Athans, M:The role and use of the stochastic linear-quadratic-Gaussian problem in control system design.IEEE Trans. Automat. Control 16, 529-552 (1971),

[3]

Ben-Israel, A, Greville, TNE:Generalized Inverses:Theory and Applications. Pure and Applied Mathematics. Wiley-Interscience[John Wiley & Sons], New York-London-Sydney (1974),

[4]

Bensoussan, A:Lectures on stochastic control. In:Nonlinear Filtering and Stochastic Control. Lecture Notes in Math, vol. 972, pp. 1-62. Springer-Verlag, Berlin (1981),

[5]

Bismut, J-M:Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 14, 419-444 (1976),

[6]

Bismut, J-M:Contrôle des systèmes linéaires quadratiques:applications de l'intégrale stochastique. In:Séminaire de Probabilités XII, Université de Strasbourg 1976/77, Lecture Notes in Math, vol. 649, pp. 180-264. Springer-Verlag, Berlin (1978),

[7]

Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L:Lp solutions of backward stochastic differential equations. Stochastic Process. Appl 108, 109-129 (2003),

[8]

Chen, S, Li, X, Zhou, X:Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim 36, 1685-1702 (1998),

[9]

Davis, MHA:Linear Estimation and Stochastic Control. Chapman and Hall Mathematics Series. Chapman and Hall, London; Halsted Press[John Wiley & Sons], New York (1977),

[10]

Delbaen, F, Tang, S:Harmonic analysis of stochastic equations and backward stochastic differential equations. Probab. Theory Relat. Fields 146, 291-336 (2010),

[11]

Frei, C, dos Reis, G:A financial market with interacting investors:does an equilibrium exist? Math. Finan.Econ 4, 161-182 (2011),

[12]

Kalman, RE:Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960),

[13]

Lü, Q, Zhang, X:Well-posedness of backward stochastic differential equations with general filtration. J. Diff. Equations 254, 3200-3227 (2013),

[14]

Lü, Q, Zhang, X:General Pontryagin-Type Stochastic Maximum Principle and Backward StochasticEvolution Equations in Infinite Dimensions. Springer Briefs in Mathematics. Springer, Cham (2014),

[15]

Lü, Q, Zhang, X:Transposition method for backward stochastic evolution equations revisited, and its application. Math. Control Relat. Fields 5, 529-555 (2015),

[16]

Lü, Q, Zhang, X:Optimal feedback for stochastic linear quadratic control and backward stochastic Riccati equations in infinite dimensions. (2017). Preprint Pardoux, E, Peng, S:Adapted solution of backward stochastic equation. Systems Control Lett 14, 55-61(1990),

[17]

Peng, S:Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim 30, 284-304 (1992),

[18]

Pham, H:Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications. (2017). arXiv:1604.06609v1,

[19]

Protter, PE:Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer-Verlag, Berlin (2005),

[20]

Reid, WT:A matrix differential equation of Riccati type. Amer. J. Math 68, 237-246 (1946),

[21]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014),

[22]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003),

[23]

Tang, S:Dynamic programming for general linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 53, 1082-1106 (2015),

[24]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681-697 (1968),

[25]

Wonham, WM:Linear Multivariable Control, a Geometric Approach. Applications of Mathematics, vol. 10. Springer-Verlag, New York (1985),

[26]

Yong, J, Lou, H:A Concise Course on Optimal Control Theory. Higher Education Press, Beijing (2006).(In Chinese),

[27]

Yong, J, Zhou, XY:Stochastic Controls:Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000),

show all references

References:
[1]

Ait Rami, M, Moore, JB, Zhou, X:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001),

[2]

Athans, M:The role and use of the stochastic linear-quadratic-Gaussian problem in control system design.IEEE Trans. Automat. Control 16, 529-552 (1971),

[3]

Ben-Israel, A, Greville, TNE:Generalized Inverses:Theory and Applications. Pure and Applied Mathematics. Wiley-Interscience[John Wiley & Sons], New York-London-Sydney (1974),

[4]

Bensoussan, A:Lectures on stochastic control. In:Nonlinear Filtering and Stochastic Control. Lecture Notes in Math, vol. 972, pp. 1-62. Springer-Verlag, Berlin (1981),

[5]

Bismut, J-M:Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 14, 419-444 (1976),

[6]

Bismut, J-M:Contrôle des systèmes linéaires quadratiques:applications de l'intégrale stochastique. In:Séminaire de Probabilités XII, Université de Strasbourg 1976/77, Lecture Notes in Math, vol. 649, pp. 180-264. Springer-Verlag, Berlin (1978),

[7]

Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L:Lp solutions of backward stochastic differential equations. Stochastic Process. Appl 108, 109-129 (2003),

[8]

Chen, S, Li, X, Zhou, X:Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim 36, 1685-1702 (1998),

[9]

Davis, MHA:Linear Estimation and Stochastic Control. Chapman and Hall Mathematics Series. Chapman and Hall, London; Halsted Press[John Wiley & Sons], New York (1977),

[10]

Delbaen, F, Tang, S:Harmonic analysis of stochastic equations and backward stochastic differential equations. Probab. Theory Relat. Fields 146, 291-336 (2010),

[11]

Frei, C, dos Reis, G:A financial market with interacting investors:does an equilibrium exist? Math. Finan.Econ 4, 161-182 (2011),

[12]

Kalman, RE:Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960),

[13]

Lü, Q, Zhang, X:Well-posedness of backward stochastic differential equations with general filtration. J. Diff. Equations 254, 3200-3227 (2013),

[14]

Lü, Q, Zhang, X:General Pontryagin-Type Stochastic Maximum Principle and Backward StochasticEvolution Equations in Infinite Dimensions. Springer Briefs in Mathematics. Springer, Cham (2014),

[15]

Lü, Q, Zhang, X:Transposition method for backward stochastic evolution equations revisited, and its application. Math. Control Relat. Fields 5, 529-555 (2015),

[16]

Lü, Q, Zhang, X:Optimal feedback for stochastic linear quadratic control and backward stochastic Riccati equations in infinite dimensions. (2017). Preprint Pardoux, E, Peng, S:Adapted solution of backward stochastic equation. Systems Control Lett 14, 55-61(1990),

[17]

Peng, S:Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim 30, 284-304 (1992),

[18]

Pham, H:Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications. (2017). arXiv:1604.06609v1,

[19]

Protter, PE:Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer-Verlag, Berlin (2005),

[20]

Reid, WT:A matrix differential equation of Riccati type. Amer. J. Math 68, 237-246 (1946),

[21]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014),

[22]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003),

[23]

Tang, S:Dynamic programming for general linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 53, 1082-1106 (2015),

[24]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681-697 (1968),

[25]

Wonham, WM:Linear Multivariable Control, a Geometric Approach. Applications of Mathematics, vol. 10. Springer-Verlag, New York (1985),

[26]

Yong, J, Lou, H:A Concise Course on Optimal Control Theory. Higher Education Press, Beijing (2006).(In Chinese),

[27]

Yong, J, Zhou, XY:Stochastic Controls:Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000),

[1]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047
[2]

Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317
[3]

Reza Chaharpashlou, Abdon Atangana, Reza Saadati. On the fuzzy stability results for fractional stochastic Volterra integral equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020432
[4]

Lorenzo Zambotti. A brief and personal history of stochastic partial differential equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 471-487. doi: 10.3934/dcds.2020264
[5]

Fathalla A. Rihan, Hebatallah J. Alsakaji. Stochastic delay differential equations of three-species prey-predator system with cooperation among prey species. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020468
[6]

Zhenzhen Wang, Tianshou Zhou. Asymptotic behaviors and stochastic traveling waves in stochastic Fisher-KPP equations. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020323
[7]

Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020384
[8]

Xuhui Peng, Rangrang Zhang. Approximations of stochastic 3D tamed Navier-Stokes equations. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5337-5365. doi: 10.3934/cpaa.2020241
[9]

Yahia Zare Mehrjerdi. A new methodology for solving bi-criterion fractional stochastic programming. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020054
[10]

Christian Beck, Lukas Gonon, Martin Hutzenthaler, Arnulf Jentzen. On existence and uniqueness properties for solutions of stochastic fixed point equations. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020320
[11]

Giuseppina Guatteri, Federica Masiero. Stochastic maximum principle for problems with delay with dependence on the past through general measures. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020048
[12]

Yangrong Li, Shuang Yang, Qiangheng Zhang. Odd random attractors for stochastic non-autonomous Kuramoto-Sivashinsky equations without dissipation. Electronic Research Archive, 2020, 28 (4) : 1529-1544. doi: 10.3934/era.2020080
[13]

Pengyu Chen. Non-autonomous stochastic evolution equations with nonlinear noise and nonlocal conditions governed by noncompact evolution families. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020383
[14]

Lin Shi, Xuemin Wang, Dingshi Li. Limiting behavior of non-autonomous stochastic reaction-diffusion equations with colored noise on unbounded thin domains. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5367-5386. doi: 10.3934/cpaa.2020242
[15]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444
[16]

Peter Poláčik, Pavol Quittner. Entire and ancient solutions of a supercritical semilinear heat equation. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 413-438. doi: 10.3934/dcds.2020136
[17]

Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345
[18]

Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079
[19]

Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081
[20]

Hirokazu Ninomiya. Entire solutions of the Allen–Cahn–Nagumo equation in a multi-dimensional space. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 395-412. doi: 10.3934/dcds.2020364

 Impact Factor: 

Tools

Metrics

  • PDF downloads (3)
  • HTML views (14)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences