American Institute of Mathematical Sciences

Advanced
January  2017, 2: 11 doi: 10.1186/s41546-017-0022-7

Characterization of optimal feedback for stochastic linear quadratic control problems

Qi Lü , Tianxiao Wang and  Xu Zhang

School of Mathematics, Sichuan University, Chengdu 610064, Sichuan Province, China

Received  June 21, 2016 Revised  September 2017 Published  March 2017

Fund Project: This work is supported by the NSF of China under grants 11471231, 11221101, 11231007, 11301298 and 11401404, the PCSIRT under grant IRT 16R53 and the Chang Jiang Scholars Program from Chinese Education Ministry, the Fundamental Research Funds for the Central Universities in China under grant 2015SCU04A02, and the NSFC-CNRS Joint Research Project under grant 11711530142.

One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the study of deterministic linear quadratic control problems is exactly to construct the desired feedbacks. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a suitable sense. We also give a counterexample showing the nonexistence of feedback controls to a solvable stochastic linear quadratic control problem. This is a new phenomenon in the stochastic setting, significantly different from its deterministic counterpart.
Keywords: Stochastic linear quadratic problem, Feedback control, Backward stochastic Riccati equation, Backward stochastic differential equation.
Mathematics Subject Classification: 93E20;93B52;93C05;60H10.
Citation: Qi Lü, Tianxiao Wang, Xu Zhang. Characterization of optimal feedback for stochastic linear quadratic control problems. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 11-. doi: 10.1186/s41546-017-0022-7
References:
[1]

Ait Rami, M, Moore, JB, Zhou, X:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001) Google Scholar

[2]

Athans, M:The role and use of the stochastic linear-quadratic-Gaussian problem in control system design.IEEE Trans. Automat. Control 16, 529-552 (1971) Google Scholar

[3]

Ben-Israel, A, Greville, TNE:Generalized Inverses:Theory and Applications. Pure and Applied Mathematics. Wiley-Interscience[John Wiley & Sons], New York-London-Sydney (1974) Google Scholar

[4]

Bensoussan, A:Lectures on stochastic control. In:Nonlinear Filtering and Stochastic Control. Lecture Notes in Math, vol. 972, pp. 1-62. Springer-Verlag, Berlin (1981) Google Scholar

[5]

Bismut, J-M:Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 14, 419-444 (1976) Google Scholar

[6]

Bismut, J-M:Contrôle des systèmes linéaires quadratiques:applications de l'intégrale stochastique. In:Séminaire de Probabilités XII, Université de Strasbourg 1976/77, Lecture Notes in Math, vol. 649, pp. 180-264. Springer-Verlag, Berlin (1978) Google Scholar

[7]

Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L:Lp solutions of backward stochastic differential equations. Stochastic Process. Appl 108, 109-129 (2003) Google Scholar

[8]

Chen, S, Li, X, Zhou, X:Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim 36, 1685-1702 (1998) Google Scholar

[9]

Davis, MHA:Linear Estimation and Stochastic Control. Chapman and Hall Mathematics Series. Chapman and Hall, London; Halsted Press[John Wiley & Sons], New York (1977) Google Scholar

[10]

Delbaen, F, Tang, S:Harmonic analysis of stochastic equations and backward stochastic differential equations. Probab. Theory Relat. Fields 146, 291-336 (2010) Google Scholar

[11]

Frei, C, dos Reis, G:A financial market with interacting investors:does an equilibrium exist? Math. Finan.Econ 4, 161-182 (2011) Google Scholar

[12]

Kalman, RE:Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960) Google Scholar

[13]

Lü, Q, Zhang, X:Well-posedness of backward stochastic differential equations with general filtration. J. Diff. Equations 254, 3200-3227 (2013) Google Scholar

[14]

Lü, Q, Zhang, X:General Pontryagin-Type Stochastic Maximum Principle and Backward StochasticEvolution Equations in Infinite Dimensions. Springer Briefs in Mathematics. Springer, Cham (2014) Google Scholar

[15]

Lü, Q, Zhang, X:Transposition method for backward stochastic evolution equations revisited, and its application. Math. Control Relat. Fields 5, 529-555 (2015) Google Scholar

[16]

Lü, Q, Zhang, X:Optimal feedback for stochastic linear quadratic control and backward stochastic Riccati equations in infinite dimensions. (2017). Preprint Pardoux, E, Peng, S:Adapted solution of backward stochastic equation. Systems Control Lett 14, 55-61(1990) Google Scholar

[17]

Peng, S:Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim 30, 284-304 (1992) Google Scholar

[18]

Pham, H:Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications. (2017). arXiv:1604.06609v1 Google Scholar

[19]

Protter, PE:Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer-Verlag, Berlin (2005) Google Scholar

[20]

Reid, WT:A matrix differential equation of Riccati type. Amer. J. Math 68, 237-246 (1946) Google Scholar

[21]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014) Google Scholar

[22]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003) Google Scholar

[23]

Tang, S:Dynamic programming for general linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 53, 1082-1106 (2015) Google Scholar

[24]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681-697 (1968) Google Scholar

[25]

Wonham, WM:Linear Multivariable Control, a Geometric Approach. Applications of Mathematics, vol. 10. Springer-Verlag, New York (1985) Google Scholar

[26]

Yong, J, Lou, H:A Concise Course on Optimal Control Theory. Higher Education Press, Beijing (2006).(In Chinese) Google Scholar

[27]

Yong, J, Zhou, XY:Stochastic Controls:Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000) Google Scholar

show all references

References:
[1]

Ait Rami, M, Moore, JB, Zhou, X:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001) Google Scholar

[2]

Athans, M:The role and use of the stochastic linear-quadratic-Gaussian problem in control system design.IEEE Trans. Automat. Control 16, 529-552 (1971) Google Scholar

[3]

Ben-Israel, A, Greville, TNE:Generalized Inverses:Theory and Applications. Pure and Applied Mathematics. Wiley-Interscience[John Wiley & Sons], New York-London-Sydney (1974) Google Scholar

[4]

Bensoussan, A:Lectures on stochastic control. In:Nonlinear Filtering and Stochastic Control. Lecture Notes in Math, vol. 972, pp. 1-62. Springer-Verlag, Berlin (1981) Google Scholar

[5]

Bismut, J-M:Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 14, 419-444 (1976) Google Scholar

[6]

Bismut, J-M:Contrôle des systèmes linéaires quadratiques:applications de l'intégrale stochastique. In:Séminaire de Probabilités XII, Université de Strasbourg 1976/77, Lecture Notes in Math, vol. 649, pp. 180-264. Springer-Verlag, Berlin (1978) Google Scholar

[7]

Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L:Lp solutions of backward stochastic differential equations. Stochastic Process. Appl 108, 109-129 (2003) Google Scholar

[8]

Chen, S, Li, X, Zhou, X:Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim 36, 1685-1702 (1998) Google Scholar

[9]

Davis, MHA:Linear Estimation and Stochastic Control. Chapman and Hall Mathematics Series. Chapman and Hall, London; Halsted Press[John Wiley & Sons], New York (1977) Google Scholar

[10]

Delbaen, F, Tang, S:Harmonic analysis of stochastic equations and backward stochastic differential equations. Probab. Theory Relat. Fields 146, 291-336 (2010) Google Scholar

[11]

Frei, C, dos Reis, G:A financial market with interacting investors:does an equilibrium exist? Math. Finan.Econ 4, 161-182 (2011) Google Scholar

[12]

Kalman, RE:Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960) Google Scholar

[13]

Lü, Q, Zhang, X:Well-posedness of backward stochastic differential equations with general filtration. J. Diff. Equations 254, 3200-3227 (2013) Google Scholar

[14]

Lü, Q, Zhang, X:General Pontryagin-Type Stochastic Maximum Principle and Backward StochasticEvolution Equations in Infinite Dimensions. Springer Briefs in Mathematics. Springer, Cham (2014) Google Scholar

[15]

Lü, Q, Zhang, X:Transposition method for backward stochastic evolution equations revisited, and its application. Math. Control Relat. Fields 5, 529-555 (2015) Google Scholar

[16]

Lü, Q, Zhang, X:Optimal feedback for stochastic linear quadratic control and backward stochastic Riccati equations in infinite dimensions. (2017). Preprint Pardoux, E, Peng, S:Adapted solution of backward stochastic equation. Systems Control Lett 14, 55-61(1990) Google Scholar

[17]

Peng, S:Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim 30, 284-304 (1992) Google Scholar

[18]

Pham, H:Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications. (2017). arXiv:1604.06609v1 Google Scholar

[19]

Protter, PE:Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer-Verlag, Berlin (2005) Google Scholar

[20]

Reid, WT:A matrix differential equation of Riccati type. Amer. J. Math 68, 237-246 (1946) Google Scholar

[21]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014) Google Scholar

[22]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003) Google Scholar

[23]

Tang, S:Dynamic programming for general linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 53, 1082-1106 (2015) Google Scholar

[24]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681-697 (1968) Google Scholar

[25]

Wonham, WM:Linear Multivariable Control, a Geometric Approach. Applications of Mathematics, vol. 10. Springer-Verlag, New York (1985) Google Scholar

[26]

Yong, J, Lou, H:A Concise Course on Optimal Control Theory. Higher Education Press, Beijing (2006).(In Chinese) Google Scholar

[27]

Yong, J, Zhou, XY:Stochastic Controls:Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000) Google Scholar

[1]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control & Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028
[2]

Renzhi Qiu, Shanjian Tang. The Cauchy problem of Backward Stochastic Super-Parabolic Equations with Quadratic Growth. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 3-. doi: 10.1186/s41546-019-0037-3
[3]

Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems & Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002
[4]

Jasmina Djordjević, Svetlana Janković. Reflected backward stochastic differential equations with perturbations. Discrete & Continuous Dynamical Systems, 2018, 38 (4) : 1833-1848. doi: 10.3934/dcds.2018075
[5]

Jan A. Van Casteren. On backward stochastic differential equations in infinite dimensions. Discrete & Continuous Dynamical Systems - S, 2013, 6 (3) : 803-824. doi: 10.3934/dcdss.2013.6.803
[6]

Joscha Diehl, Jianfeng Zhang. Backward stochastic differential equations with Young drift. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 5-. doi: 10.1186/s41546-017-0016-5
[7]

Ishak Alia. Time-inconsistent stochastic optimal control problems: a backward stochastic partial differential equations approach. Mathematical Control & Related Fields, 2020, 10 (4) : 785-826. doi: 10.3934/mcrf.2020020
[8]

Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115
[9]

Ying Hu, Shanjian Tang. Switching game of backward stochastic differential equations and associated system of obliquely reflected backward stochastic differential equations. Discrete & Continuous Dynamical Systems, 2015, 35 (11) : 5447-5465. doi: 10.3934/dcds.2015.35.5447
[10]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[11]

Editorial Office. Retraction: Xiao-Qian Jiang and Lun-Chuan Zhang, A pricing option approach based on backward stochastic differential equation theory. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 969-969. doi: 10.3934/dcdss.2019065
[12]

Shaokuan Chen, Shanjian Tang. Semi-linear backward stochastic integral partial differential equations driven by a Brownian motion and a Poisson point process. Mathematical Control & Related Fields, 2015, 5 (3) : 401-434. doi: 10.3934/mcrf.2015.5.401
[13]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[14]

Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control & Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022
[15]

Linda J. S. Allen, P. van den Driessche. Stochastic epidemic models with a backward bifurcation. Mathematical Biosciences & Engineering, 2006, 3 (3) : 445-458. doi: 10.3934/mbe.2006.3.445
[16]

Qi Zhang, Huaizhong Zhao. Backward doubly stochastic differential equations with polynomial growth coefficients. Discrete & Continuous Dynamical Systems, 2015, 35 (11) : 5285-5315. doi: 10.3934/dcds.2015.35.5285
[17]

Yufeng Shi, Qingfeng Zhu. A Kneser-type theorem for backward doubly stochastic differential equations. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1565-1579. doi: 10.3934/dcdsb.2010.14.1565
[18]

Yanqing Wang. A semidiscrete Galerkin scheme for backward stochastic parabolic differential equations. Mathematical Control & Related Fields, 2016, 6 (3) : 489-515. doi: 10.3934/mcrf.2016013
[19]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047
[20]

Weidong Zhao, Jinlei Wang, Shige Peng. Error estimates of the $\theta$-scheme for backward stochastic differential equations. Discrete & Continuous Dynamical Systems - B, 2009, 12 (4) : 905-924. doi: 10.3934/dcdsb.2009.12.905

 Impact Factor: 

Tools

Metrics

  • PDF downloads (14)
  • HTML views (255)
  • Cited by (6)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences