American Institute of Mathematical Sciences

Advanced
January  2017, 2: 11 doi: 10.1186/s41546-017-0022-7

Characterization of optimal feedback for stochastic linear quadratic control problems

Qi Lü , Tianxiao Wang and  Xu Zhang

School of Mathematics, Sichuan University, Chengdu 610064, Sichuan Province, China

Received  June 21, 2016 Revised  September 2017 Published  March 2017

Fund Project: This work is supported by the NSF of China under grants 11471231, 11221101, 11231007, 11301298 and 11401404, the PCSIRT under grant IRT 16R53 and the Chang Jiang Scholars Program from Chinese Education Ministry, the Fundamental Research Funds for the Central Universities in China under grant 2015SCU04A02, and the NSFC-CNRS Joint Research Project under grant 11711530142.

One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the study of deterministic linear quadratic control problems is exactly to construct the desired feedbacks. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a suitable sense. We also give a counterexample showing the nonexistence of feedback controls to a solvable stochastic linear quadratic control problem. This is a new phenomenon in the stochastic setting, significantly different from its deterministic counterpart.
Keywords: Stochastic linear quadratic problem, Feedback control, Backward stochastic Riccati equation, Backward stochastic differential equation.
Mathematics Subject Classification: 93E20;93B52;93C05;60H10.
Citation: Qi Lü, Tianxiao Wang, Xu Zhang. Characterization of optimal feedback for stochastic linear quadratic control problems. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 11-. doi: 10.1186/s41546-017-0022-7
References:
[1]

Ait Rami, M, Moore, JB, Zhou, X:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001)

[2]

Athans, M:The role and use of the stochastic linear-quadratic-Gaussian problem in control system design.IEEE Trans. Automat. Control 16, 529-552 (1971)

[3]

Ben-Israel, A, Greville, TNE:Generalized Inverses:Theory and Applications. Pure and Applied Mathematics. Wiley-Interscience[John Wiley & Sons], New York-London-Sydney (1974)

[4]

Bensoussan, A:Lectures on stochastic control. In:Nonlinear Filtering and Stochastic Control. Lecture Notes in Math, vol. 972, pp. 1-62. Springer-Verlag, Berlin (1981)

[5]

Bismut, J-M:Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 14, 419-444 (1976)

[6]

Bismut, J-M:Contrôle des systèmes linéaires quadratiques:applications de l'intégrale stochastique. In:Séminaire de Probabilités XII, Université de Strasbourg 1976/77, Lecture Notes in Math, vol. 649, pp. 180-264. Springer-Verlag, Berlin (1978)

[7]

Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L:Lp solutions of backward stochastic differential equations. Stochastic Process. Appl 108, 109-129 (2003)

[8]

Chen, S, Li, X, Zhou, X:Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim 36, 1685-1702 (1998)

[9]

Davis, MHA:Linear Estimation and Stochastic Control. Chapman and Hall Mathematics Series. Chapman and Hall, London; Halsted Press[John Wiley & Sons], New York (1977)

[10]

Delbaen, F, Tang, S:Harmonic analysis of stochastic equations and backward stochastic differential equations. Probab. Theory Relat. Fields 146, 291-336 (2010)

[11]

Frei, C, dos Reis, G:A financial market with interacting investors:does an equilibrium exist? Math. Finan.Econ 4, 161-182 (2011)

[12]

Kalman, RE:Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960)

[13]

Lü, Q, Zhang, X:Well-posedness of backward stochastic differential equations with general filtration. J. Diff. Equations 254, 3200-3227 (2013)

[14]

Lü, Q, Zhang, X:General Pontryagin-Type Stochastic Maximum Principle and Backward StochasticEvolution Equations in Infinite Dimensions. Springer Briefs in Mathematics. Springer, Cham (2014)

[15]

Lü, Q, Zhang, X:Transposition method for backward stochastic evolution equations revisited, and its application. Math. Control Relat. Fields 5, 529-555 (2015)

[16]

Lü, Q, Zhang, X:Optimal feedback for stochastic linear quadratic control and backward stochastic Riccati equations in infinite dimensions. (2017). Preprint Pardoux, E, Peng, S:Adapted solution of backward stochastic equation. Systems Control Lett 14, 55-61(1990)

[17]

Peng, S:Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim 30, 284-304 (1992)

[18]

Pham, H:Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications. (2017). arXiv:1604.06609v1

[19]

Protter, PE:Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer-Verlag, Berlin (2005)

[20]

Reid, WT:A matrix differential equation of Riccati type. Amer. J. Math 68, 237-246 (1946)

[21]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014)

[22]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003)

[23]

Tang, S:Dynamic programming for general linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 53, 1082-1106 (2015)

[24]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681-697 (1968)

[25]

Wonham, WM:Linear Multivariable Control, a Geometric Approach. Applications of Mathematics, vol. 10. Springer-Verlag, New York (1985)

[26]

Yong, J, Lou, H:A Concise Course on Optimal Control Theory. Higher Education Press, Beijing (2006).(In Chinese)

[27]

Yong, J, Zhou, XY:Stochastic Controls:Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000)

show all references

References:
[1]

Ait Rami, M, Moore, JB, Zhou, X:Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim 40, 1296-1311 (2001)

[2]

Athans, M:The role and use of the stochastic linear-quadratic-Gaussian problem in control system design.IEEE Trans. Automat. Control 16, 529-552 (1971)

[3]

Ben-Israel, A, Greville, TNE:Generalized Inverses:Theory and Applications. Pure and Applied Mathematics. Wiley-Interscience[John Wiley & Sons], New York-London-Sydney (1974)

[4]

Bensoussan, A:Lectures on stochastic control. In:Nonlinear Filtering and Stochastic Control. Lecture Notes in Math, vol. 972, pp. 1-62. Springer-Verlag, Berlin (1981)

[5]

Bismut, J-M:Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 14, 419-444 (1976)

[6]

Bismut, J-M:Contrôle des systèmes linéaires quadratiques:applications de l'intégrale stochastique. In:Séminaire de Probabilités XII, Université de Strasbourg 1976/77, Lecture Notes in Math, vol. 649, pp. 180-264. Springer-Verlag, Berlin (1978)

[7]

Briand, PH, Delyon, B, Hu, Y, Pardoux, E, Stoica, L:Lp solutions of backward stochastic differential equations. Stochastic Process. Appl 108, 109-129 (2003)

[8]

Chen, S, Li, X, Zhou, X:Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim 36, 1685-1702 (1998)

[9]

Davis, MHA:Linear Estimation and Stochastic Control. Chapman and Hall Mathematics Series. Chapman and Hall, London; Halsted Press[John Wiley & Sons], New York (1977)

[10]

Delbaen, F, Tang, S:Harmonic analysis of stochastic equations and backward stochastic differential equations. Probab. Theory Relat. Fields 146, 291-336 (2010)

[11]

Frei, C, dos Reis, G:A financial market with interacting investors:does an equilibrium exist? Math. Finan.Econ 4, 161-182 (2011)

[12]

Kalman, RE:Contributions to the theory of optimal control. Bol. Soc. Mat. Mexicana 5, 102-119 (1960)

[13]

Lü, Q, Zhang, X:Well-posedness of backward stochastic differential equations with general filtration. J. Diff. Equations 254, 3200-3227 (2013)

[14]

Lü, Q, Zhang, X:General Pontryagin-Type Stochastic Maximum Principle and Backward StochasticEvolution Equations in Infinite Dimensions. Springer Briefs in Mathematics. Springer, Cham (2014)

[15]

Lü, Q, Zhang, X:Transposition method for backward stochastic evolution equations revisited, and its application. Math. Control Relat. Fields 5, 529-555 (2015)

[16]

Lü, Q, Zhang, X:Optimal feedback for stochastic linear quadratic control and backward stochastic Riccati equations in infinite dimensions. (2017). Preprint Pardoux, E, Peng, S:Adapted solution of backward stochastic equation. Systems Control Lett 14, 55-61(1990)

[17]

Peng, S:Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim 30, 284-304 (1992)

[18]

Pham, H:Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications. (2017). arXiv:1604.06609v1

[19]

Protter, PE:Stochastic Integration and Differential Equations. Stochastic Modelling and Applied Probability, vol. 21. Springer-Verlag, Berlin (2005)

[20]

Reid, WT:A matrix differential equation of Riccati type. Amer. J. Math 68, 237-246 (1946)

[21]

Sun, J, Yong, J:Linear quadratic stochastic differential games:open-loop and closed-loop saddle points.SIAM J. Control Optim 52, 4082-4121 (2014)

[22]

Tang, S:General linear quadratic optimal stochastic control problems with random coefficients:linear stochastic Hamilton systems and backward stochastic Riccati equations. SIAM J. Control Optim 42, 53-75 (2003)

[23]

Tang, S:Dynamic programming for general linear quadratic optimal stochastic control with random coefficients. SIAM J. Control Optim 53, 1082-1106 (2015)

[24]

Wonham, WM:On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681-697 (1968)

[25]

Wonham, WM:Linear Multivariable Control, a Geometric Approach. Applications of Mathematics, vol. 10. Springer-Verlag, New York (1985)

[26]

Yong, J, Lou, H:A Concise Course on Optimal Control Theory. Higher Education Press, Beijing (2006).(In Chinese)

[27]

Yong, J, Zhou, XY:Stochastic Controls:Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000)

[1]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control and Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028
[2]

Renzhi Qiu, Shanjian Tang. The Cauchy problem of Backward Stochastic Super-Parabolic Equations with Quadratic Growth. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 3-. doi: 10.1186/s41546-019-0037-3
[3]

Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems and Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002
[4]

Jasmina Djordjević, Svetlana Janković. Reflected backward stochastic differential equations with perturbations. Discrete and Continuous Dynamical Systems, 2018, 38 (4) : 1833-1848. doi: 10.3934/dcds.2018075
[5]

Jan A. Van Casteren. On backward stochastic differential equations in infinite dimensions. Discrete and Continuous Dynamical Systems - S, 2013, 6 (3) : 803-824. doi: 10.3934/dcdss.2013.6.803
[6]

Joscha Diehl, Jianfeng Zhang. Backward stochastic differential equations with Young drift. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 5-. doi: 10.1186/s41546-017-0016-5
[7]

Ishak Alia. Time-inconsistent stochastic optimal control problems: a backward stochastic partial differential equations approach. Mathematical Control and Related Fields, 2020, 10 (4) : 785-826. doi: 10.3934/mcrf.2020020
[8]

Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115
[9]

Ying Hu, Shanjian Tang. Switching game of backward stochastic differential equations and associated system of obliquely reflected backward stochastic differential equations. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5447-5465. doi: 10.3934/dcds.2015.35.5447
[10]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control and Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[11]

Editorial Office. Retraction: Xiao-Qian Jiang and Lun-Chuan Zhang, A pricing option approach based on backward stochastic differential equation theory. Discrete and Continuous Dynamical Systems - S, 2019, 12 (4&5) : 969-969. doi: 10.3934/dcdss.2019065
[12]

Shaokuan Chen, Shanjian Tang. Semi-linear backward stochastic integral partial differential equations driven by a Brownian motion and a Poisson point process. Mathematical Control and Related Fields, 2015, 5 (3) : 401-434. doi: 10.3934/mcrf.2015.5.401
[13]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[14]

Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control and Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022
[15]

Linda J. S. Allen, P. van den Driessche. Stochastic epidemic models with a backward bifurcation. Mathematical Biosciences & Engineering, 2006, 3 (3) : 445-458. doi: 10.3934/mbe.2006.3.445
[16]

Monika Eisenmann, Etienne Emmrich, Volker Mehrmann. Convergence of the backward Euler scheme for the operator-valued Riccati differential equation with semi-definite data. Evolution Equations and Control Theory, 2019, 8 (2) : 315-342. doi: 10.3934/eect.2019017
[17]

Qi Zhang, Huaizhong Zhao. Backward doubly stochastic differential equations with polynomial growth coefficients. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5285-5315. doi: 10.3934/dcds.2015.35.5285
[18]

Yufeng Shi, Qingfeng Zhu. A Kneser-type theorem for backward doubly stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1565-1579. doi: 10.3934/dcdsb.2010.14.1565
[19]

Yanqing Wang. A semidiscrete Galerkin scheme for backward stochastic parabolic differential equations. Mathematical Control and Related Fields, 2016, 6 (3) : 489-515. doi: 10.3934/mcrf.2016013
[20]

Weidong Zhao, Jinlei Wang, Shige Peng. Error estimates of the $\theta$-scheme for backward stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2009, 12 (4) : 905-924. doi: 10.3934/dcdsb.2009.12.905

 Impact Factor: 

Tools

Metrics

  • PDF downloads (28)
  • HTML views (523)
  • Cited by (6)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy