doi: 10.3934/mcrf.2020002

Necessary condition for optimal control of doubly stochastic systems

School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

* Corresponding author: Liangquan Zhang

Received  November 2018 Revised  March 2019 Published  November 2019

Fund Project: The first author is supported partly by the National Nature Science Foundation of China (Grant No. 11701040, 61871058, 11871010 & 61603049) and the Fundamental Research Funds for the Central Universities (No.2019XD-A11).
The second author is supported partly by the National Nature Science Foundation of China (Grant No. 11871010 & 11471051).
The third author is supported partly by the National Nature Science Foundation of China (Grant No. 11501046)

The aim of this paper is to establish a necessary condition for optimal stochastic controls where the systems governed by forward-backward doubly stochastic differential equations (FBDSDEs in short). The control constraints need not to be convex. This condition is described by two kinds of new adjoint processes containing two Brownian motions, corresponding to the forward and backward components and a maximum condition on the Hamiltonian. The proof of the main result is based on spike's variational principle, duality technique and delicate estimates on the state and the adjoint processes with respect to the control variable. An example is provided for illustration.

Citation: Liangquan Zhang, Qing Zhou, Juan Yang. Necessary condition for optimal control of doubly stochastic systems. Mathematical Control & Related Fields, doi: 10.3934/mcrf.2020002
References:
[1]

A. Aman, $L^{p}$-solutions of backward doubly stochastic differential equations, Stoch. Dyn., 12 (2012), 19pp. doi: 10.1142/S0219493711500250.  Google Scholar

[2]

J. Bismut, Théorie Probabiliste du Contrôle des Diffusions, Mem. Amer. Math. Soc., 4, Providence, RI, 1976. doi: 10.1090/memo/0167.  Google Scholar

[3]

J. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Rev., 20 (1978), 62-78.  doi: 10.1137/1020004.  Google Scholar

[4]

A. Bensoussan and J. L. Lions, Applications des Inéquations Variationnelles en Contrôle Stochastique, Méthodes Mathématiques de l'Informatique, 6, Dunod, Paris, 1978.  Google Scholar

[5]

B. BoufoussiJ. V. Casteren and N. Mrhardy, Generalized backward doubly stochastic differential equations and SPDEs with nonlinear Neumann boundary conditions, Bernoulli, 13 (2007), 423-446.  doi: 10.3150/07-BEJ5092.  Google Scholar

[6]

R. Buchdahn and J. Ma, Stochastic viscosity solutions for nonlinear stochastic partial differential equations. Part I, Stochastic Process. Appl., 93 (2001), 181-204.  doi: 10.1016/S0304-4149(00)00093-4.  Google Scholar

[7]

R. Buckdahn and J. Ma, Pathwise stochastic Taylor expansions and stochastic viscosity solutions for fully nonlinear stochastic PDEs, Ann. Probab., 30 (2002), 1131-1171.  doi: 10.1214/aop/1029867123.  Google Scholar

[8]

S. Bahlali and B. Gherbal, Optimality conditions of controlled backward doubly stochastic differential equations, Random Oper. Stoch. Equ., 18 (2010), 247-265.  doi: 10.1515/ROSE.2010.014.  Google Scholar

[9]

A. Chala, On optimal control problem for backward stochastic doubly systems, ISRN Appl. Math., 2014 (2014), Art. ID 903912, 10pp. doi: 10.1155/2014/903912.  Google Scholar

[10]

L. Denis, A general analytical result for non-linear SPDE's and applications, Electron. J. Probab., 9 (2004), 674-709.  doi: 10.1214/EJP.v9-223.  Google Scholar

[11]

L. Denis, Solutions of stochastic partial differential equations considered as Dirichlet processes, Bernoulli, 10 (2004), 783-827.  doi: 10.3150/bj/1099579156.  Google Scholar

[12]

K. DuJ. Qiu and S. Tang, $L^{p}$ theory for super-parabolic backward stochastic partial differential equations in the whole space, Appl. Math. Optim., 65 (2012), 175-219.  doi: 10.1007/s00245-011-9154-9.  Google Scholar

[13]

N. Englezos and I. Karatzas, Utility maximization with habit formation: Dynamic programming and stochastic PDEs, SIAM J. Control Optim., 48 (2009), 481-520.  doi: 10.1137/070686998.  Google Scholar

[14]

Y. HanS. Peng and Z. Wu, Maximum principle for backward doubly stochastic control systems with applications, SIAM J. Control Optim., 48 (2010), 4224-4241.  doi: 10.1137/080743561.  Google Scholar

[15]

Y. HuJ. Ma and J. Yong, On semi-linear degenerate backward stochastic partial differential equations, Probab. Theory Related Fields, 123 (2002), 381-411.  doi: 10.1007/s004400100193.  Google Scholar

[16]

M. Hu, Stochastic global maximum principle for optimization with recursive utilities, Probab. Uncertain. Quant. Risk, 2 (2017), 20pp. doi: 10.1186/s41546-017-0014-7.  Google Scholar

[17]

N. Ichihara, Homogenization problem for stochastic partial differential equations of Zakai type, Stoch. Stoch. Rep., 76 (2004), 243-266.  doi: 10.1080/10451120410001714107.  Google Scholar

[18]

S. Ji, Q. Wei and X. Zhang, A maximum principle for controlled time-symmetric forward-backward doubly stochastic differential equation with initial-terminal state constraints, Abstr. Appl. Anal., 2012 (2012), Art. ID 537376, 29pp. doi: 10.1155/2012/537376.  Google Scholar

[19]

N. E. KarouiS. Peng and M. C. Quenez, Backward stochastic differential equations in finance, Math. Finance, 7 (1997), 1-71.  doi: 10.1111/1467-9965.00022.  Google Scholar

[20]

N. V. Krylov and B. L. Rozovski, Stochastic evolution equations, J. Sov. Math., 16 (1981), 1233-1277. doi: 10.1007/BF01084893.  Google Scholar

[21]

A. M. Márquez-Durán and J. Real, Some results on nonlinear backward stochastic evolution equations, Stochastic Anal. Appl., 22 (2004), 1273-1293.  doi: 10.1081/SAP-200026462.  Google Scholar

[22]

A. Matoussi and L. Stoica, The obstacle problem for quasilinear stochastic PDE's, Ann. Probab., 38 (2010), 1143-1179.  doi: 10.1214/09-AOP507.  Google Scholar

[23]

D. Nualart and E. Pardoux, Stochastic calculus with anticipating integrands, Probab. Theory Related Fields, 78 (1988), 535-581.  doi: 10.1007/BF00353876.  Google Scholar

[24]

B. ØsendalA. Sulem and T. Zhang, Singular control and optimal stopping of SPDEs, and backward SPDEs with reflection, Math. Oper. Res., 39 (2014), 464-486.  doi: 10.1287/moor.2013.0602.  Google Scholar

[25]

E. Pardoux, Stochastic partial differential equations and filtering of diffusion processes, Stochastics, 3 (1979), 127-167.  doi: 10.1080/17442507908833142.  Google Scholar

[26]

E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett., 14 (1990), 55-61.  doi: 10.1016/0167-6911(90)90082-6.  Google Scholar

[27]

E. Pardoux and S. Peng, Backward doubly stochastic differential equations and systems of quasilinear SPDEs, Probab. Theory Related Fields, 98 (1994), 209-227.  doi: 10.1007/BF01192514.  Google Scholar

[28]

S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.  Google Scholar

[29]

S. Peng, Stochastic Hamilton-Jacobi-Bellman equations, SIAM J. Control Optim., 30 (1992), 284-304.  doi: 10.1137/0330018.  Google Scholar

[30]

J. Qiu and S. Tang, Maximum principles for backward stochastic partial differential equations, J. Funct. Anal., 262 (2012), 2436-2480.  doi: 10.1016/j.jfa.2011.12.002.  Google Scholar

[31]

J. Qiu and S. Tang, On backward doubly stochastic differential evolutionary system, preprint, arXiv: math/1309.4152.  Google Scholar

[32]

S. Tang and X. Li, Necessary conditions for optimal control of stochastic systems with random jumps, SIAM J. Control Optim., 32 (1994), 1447-1475.  doi: 10.1137/S0363012992233858.  Google Scholar

[33]

S. Tang, The maximum principle for partially observed optimal control of stochastic differential equations, SIAM J. Control Optim., 36 (1998), 1596-1617.  doi: 10.1137/S0363012996313100.  Google Scholar

[34]

S. Tang, On backward stochastic partial differential equations, 34th SPA Conference, Osaka, 2010. Google Scholar

[35]

Q. Zhang and H. Zhao, Stationary solutions of SPDEs and infinite horizon BDSDEs, J. Funct. Anal., 252 (2007), 171-219.  doi: 10.1016/j.jfa.2007.06.019.  Google Scholar

[36]

Z. Wu, Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems, Systems Sci. Math. Sci., 11 (1998), 249-259.   Google Scholar

[37]

Z. Wu, A general maximum principle for optimal control of forward-backward stochastic systems, Automatica J. IFAC, 49 (2013), 1473-1480.  doi: 10.1016/j.automatica.2013.02.005.  Google Scholar

[38]

W. Wang and B. Liu, Second-order Taylor expansion for backward doubly stochastic control system, Internat. J. Control, 86 (2013), 942-952.  doi: 10.1080/00207179.2013.766940.  Google Scholar

[39]

W. Wang and B. Liu, Necessary conditions for backward doubly stochastic control system, Electronic J. Math. Anal. Appl., 2 (2013), 260-272.   Google Scholar

[40]

X. Zhou, A duality analysis on stochastic partial differential equations, J. Funct. Anal., 103 (1992), 275-293.  doi: 10.1016/0022-1236(92)90122-Y.  Google Scholar

[41]

J. Yong and X. Y. Zhou, Stochastic Controls. Hamiltonian Systems and HJB Equations, Applications of Mathematics, 43, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[42]

L. Zhang and Y. Shi, Maximum principle for forward-backward doubly stochastic control systems and applications, ESAIM Control Optim. Calc. Var., 17 (2011), 1174-1197.  doi: 10.1051/cocv/2010042.  Google Scholar

show all references

References:
[1]

A. Aman, $L^{p}$-solutions of backward doubly stochastic differential equations, Stoch. Dyn., 12 (2012), 19pp. doi: 10.1142/S0219493711500250.  Google Scholar

[2]

J. Bismut, Théorie Probabiliste du Contrôle des Diffusions, Mem. Amer. Math. Soc., 4, Providence, RI, 1976. doi: 10.1090/memo/0167.  Google Scholar

[3]

J. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Rev., 20 (1978), 62-78.  doi: 10.1137/1020004.  Google Scholar

[4]

A. Bensoussan and J. L. Lions, Applications des Inéquations Variationnelles en Contrôle Stochastique, Méthodes Mathématiques de l'Informatique, 6, Dunod, Paris, 1978.  Google Scholar

[5]

B. BoufoussiJ. V. Casteren and N. Mrhardy, Generalized backward doubly stochastic differential equations and SPDEs with nonlinear Neumann boundary conditions, Bernoulli, 13 (2007), 423-446.  doi: 10.3150/07-BEJ5092.  Google Scholar

[6]

R. Buchdahn and J. Ma, Stochastic viscosity solutions for nonlinear stochastic partial differential equations. Part I, Stochastic Process. Appl., 93 (2001), 181-204.  doi: 10.1016/S0304-4149(00)00093-4.  Google Scholar

[7]

R. Buckdahn and J. Ma, Pathwise stochastic Taylor expansions and stochastic viscosity solutions for fully nonlinear stochastic PDEs, Ann. Probab., 30 (2002), 1131-1171.  doi: 10.1214/aop/1029867123.  Google Scholar

[8]

S. Bahlali and B. Gherbal, Optimality conditions of controlled backward doubly stochastic differential equations, Random Oper. Stoch. Equ., 18 (2010), 247-265.  doi: 10.1515/ROSE.2010.014.  Google Scholar

[9]

A. Chala, On optimal control problem for backward stochastic doubly systems, ISRN Appl. Math., 2014 (2014), Art. ID 903912, 10pp. doi: 10.1155/2014/903912.  Google Scholar

[10]

L. Denis, A general analytical result for non-linear SPDE's and applications, Electron. J. Probab., 9 (2004), 674-709.  doi: 10.1214/EJP.v9-223.  Google Scholar

[11]

L. Denis, Solutions of stochastic partial differential equations considered as Dirichlet processes, Bernoulli, 10 (2004), 783-827.  doi: 10.3150/bj/1099579156.  Google Scholar

[12]

K. DuJ. Qiu and S. Tang, $L^{p}$ theory for super-parabolic backward stochastic partial differential equations in the whole space, Appl. Math. Optim., 65 (2012), 175-219.  doi: 10.1007/s00245-011-9154-9.  Google Scholar

[13]

N. Englezos and I. Karatzas, Utility maximization with habit formation: Dynamic programming and stochastic PDEs, SIAM J. Control Optim., 48 (2009), 481-520.  doi: 10.1137/070686998.  Google Scholar

[14]

Y. HanS. Peng and Z. Wu, Maximum principle for backward doubly stochastic control systems with applications, SIAM J. Control Optim., 48 (2010), 4224-4241.  doi: 10.1137/080743561.  Google Scholar

[15]

Y. HuJ. Ma and J. Yong, On semi-linear degenerate backward stochastic partial differential equations, Probab. Theory Related Fields, 123 (2002), 381-411.  doi: 10.1007/s004400100193.  Google Scholar

[16]

M. Hu, Stochastic global maximum principle for optimization with recursive utilities, Probab. Uncertain. Quant. Risk, 2 (2017), 20pp. doi: 10.1186/s41546-017-0014-7.  Google Scholar

[17]

N. Ichihara, Homogenization problem for stochastic partial differential equations of Zakai type, Stoch. Stoch. Rep., 76 (2004), 243-266.  doi: 10.1080/10451120410001714107.  Google Scholar

[18]

S. Ji, Q. Wei and X. Zhang, A maximum principle for controlled time-symmetric forward-backward doubly stochastic differential equation with initial-terminal state constraints, Abstr. Appl. Anal., 2012 (2012), Art. ID 537376, 29pp. doi: 10.1155/2012/537376.  Google Scholar

[19]

N. E. KarouiS. Peng and M. C. Quenez, Backward stochastic differential equations in finance, Math. Finance, 7 (1997), 1-71.  doi: 10.1111/1467-9965.00022.  Google Scholar

[20]

N. V. Krylov and B. L. Rozovski, Stochastic evolution equations, J. Sov. Math., 16 (1981), 1233-1277. doi: 10.1007/BF01084893.  Google Scholar

[21]

A. M. Márquez-Durán and J. Real, Some results on nonlinear backward stochastic evolution equations, Stochastic Anal. Appl., 22 (2004), 1273-1293.  doi: 10.1081/SAP-200026462.  Google Scholar

[22]

A. Matoussi and L. Stoica, The obstacle problem for quasilinear stochastic PDE's, Ann. Probab., 38 (2010), 1143-1179.  doi: 10.1214/09-AOP507.  Google Scholar

[23]

D. Nualart and E. Pardoux, Stochastic calculus with anticipating integrands, Probab. Theory Related Fields, 78 (1988), 535-581.  doi: 10.1007/BF00353876.  Google Scholar

[24]

B. ØsendalA. Sulem and T. Zhang, Singular control and optimal stopping of SPDEs, and backward SPDEs with reflection, Math. Oper. Res., 39 (2014), 464-486.  doi: 10.1287/moor.2013.0602.  Google Scholar

[25]

E. Pardoux, Stochastic partial differential equations and filtering of diffusion processes, Stochastics, 3 (1979), 127-167.  doi: 10.1080/17442507908833142.  Google Scholar

[26]

E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett., 14 (1990), 55-61.  doi: 10.1016/0167-6911(90)90082-6.  Google Scholar

[27]

E. Pardoux and S. Peng, Backward doubly stochastic differential equations and systems of quasilinear SPDEs, Probab. Theory Related Fields, 98 (1994), 209-227.  doi: 10.1007/BF01192514.  Google Scholar

[28]

S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.  Google Scholar

[29]

S. Peng, Stochastic Hamilton-Jacobi-Bellman equations, SIAM J. Control Optim., 30 (1992), 284-304.  doi: 10.1137/0330018.  Google Scholar

[30]

J. Qiu and S. Tang, Maximum principles for backward stochastic partial differential equations, J. Funct. Anal., 262 (2012), 2436-2480.  doi: 10.1016/j.jfa.2011.12.002.  Google Scholar

[31]

J. Qiu and S. Tang, On backward doubly stochastic differential evolutionary system, preprint, arXiv: math/1309.4152.  Google Scholar

[32]

S. Tang and X. Li, Necessary conditions for optimal control of stochastic systems with random jumps, SIAM J. Control Optim., 32 (1994), 1447-1475.  doi: 10.1137/S0363012992233858.  Google Scholar

[33]

S. Tang, The maximum principle for partially observed optimal control of stochastic differential equations, SIAM J. Control Optim., 36 (1998), 1596-1617.  doi: 10.1137/S0363012996313100.  Google Scholar

[34]

S. Tang, On backward stochastic partial differential equations, 34th SPA Conference, Osaka, 2010. Google Scholar

[35]

Q. Zhang and H. Zhao, Stationary solutions of SPDEs and infinite horizon BDSDEs, J. Funct. Anal., 252 (2007), 171-219.  doi: 10.1016/j.jfa.2007.06.019.  Google Scholar

[36]

Z. Wu, Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems, Systems Sci. Math. Sci., 11 (1998), 249-259.   Google Scholar

[37]

Z. Wu, A general maximum principle for optimal control of forward-backward stochastic systems, Automatica J. IFAC, 49 (2013), 1473-1480.  doi: 10.1016/j.automatica.2013.02.005.  Google Scholar

[38]

W. Wang and B. Liu, Second-order Taylor expansion for backward doubly stochastic control system, Internat. J. Control, 86 (2013), 942-952.  doi: 10.1080/00207179.2013.766940.  Google Scholar

[39]

W. Wang and B. Liu, Necessary conditions for backward doubly stochastic control system, Electronic J. Math. Anal. Appl., 2 (2013), 260-272.   Google Scholar

[40]

X. Zhou, A duality analysis on stochastic partial differential equations, J. Funct. Anal., 103 (1992), 275-293.  doi: 10.1016/0022-1236(92)90122-Y.  Google Scholar

[41]

J. Yong and X. Y. Zhou, Stochastic Controls. Hamiltonian Systems and HJB Equations, Applications of Mathematics, 43, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[42]

L. Zhang and Y. Shi, Maximum principle for forward-backward doubly stochastic control systems and applications, ESAIM Control Optim. Calc. Var., 17 (2011), 1174-1197.  doi: 10.1051/cocv/2010042.  Google Scholar

[1]

Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control & Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613

[2]

Haiyang Wang, Jianfeng Zhang. Forward backward SDEs in weak formulation. Mathematical Control & Related Fields, 2018, 8 (3&4) : 1021-1049. doi: 10.3934/mcrf.2018044

[3]

Fabio Paronetto. A Harnack type inequality and a maximum principle for an elliptic-parabolic and forward-backward parabolic De Giorgi class. Discrete & Continuous Dynamical Systems - S, 2017, 10 (4) : 853-866. doi: 10.3934/dcdss.2017043

[4]

Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control & Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195

[5]

Md. Haider Ali Biswas, Maria do Rosário de Pinho. A nonsmooth maximum principle for optimal control problems with state and mixed constraints - convex case. Conference Publications, 2011, 2011 (Special) : 174-183. doi: 10.3934/proc.2011.2011.174

[6]

Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161

[7]

Monia Karouf. Reflected solutions of backward doubly SDEs driven by Brownian motion and Poisson random measure. Discrete & Continuous Dynamical Systems - A, 2019, 39 (10) : 5571-5601. doi: 10.3934/dcds.2019245

[8]

Yu Fu, Weidong Zhao, Tao Zhou. Efficient spectral sparse grid approximations for solving multi-dimensional forward backward SDEs. Discrete & Continuous Dynamical Systems - B, 2017, 22 (9) : 3439-3458. doi: 10.3934/dcdsb.2017174

[9]

Jiongmin Yong. Forward-backward evolution equations and applications. Mathematical Control & Related Fields, 2016, 6 (4) : 653-704. doi: 10.3934/mcrf.2016019

[10]

Fabio Paronetto. Elliptic approximation of forward-backward parabolic equations. Communications on Pure & Applied Analysis, 2020, 19 (2) : 1017-1036. doi: 10.3934/cpaa.2020047

[11]

Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems & Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002

[12]

H. O. Fattorini. The maximum principle for linear infinite dimensional control systems with state constraints. Discrete & Continuous Dynamical Systems - A, 1995, 1 (1) : 77-101. doi: 10.3934/dcds.1995.1.77

[13]

Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control & Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022

[14]

Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial & Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067

[15]

Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581

[16]

Qi Zhang, Huaizhong Zhao. Backward doubly stochastic differential equations with polynomial growth coefficients. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5285-5315. doi: 10.3934/dcds.2015.35.5285

[17]

Yufeng Shi, Qingfeng Zhu. A Kneser-type theorem for backward doubly stochastic differential equations. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1565-1579. doi: 10.3934/dcdsb.2010.14.1565

[18]

G. Bellettini, Giorgio Fusco, Nicola Guglielmi. A concept of solution and numerical experiments for forward-backward diffusion equations. Discrete & Continuous Dynamical Systems - A, 2006, 16 (4) : 783-842. doi: 10.3934/dcds.2006.16.783

[19]

Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115

[20]

Lianzhang Bao, Zhengfang Zhou. Traveling wave in backward and forward parabolic equations from population dynamics. Discrete & Continuous Dynamical Systems - B, 2014, 19 (6) : 1507-1522. doi: 10.3934/dcdsb.2014.19.1507

2018 Impact Factor: 1.292

Article outline

[Back to Top]