June  2020, 10(2): 379-403. doi: 10.3934/mcrf.2020002

Necessary condition for optimal control of doubly stochastic systems

School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

* Corresponding author: Liangquan Zhang

Received  November 2018 Revised  March 2019 Published  June 2020 Early access  November 2019

Fund Project: The first author is supported partly by the National Nature Science Foundation of China (Grant No. 11701040, 61871058, 11871010 & 61603049) and the Fundamental Research Funds for the Central Universities (No.2019XD-A11).
The second author is supported partly by the National Nature Science Foundation of China (Grant No. 11871010 & 11471051).
The third author is supported partly by the National Nature Science Foundation of China (Grant No. 11501046)

The aim of this paper is to establish a necessary condition for optimal stochastic controls where the systems governed by forward-backward doubly stochastic differential equations (FBDSDEs in short). The control constraints need not to be convex. This condition is described by two kinds of new adjoint processes containing two Brownian motions, corresponding to the forward and backward components and a maximum condition on the Hamiltonian. The proof of the main result is based on spike's variational principle, duality technique and delicate estimates on the state and the adjoint processes with respect to the control variable. An example is provided for illustration.

Citation: Liangquan Zhang, Qing Zhou, Juan Yang. Necessary condition for optimal control of doubly stochastic systems. Mathematical Control and Related Fields, 2020, 10 (2) : 379-403. doi: 10.3934/mcrf.2020002
References:
[1]

A. Aman, $L^{p}$-solutions of backward doubly stochastic differential equations, Stoch. Dyn., 12 (2012), 19pp. doi: 10.1142/S0219493711500250.

[2]

J. Bismut, Théorie Probabiliste du Contrôle des Diffusions, Mem. Amer. Math. Soc., 4, Providence, RI, 1976. doi: 10.1090/memo/0167.

[3]

J. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Rev., 20 (1978), 62-78.  doi: 10.1137/1020004.

[4]

A. Bensoussan and J. L. Lions, Applications des Inéquations Variationnelles en Contrôle Stochastique, Méthodes Mathématiques de l'Informatique, 6, Dunod, Paris, 1978.

[5]

B. BoufoussiJ. V. Casteren and N. Mrhardy, Generalized backward doubly stochastic differential equations and SPDEs with nonlinear Neumann boundary conditions, Bernoulli, 13 (2007), 423-446.  doi: 10.3150/07-BEJ5092.

[6]

R. Buchdahn and J. Ma, Stochastic viscosity solutions for nonlinear stochastic partial differential equations. Part I, Stochastic Process. Appl., 93 (2001), 181-204.  doi: 10.1016/S0304-4149(00)00093-4.

[7]

R. Buckdahn and J. Ma, Pathwise stochastic Taylor expansions and stochastic viscosity solutions for fully nonlinear stochastic PDEs, Ann. Probab., 30 (2002), 1131-1171.  doi: 10.1214/aop/1029867123.

[8]

S. Bahlali and B. Gherbal, Optimality conditions of controlled backward doubly stochastic differential equations, Random Oper. Stoch. Equ., 18 (2010), 247-265.  doi: 10.1515/ROSE.2010.014.

[9]

A. Chala, On optimal control problem for backward stochastic doubly systems, ISRN Appl. Math., 2014 (2014), Art. ID 903912, 10pp. doi: 10.1155/2014/903912.

[10]

L. Denis, A general analytical result for non-linear SPDE's and applications, Electron. J. Probab., 9 (2004), 674-709.  doi: 10.1214/EJP.v9-223.

[11]

L. Denis, Solutions of stochastic partial differential equations considered as Dirichlet processes, Bernoulli, 10 (2004), 783-827.  doi: 10.3150/bj/1099579156.

[12]

K. DuJ. Qiu and S. Tang, $L^{p}$ theory for super-parabolic backward stochastic partial differential equations in the whole space, Appl. Math. Optim., 65 (2012), 175-219.  doi: 10.1007/s00245-011-9154-9.

[13]

N. Englezos and I. Karatzas, Utility maximization with habit formation: Dynamic programming and stochastic PDEs, SIAM J. Control Optim., 48 (2009), 481-520.  doi: 10.1137/070686998.

[14]

Y. HanS. Peng and Z. Wu, Maximum principle for backward doubly stochastic control systems with applications, SIAM J. Control Optim., 48 (2010), 4224-4241.  doi: 10.1137/080743561.

[15]

Y. HuJ. Ma and J. Yong, On semi-linear degenerate backward stochastic partial differential equations, Probab. Theory Related Fields, 123 (2002), 381-411.  doi: 10.1007/s004400100193.

[16]

M. Hu, Stochastic global maximum principle for optimization with recursive utilities, Probab. Uncertain. Quant. Risk, 2 (2017), 20pp. doi: 10.1186/s41546-017-0014-7.

[17]

N. Ichihara, Homogenization problem for stochastic partial differential equations of Zakai type, Stoch. Stoch. Rep., 76 (2004), 243-266.  doi: 10.1080/10451120410001714107.

[18]

S. Ji, Q. Wei and X. Zhang, A maximum principle for controlled time-symmetric forward-backward doubly stochastic differential equation with initial-terminal state constraints, Abstr. Appl. Anal., 2012 (2012), Art. ID 537376, 29pp. doi: 10.1155/2012/537376.

[19]

N. E. KarouiS. Peng and M. C. Quenez, Backward stochastic differential equations in finance, Math. Finance, 7 (1997), 1-71.  doi: 10.1111/1467-9965.00022.

[20]

N. V. Krylov and B. L. Rozovski, Stochastic evolution equations, J. Sov. Math., 16 (1981), 1233-1277. doi: 10.1007/BF01084893.

[21]

A. M. Márquez-Durán and J. Real, Some results on nonlinear backward stochastic evolution equations, Stochastic Anal. Appl., 22 (2004), 1273-1293.  doi: 10.1081/SAP-200026462.

[22]

A. Matoussi and L. Stoica, The obstacle problem for quasilinear stochastic PDE's, Ann. Probab., 38 (2010), 1143-1179.  doi: 10.1214/09-AOP507.

[23]

D. Nualart and E. Pardoux, Stochastic calculus with anticipating integrands, Probab. Theory Related Fields, 78 (1988), 535-581.  doi: 10.1007/BF00353876.

[24]

B. ØsendalA. Sulem and T. Zhang, Singular control and optimal stopping of SPDEs, and backward SPDEs with reflection, Math. Oper. Res., 39 (2014), 464-486.  doi: 10.1287/moor.2013.0602.

[25]

E. Pardoux, Stochastic partial differential equations and filtering of diffusion processes, Stochastics, 3 (1979), 127-167.  doi: 10.1080/17442507908833142.

[26]

E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett., 14 (1990), 55-61.  doi: 10.1016/0167-6911(90)90082-6.

[27]

E. Pardoux and S. Peng, Backward doubly stochastic differential equations and systems of quasilinear SPDEs, Probab. Theory Related Fields, 98 (1994), 209-227.  doi: 10.1007/BF01192514.

[28]

S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.

[29]

S. Peng, Stochastic Hamilton-Jacobi-Bellman equations, SIAM J. Control Optim., 30 (1992), 284-304.  doi: 10.1137/0330018.

[30]

J. Qiu and S. Tang, Maximum principles for backward stochastic partial differential equations, J. Funct. Anal., 262 (2012), 2436-2480.  doi: 10.1016/j.jfa.2011.12.002.

[31]

J. Qiu and S. Tang, On backward doubly stochastic differential evolutionary system, preprint, arXiv: math/1309.4152.

[32]

S. Tang and X. Li, Necessary conditions for optimal control of stochastic systems with random jumps, SIAM J. Control Optim., 32 (1994), 1447-1475.  doi: 10.1137/S0363012992233858.

[33]

S. Tang, The maximum principle for partially observed optimal control of stochastic differential equations, SIAM J. Control Optim., 36 (1998), 1596-1617.  doi: 10.1137/S0363012996313100.

[34]

S. Tang, On backward stochastic partial differential equations, 34th SPA Conference, Osaka, 2010.

[35]

Q. Zhang and H. Zhao, Stationary solutions of SPDEs and infinite horizon BDSDEs, J. Funct. Anal., 252 (2007), 171-219.  doi: 10.1016/j.jfa.2007.06.019.

[36]

Z. Wu, Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems, Systems Sci. Math. Sci., 11 (1998), 249-259. 

[37]

Z. Wu, A general maximum principle for optimal control of forward-backward stochastic systems, Automatica J. IFAC, 49 (2013), 1473-1480.  doi: 10.1016/j.automatica.2013.02.005.

[38]

W. Wang and B. Liu, Second-order Taylor expansion for backward doubly stochastic control system, Internat. J. Control, 86 (2013), 942-952.  doi: 10.1080/00207179.2013.766940.

[39]

W. Wang and B. Liu, Necessary conditions for backward doubly stochastic control system, Electronic J. Math. Anal. Appl., 2 (2013), 260-272. 

[40]

X. Zhou, A duality analysis on stochastic partial differential equations, J. Funct. Anal., 103 (1992), 275-293.  doi: 10.1016/0022-1236(92)90122-Y.

[41]

J. Yong and X. Y. Zhou, Stochastic Controls. Hamiltonian Systems and HJB Equations, Applications of Mathematics, 43, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[42]

L. Zhang and Y. Shi, Maximum principle for forward-backward doubly stochastic control systems and applications, ESAIM Control Optim. Calc. Var., 17 (2011), 1174-1197.  doi: 10.1051/cocv/2010042.

show all references

References:
[1]

A. Aman, $L^{p}$-solutions of backward doubly stochastic differential equations, Stoch. Dyn., 12 (2012), 19pp. doi: 10.1142/S0219493711500250.

[2]

J. Bismut, Théorie Probabiliste du Contrôle des Diffusions, Mem. Amer. Math. Soc., 4, Providence, RI, 1976. doi: 10.1090/memo/0167.

[3]

J. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Rev., 20 (1978), 62-78.  doi: 10.1137/1020004.

[4]

A. Bensoussan and J. L. Lions, Applications des Inéquations Variationnelles en Contrôle Stochastique, Méthodes Mathématiques de l'Informatique, 6, Dunod, Paris, 1978.

[5]

B. BoufoussiJ. V. Casteren and N. Mrhardy, Generalized backward doubly stochastic differential equations and SPDEs with nonlinear Neumann boundary conditions, Bernoulli, 13 (2007), 423-446.  doi: 10.3150/07-BEJ5092.

[6]

R. Buchdahn and J. Ma, Stochastic viscosity solutions for nonlinear stochastic partial differential equations. Part I, Stochastic Process. Appl., 93 (2001), 181-204.  doi: 10.1016/S0304-4149(00)00093-4.

[7]

R. Buckdahn and J. Ma, Pathwise stochastic Taylor expansions and stochastic viscosity solutions for fully nonlinear stochastic PDEs, Ann. Probab., 30 (2002), 1131-1171.  doi: 10.1214/aop/1029867123.

[8]

S. Bahlali and B. Gherbal, Optimality conditions of controlled backward doubly stochastic differential equations, Random Oper. Stoch. Equ., 18 (2010), 247-265.  doi: 10.1515/ROSE.2010.014.

[9]

A. Chala, On optimal control problem for backward stochastic doubly systems, ISRN Appl. Math., 2014 (2014), Art. ID 903912, 10pp. doi: 10.1155/2014/903912.

[10]

L. Denis, A general analytical result for non-linear SPDE's and applications, Electron. J. Probab., 9 (2004), 674-709.  doi: 10.1214/EJP.v9-223.

[11]

L. Denis, Solutions of stochastic partial differential equations considered as Dirichlet processes, Bernoulli, 10 (2004), 783-827.  doi: 10.3150/bj/1099579156.

[12]

K. DuJ. Qiu and S. Tang, $L^{p}$ theory for super-parabolic backward stochastic partial differential equations in the whole space, Appl. Math. Optim., 65 (2012), 175-219.  doi: 10.1007/s00245-011-9154-9.

[13]

N. Englezos and I. Karatzas, Utility maximization with habit formation: Dynamic programming and stochastic PDEs, SIAM J. Control Optim., 48 (2009), 481-520.  doi: 10.1137/070686998.

[14]

Y. HanS. Peng and Z. Wu, Maximum principle for backward doubly stochastic control systems with applications, SIAM J. Control Optim., 48 (2010), 4224-4241.  doi: 10.1137/080743561.

[15]

Y. HuJ. Ma and J. Yong, On semi-linear degenerate backward stochastic partial differential equations, Probab. Theory Related Fields, 123 (2002), 381-411.  doi: 10.1007/s004400100193.

[16]

M. Hu, Stochastic global maximum principle for optimization with recursive utilities, Probab. Uncertain. Quant. Risk, 2 (2017), 20pp. doi: 10.1186/s41546-017-0014-7.

[17]

N. Ichihara, Homogenization problem for stochastic partial differential equations of Zakai type, Stoch. Stoch. Rep., 76 (2004), 243-266.  doi: 10.1080/10451120410001714107.

[18]

S. Ji, Q. Wei and X. Zhang, A maximum principle for controlled time-symmetric forward-backward doubly stochastic differential equation with initial-terminal state constraints, Abstr. Appl. Anal., 2012 (2012), Art. ID 537376, 29pp. doi: 10.1155/2012/537376.

[19]

N. E. KarouiS. Peng and M. C. Quenez, Backward stochastic differential equations in finance, Math. Finance, 7 (1997), 1-71.  doi: 10.1111/1467-9965.00022.

[20]

N. V. Krylov and B. L. Rozovski, Stochastic evolution equations, J. Sov. Math., 16 (1981), 1233-1277. doi: 10.1007/BF01084893.

[21]

A. M. Márquez-Durán and J. Real, Some results on nonlinear backward stochastic evolution equations, Stochastic Anal. Appl., 22 (2004), 1273-1293.  doi: 10.1081/SAP-200026462.

[22]

A. Matoussi and L. Stoica, The obstacle problem for quasilinear stochastic PDE's, Ann. Probab., 38 (2010), 1143-1179.  doi: 10.1214/09-AOP507.

[23]

D. Nualart and E. Pardoux, Stochastic calculus with anticipating integrands, Probab. Theory Related Fields, 78 (1988), 535-581.  doi: 10.1007/BF00353876.

[24]

B. ØsendalA. Sulem and T. Zhang, Singular control and optimal stopping of SPDEs, and backward SPDEs with reflection, Math. Oper. Res., 39 (2014), 464-486.  doi: 10.1287/moor.2013.0602.

[25]

E. Pardoux, Stochastic partial differential equations and filtering of diffusion processes, Stochastics, 3 (1979), 127-167.  doi: 10.1080/17442507908833142.

[26]

E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett., 14 (1990), 55-61.  doi: 10.1016/0167-6911(90)90082-6.

[27]

E. Pardoux and S. Peng, Backward doubly stochastic differential equations and systems of quasilinear SPDEs, Probab. Theory Related Fields, 98 (1994), 209-227.  doi: 10.1007/BF01192514.

[28]

S. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.

[29]

S. Peng, Stochastic Hamilton-Jacobi-Bellman equations, SIAM J. Control Optim., 30 (1992), 284-304.  doi: 10.1137/0330018.

[30]

J. Qiu and S. Tang, Maximum principles for backward stochastic partial differential equations, J. Funct. Anal., 262 (2012), 2436-2480.  doi: 10.1016/j.jfa.2011.12.002.

[31]

J. Qiu and S. Tang, On backward doubly stochastic differential evolutionary system, preprint, arXiv: math/1309.4152.

[32]

S. Tang and X. Li, Necessary conditions for optimal control of stochastic systems with random jumps, SIAM J. Control Optim., 32 (1994), 1447-1475.  doi: 10.1137/S0363012992233858.

[33]

S. Tang, The maximum principle for partially observed optimal control of stochastic differential equations, SIAM J. Control Optim., 36 (1998), 1596-1617.  doi: 10.1137/S0363012996313100.

[34]

S. Tang, On backward stochastic partial differential equations, 34th SPA Conference, Osaka, 2010.

[35]

Q. Zhang and H. Zhao, Stationary solutions of SPDEs and infinite horizon BDSDEs, J. Funct. Anal., 252 (2007), 171-219.  doi: 10.1016/j.jfa.2007.06.019.

[36]

Z. Wu, Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems, Systems Sci. Math. Sci., 11 (1998), 249-259. 

[37]

Z. Wu, A general maximum principle for optimal control of forward-backward stochastic systems, Automatica J. IFAC, 49 (2013), 1473-1480.  doi: 10.1016/j.automatica.2013.02.005.

[38]

W. Wang and B. Liu, Second-order Taylor expansion for backward doubly stochastic control system, Internat. J. Control, 86 (2013), 942-952.  doi: 10.1080/00207179.2013.766940.

[39]

W. Wang and B. Liu, Necessary conditions for backward doubly stochastic control system, Electronic J. Math. Anal. Appl., 2 (2013), 260-272. 

[40]

X. Zhou, A duality analysis on stochastic partial differential equations, J. Funct. Anal., 103 (1992), 275-293.  doi: 10.1016/0022-1236(92)90122-Y.

[41]

J. Yong and X. Y. Zhou, Stochastic Controls. Hamiltonian Systems and HJB Equations, Applications of Mathematics, 43, Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[42]

L. Zhang and Y. Shi, Maximum principle for forward-backward doubly stochastic control systems and applications, ESAIM Control Optim. Calc. Var., 17 (2011), 1174-1197.  doi: 10.1051/cocv/2010042.

[1]

Adel Chala, Dahbia Hafayed. On stochastic maximum principle for risk-sensitive of fully coupled forward-backward stochastic control of mean-field type with application. Evolution Equations and Control Theory, 2020, 9 (3) : 817-843. doi: 10.3934/eect.2020035

[2]

Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control and Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613

[3]

Haiyang Wang, Jianfeng Zhang. Forward backward SDEs in weak formulation. Mathematical Control and Related Fields, 2018, 8 (3&4) : 1021-1049. doi: 10.3934/mcrf.2018044

[4]

Fabio Paronetto. A Harnack type inequality and a maximum principle for an elliptic-parabolic and forward-backward parabolic De Giorgi class. Discrete and Continuous Dynamical Systems - S, 2017, 10 (4) : 853-866. doi: 10.3934/dcdss.2017043

[5]

Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations and Control Theory, 2022, 11 (2) : 347-371. doi: 10.3934/eect.2020110

[6]

Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control and Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195

[7]

Kaitong Hu, Zhenjie Ren, Nizar Touzi. On path-dependent multidimensional forward-backward SDEs. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022010

[8]

Md. Haider Ali Biswas, Maria do Rosário de Pinho. A nonsmooth maximum principle for optimal control problems with state and mixed constraints - convex case. Conference Publications, 2011, 2011 (Special) : 174-183. doi: 10.3934/proc.2011.2011.174

[9]

Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161

[10]

Zhen Wu, Feng Zhang. Maximum principle for discrete-time stochastic optimal control problem and stochastic game. Mathematical Control and Related Fields, 2022, 12 (2) : 475-493. doi: 10.3934/mcrf.2021031

[11]

Monia Karouf. Reflected solutions of backward doubly SDEs driven by Brownian motion and Poisson random measure. Discrete and Continuous Dynamical Systems, 2019, 39 (10) : 5571-5601. doi: 10.3934/dcds.2019245

[12]

Yu Fu, Weidong Zhao, Tao Zhou. Efficient spectral sparse grid approximations for solving multi-dimensional forward backward SDEs. Discrete and Continuous Dynamical Systems - B, 2017, 22 (9) : 3439-3458. doi: 10.3934/dcdsb.2017174

[13]

Jiongmin Yong. Forward-backward evolution equations and applications. Mathematical Control and Related Fields, 2016, 6 (4) : 653-704. doi: 10.3934/mcrf.2016019

[14]

Fabio Paronetto. Elliptic approximation of forward-backward parabolic equations. Communications on Pure and Applied Analysis, 2020, 19 (2) : 1017-1036. doi: 10.3934/cpaa.2020047

[15]

Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems and Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002

[16]

Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial and Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067

[17]

Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581

[18]

Qi Zhang, Huaizhong Zhao. Backward doubly stochastic differential equations with polynomial growth coefficients. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5285-5315. doi: 10.3934/dcds.2015.35.5285

[19]

Yufeng Shi, Qingfeng Zhu. A Kneser-type theorem for backward doubly stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1565-1579. doi: 10.3934/dcdsb.2010.14.1565

[20]

Yinggu Chen, Said HamadÈne, Tingshu Mu. Mean-field doubly reflected backward stochastic differential equations. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022012

2021 Impact Factor: 1.141

Metrics

  • PDF downloads (327)
  • HTML views (386)
  • Cited by (0)

Other articles
by authors

[Back to Top]