doi: 10.3934/mcrf.2020008

Optimality conditions in variational form for non-linear constrained stochastic control problems

Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria

Dedicated to Prof. Dr. Fréderic Bonnans on the occasion of his 60th birthday

Received  February 2018 Revised  December 2018 Published  December 2019

Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

Citation: Laurent Pfeiffer. Optimality conditions in variational form for non-linear constrained stochastic control problems. Mathematical Control & Related Fields, doi: 10.3934/mcrf.2020008
References:
[1]

Y. Achdou and M. Laurière, On the system of partial differential equations arising in mean field type control, Discrete and Continuous Dynamical Systems, 35 (2015), 3879-3900.  doi: 10.3934/dcds.2015.35.3879.  Google Scholar

[2]

G. AlbiY.-P. ChoiM. Fornasier and D. Kalise, Mean field control hierarchy, Appl. Math. Optim., 76 (2017), 93-135.  doi: 10.1007/s00245-017-9429-x.  Google Scholar

[3]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[4]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics, Springer, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[5]

J. F. BonnansL. Pfeiffer and O. S. Serea, Sensitivity analysis for relaxed optimal control problems with final-state constraints, Nonlinear Anal., 89 (2013), 55-80.  doi: 10.1016/j.na.2013.04.013.  Google Scholar

[6]

J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Series in Operations Research, Springer-Verlag, New York, 2000. doi: 10.1007/978-1-4612-1394-9.  Google Scholar

[7]

J. F. Bonnans and F. J. Silva., First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65 (2012), 403-439.  doi: 10.1007/s00245-012-9162-4.  Google Scholar

[8]

B. BouchardR. Elie and N. Touzi, Stochastic target problems with controlled loss, SIAM J. Control Optim., 48 (2009/10), 3123-3150.  doi: 10.1137/08073593X.  Google Scholar

[9]

R. BuckdahnB. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Appl. Math. Optim., 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[10]

P. H. Calamai and J. J. Moré, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), 93-116.  doi: 10.1007/BF02592073.  Google Scholar

[11]

F. Camilli and M. Falcone, An approximation scheme for the optimal control of diffusion processes, RAIRO Modél. Math. Anal. Numér., 29 (1995), 97-122.  doi: 10.1051/m2an/1995290100971.  Google Scholar

[12]

P. Cardaliaguet, Notes on Mean Field Games, 2012. Google Scholar

[13]

R. Carmona and F. Delarue, Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics, Ann. Probab., 43 (2015), 2647-2700.  doi: 10.1214/14-AOP946.  Google Scholar

[14]

A. R. Conn, N. I. M. Gould and P. L. Toint, LANCELOT. A Fortran Package for Large-Scale Nonlinear Optimization (Release A), Springer Series in Computational Mathematics, 17. Springer-Verlag, Berlin, 1992. doi: 10.1007/978-3-662-12211-2.  Google Scholar

[15]

A. Fleig and R. Guglielmi, Optimal control of the Fokker-Planck equation with space-dependent controls, Journal of Optimization Theory and Applications, 174 (2017), 408-427.  doi: 10.1007/s10957-017-1120-5.  Google Scholar

[16]

W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006.  Google Scholar

[17]

N. V. Krylov, Controlled Diffusion Processes, Applications of Mathematics, 14. Springer-Verlag, New York-Berlin, 1980.  Google Scholar

[18]

M. Laurière and O. Pironneau, Dynamic programming for mean-field type control, Journal of Optimization Theory and Applications, 169 (2016), 902-924.  doi: 10.1007/s10957-015-0785-x.  Google Scholar

[19]

C. W. Miller and I. Yang, Optimal control of conditional value-at-risk in continuous time, SIAM J. Control Optim., 55 (2017), 856-884.  doi: 10.1137/16M1058492.  Google Scholar

[20]

J. Nocedal and S. J. Wright, Numerical Optimization, Second edition, Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.  Google Scholar

[21]

B. Øksendal, Stochastic Differential Equations: An Introduction with Applications, Hochschultext / Universitext. Springer, 2003. Google Scholar

[22]

J. L. Pedersen and G. Peskir, Optimal mean-variance portfolio selection, Math. Financ. Econ., 11 (2017), 137-160.  doi: 10.1007/s11579-016-0174-8.  Google Scholar

[23]

S. G. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.  Google Scholar

[24]

L. Pfeiffer, Optimality conditions for mean-field type optimal control problems, SFB-report 2015-015, (2015). Google Scholar

[25]

L. Pfeiffer, Numerical methods for mean-field type optimal control problems, Pure and Applied Functional Analysis, 1 (2016), 629-655.   Google Scholar

[26]

L. Pfeiffer, Risk-averse Merton's portfolio problem, IFAC-PapersOnLine, 49 (2016), 266-271.  doi: 10.1016/j.ifacol.2016.07.452.  Google Scholar

[27]

L. Pfeiffer, Two approaches to stochastic optimal control problems with a final time expectation constraint, Appl. Math. Optim., 77 (2018), 377-404.  doi: 10.1007/s00245-016-9378-9.  Google Scholar

[28]

H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Modelling and Applied Probability, 61. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-89500-8.  Google Scholar

[29]

H. Pham and X. L. Wei, Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics, SIAM Journal on Control and Optimization, 55 (2017), 1069-1101.  doi: 10.1137/16M1071390.  Google Scholar

[30]

H. Pham and X. L. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM Control Optim. Calc. Var., 24 (2018), 437-461.  doi: 10.1051/cocv/2017019.  Google Scholar

[31]

A. Shapiro, D. Dentcheva and A. Ruszczyński, Lectures on Stochastic Programming. Modeling and Theory, Second edition, MOS-SIAM Series on Optimization, 9. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, Mathematical Optimization Society, Philadelphia, PA, 2014.  Google Scholar

[32]

X. L. Tan and N. Touzi, Optimal transportation under controlled stochastic dynamics, Ann. Probab., 41 (2013), 3201-3240.  doi: 10.1214/12-AOP797.  Google Scholar

[33]

N. Touzi, Direct characterization of the value of super-replication under stochastic volatility and portfolio constraints, Stochastic Process. Appl., 88 (2000), 305-328.  doi: 10.1016/S0304-4149(00)00007-7.  Google Scholar

[34]

C. Villani, Optimal Transport. Old and New, Grundlehren der Mathematischen Wissenschaften, 338. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-71050-9.  Google Scholar

[35]

J. M. Yong and X. Y. Zhou., Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43. Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

Y. Achdou and M. Laurière, On the system of partial differential equations arising in mean field type control, Discrete and Continuous Dynamical Systems, 35 (2015), 3879-3900.  doi: 10.3934/dcds.2015.35.3879.  Google Scholar

[2]

G. AlbiY.-P. ChoiM. Fornasier and D. Kalise, Mean field control hierarchy, Appl. Math. Optim., 76 (2017), 93-135.  doi: 10.1007/s00245-017-9429-x.  Google Scholar

[3]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[4]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics, Springer, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[5]

J. F. BonnansL. Pfeiffer and O. S. Serea, Sensitivity analysis for relaxed optimal control problems with final-state constraints, Nonlinear Anal., 89 (2013), 55-80.  doi: 10.1016/j.na.2013.04.013.  Google Scholar

[6]

J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Series in Operations Research, Springer-Verlag, New York, 2000. doi: 10.1007/978-1-4612-1394-9.  Google Scholar

[7]

J. F. Bonnans and F. J. Silva., First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65 (2012), 403-439.  doi: 10.1007/s00245-012-9162-4.  Google Scholar

[8]

B. BouchardR. Elie and N. Touzi, Stochastic target problems with controlled loss, SIAM J. Control Optim., 48 (2009/10), 3123-3150.  doi: 10.1137/08073593X.  Google Scholar

[9]

R. BuckdahnB. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Appl. Math. Optim., 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[10]

P. H. Calamai and J. J. Moré, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), 93-116.  doi: 10.1007/BF02592073.  Google Scholar

[11]

F. Camilli and M. Falcone, An approximation scheme for the optimal control of diffusion processes, RAIRO Modél. Math. Anal. Numér., 29 (1995), 97-122.  doi: 10.1051/m2an/1995290100971.  Google Scholar

[12]

P. Cardaliaguet, Notes on Mean Field Games, 2012. Google Scholar

[13]

R. Carmona and F. Delarue, Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics, Ann. Probab., 43 (2015), 2647-2700.  doi: 10.1214/14-AOP946.  Google Scholar

[14]

A. R. Conn, N. I. M. Gould and P. L. Toint, LANCELOT. A Fortran Package for Large-Scale Nonlinear Optimization (Release A), Springer Series in Computational Mathematics, 17. Springer-Verlag, Berlin, 1992. doi: 10.1007/978-3-662-12211-2.  Google Scholar

[15]

A. Fleig and R. Guglielmi, Optimal control of the Fokker-Planck equation with space-dependent controls, Journal of Optimization Theory and Applications, 174 (2017), 408-427.  doi: 10.1007/s10957-017-1120-5.  Google Scholar

[16]

W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006.  Google Scholar

[17]

N. V. Krylov, Controlled Diffusion Processes, Applications of Mathematics, 14. Springer-Verlag, New York-Berlin, 1980.  Google Scholar

[18]

M. Laurière and O. Pironneau, Dynamic programming for mean-field type control, Journal of Optimization Theory and Applications, 169 (2016), 902-924.  doi: 10.1007/s10957-015-0785-x.  Google Scholar

[19]

C. W. Miller and I. Yang, Optimal control of conditional value-at-risk in continuous time, SIAM J. Control Optim., 55 (2017), 856-884.  doi: 10.1137/16M1058492.  Google Scholar

[20]

J. Nocedal and S. J. Wright, Numerical Optimization, Second edition, Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.  Google Scholar

[21]

B. Øksendal, Stochastic Differential Equations: An Introduction with Applications, Hochschultext / Universitext. Springer, 2003. Google Scholar

[22]

J. L. Pedersen and G. Peskir, Optimal mean-variance portfolio selection, Math. Financ. Econ., 11 (2017), 137-160.  doi: 10.1007/s11579-016-0174-8.  Google Scholar

[23]

S. G. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.  Google Scholar

[24]

L. Pfeiffer, Optimality conditions for mean-field type optimal control problems, SFB-report 2015-015, (2015). Google Scholar

[25]

L. Pfeiffer, Numerical methods for mean-field type optimal control problems, Pure and Applied Functional Analysis, 1 (2016), 629-655.   Google Scholar

[26]

L. Pfeiffer, Risk-averse Merton's portfolio problem, IFAC-PapersOnLine, 49 (2016), 266-271.  doi: 10.1016/j.ifacol.2016.07.452.  Google Scholar

[27]

L. Pfeiffer, Two approaches to stochastic optimal control problems with a final time expectation constraint, Appl. Math. Optim., 77 (2018), 377-404.  doi: 10.1007/s00245-016-9378-9.  Google Scholar

[28]

H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Modelling and Applied Probability, 61. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-89500-8.  Google Scholar

[29]

H. Pham and X. L. Wei, Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics, SIAM Journal on Control and Optimization, 55 (2017), 1069-1101.  doi: 10.1137/16M1071390.  Google Scholar

[30]

H. Pham and X. L. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM Control Optim. Calc. Var., 24 (2018), 437-461.  doi: 10.1051/cocv/2017019.  Google Scholar

[31]

A. Shapiro, D. Dentcheva and A. Ruszczyński, Lectures on Stochastic Programming. Modeling and Theory, Second edition, MOS-SIAM Series on Optimization, 9. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, Mathematical Optimization Society, Philadelphia, PA, 2014.  Google Scholar

[32]

X. L. Tan and N. Touzi, Optimal transportation under controlled stochastic dynamics, Ann. Probab., 41 (2013), 3201-3240.  doi: 10.1214/12-AOP797.  Google Scholar

[33]

N. Touzi, Direct characterization of the value of super-replication under stochastic volatility and portfolio constraints, Stochastic Process. Appl., 88 (2000), 305-328.  doi: 10.1016/S0304-4149(00)00007-7.  Google Scholar

[34]

C. Villani, Optimal Transport. Old and New, Grundlehren der Mathematischen Wissenschaften, 338. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-71050-9.  Google Scholar

[35]

J. M. Yong and X. Y. Zhou., Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43. Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

Figure 1.  Convergence results for the Test Case 1
Figure 2.  Numerical results
Figure 3.  Convergence results for the Test Case 2
[1]

Heinz Schättler, Urszula Ledzewicz, Helmut Maurer. Sufficient conditions for strong local optimality in optimal control problems with $L_{2}$-type objectives and control constraints. Discrete & Continuous Dynamical Systems - B, 2014, 19 (8) : 2657-2679. doi: 10.3934/dcdsb.2014.19.2657

[2]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97

[3]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control & Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026

[4]

Lucas Bonifacius, Ira Neitzel. Second order optimality conditions for optimal control of quasilinear parabolic equations. Mathematical Control & Related Fields, 2018, 8 (1) : 1-34. doi: 10.3934/mcrf.2018001

[5]

Sofia O. Lopes, Fernando A. C. C. Fontes, Maria do Rosário de Pinho. On constraint qualifications for nondegenerate necessary conditions of optimality applied to optimal control problems. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 559-575. doi: 10.3934/dcds.2011.29.559

[6]

Wei Zhu, Xue-Cheng Tai, Tony Chan. Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Problems & Imaging, 2013, 7 (4) : 1409-1432. doi: 10.3934/ipi.2013.7.1409

[7]

Yves Achdou, Mathieu Laurière. On the system of partial differential equations arising in mean field type control. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3879-3900. doi: 10.3934/dcds.2015.35.3879

[8]

Jianxiong Ye, An Li. Necessary optimality conditions for nonautonomous optimal control problems and its applications to bilevel optimal control. Journal of Industrial & Management Optimization, 2019, 15 (3) : 1399-1419. doi: 10.3934/jimo.2018101

[9]

Shahlar F. Maharramov. Necessary optimality conditions for switching control problems. Journal of Industrial & Management Optimization, 2010, 6 (1) : 47-55. doi: 10.3934/jimo.2010.6.47

[10]

Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017

[11]

Ciro D'Apice, Olha P. Kupenko, Rosanna Manzo. On boundary optimal control problem for an arterial system: First-order optimality conditions. Networks & Heterogeneous Media, 2018, 13 (4) : 585-607. doi: 10.3934/nhm.2018027

[12]

Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time. Discrete & Continuous Dynamical Systems - B, 2019, 24 (4) : 1743-1767. doi: 10.3934/dcdsb.2018235

[13]

Wei Zhu. A numerical study of a mean curvature denoising model using a novel augmented Lagrangian method. Inverse Problems & Imaging, 2017, 11 (6) : 975-996. doi: 10.3934/ipi.2017045

[14]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018

[15]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks & Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699

[16]

Qun Lin, Ryan Loxton, Kok Lay Teo. The control parameterization method for nonlinear optimal control: A survey. Journal of Industrial & Management Optimization, 2014, 10 (1) : 275-309. doi: 10.3934/jimo.2014.10.275

[17]

Miniak-Górecka Alicja, Nowakowski Andrzej. Sufficient optimality conditions for a class of epidemic problems with control on the boundary. Mathematical Biosciences & Engineering, 2017, 14 (1) : 263-275. doi: 10.3934/mbe.2017017

[18]

Lihua Li, Yan Gao, Hongjie Wang. Second order sufficient optimality conditions for hybrid control problems with state jump. Journal of Industrial & Management Optimization, 2015, 11 (1) : 329-343. doi: 10.3934/jimo.2015.11.329

[19]

María Teresa V. Martínez-Palacios, Adrián Hernández-Del-Valle, Ambrosio Ortiz-Ramírez. On the pricing of Asian options with geometric average of American type with stochastic interest rate: A stochastic optimal control approach. Journal of Dynamics & Games, 2019, 6 (1) : 53-64. doi: 10.3934/jdg.2019004

[20]

Ana P. Lemos-Paião, Cristiana J. Silva, Delfim F. M. Torres. A sufficient optimality condition for delayed state-linear optimal control problems. Discrete & Continuous Dynamical Systems - B, 2019, 24 (5) : 2293-2313. doi: 10.3934/dcdsb.2019096

2018 Impact Factor: 1.292

Metrics

  • PDF downloads (9)
  • HTML views (24)
  • Cited by (0)

Other articles
by authors

[Back to Top]