September  2020, 10(3): 493-526. doi: 10.3934/mcrf.2020008

Optimality conditions in variational form for non-linear constrained stochastic control problems

Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria

Dedicated to Prof. Dr. Fréderic Bonnans on the occasion of his 60th birthday

Received  February 2018 Revised  December 2018 Published  December 2019

Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

Citation: Laurent Pfeiffer. Optimality conditions in variational form for non-linear constrained stochastic control problems. Mathematical Control & Related Fields, 2020, 10 (3) : 493-526. doi: 10.3934/mcrf.2020008
References:
[1]

Y. Achdou and M. Laurière, On the system of partial differential equations arising in mean field type control, Discrete and Continuous Dynamical Systems, 35 (2015), 3879-3900.  doi: 10.3934/dcds.2015.35.3879.  Google Scholar

[2]

G. AlbiY.-P. ChoiM. Fornasier and D. Kalise, Mean field control hierarchy, Appl. Math. Optim., 76 (2017), 93-135.  doi: 10.1007/s00245-017-9429-x.  Google Scholar

[3]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[4]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics, Springer, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[5]

J. F. BonnansL. Pfeiffer and O. S. Serea, Sensitivity analysis for relaxed optimal control problems with final-state constraints, Nonlinear Anal., 89 (2013), 55-80.  doi: 10.1016/j.na.2013.04.013.  Google Scholar

[6]

J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Series in Operations Research, Springer-Verlag, New York, 2000. doi: 10.1007/978-1-4612-1394-9.  Google Scholar

[7]

J. F. Bonnans and F. J. Silva., First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65 (2012), 403-439.  doi: 10.1007/s00245-012-9162-4.  Google Scholar

[8]

B. BouchardR. Elie and N. Touzi, Stochastic target problems with controlled loss, SIAM J. Control Optim., 48 (2009/10), 3123-3150.  doi: 10.1137/08073593X.  Google Scholar

[9]

R. BuckdahnB. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Appl. Math. Optim., 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[10]

P. H. Calamai and J. J. Moré, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), 93-116.  doi: 10.1007/BF02592073.  Google Scholar

[11]

F. Camilli and M. Falcone, An approximation scheme for the optimal control of diffusion processes, RAIRO Modél. Math. Anal. Numér., 29 (1995), 97-122.  doi: 10.1051/m2an/1995290100971.  Google Scholar

[12]

P. Cardaliaguet, Notes on Mean Field Games, 2012. Google Scholar

[13]

R. Carmona and F. Delarue, Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics, Ann. Probab., 43 (2015), 2647-2700.  doi: 10.1214/14-AOP946.  Google Scholar

[14]

A. R. Conn, N. I. M. Gould and P. L. Toint, LANCELOT. A Fortran Package for Large-Scale Nonlinear Optimization (Release A), Springer Series in Computational Mathematics, 17. Springer-Verlag, Berlin, 1992. doi: 10.1007/978-3-662-12211-2.  Google Scholar

[15]

A. Fleig and R. Guglielmi, Optimal control of the Fokker-Planck equation with space-dependent controls, Journal of Optimization Theory and Applications, 174 (2017), 408-427.  doi: 10.1007/s10957-017-1120-5.  Google Scholar

[16]

W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006.  Google Scholar

[17]

N. V. Krylov, Controlled Diffusion Processes, Applications of Mathematics, 14. Springer-Verlag, New York-Berlin, 1980.  Google Scholar

[18]

M. Laurière and O. Pironneau, Dynamic programming for mean-field type control, Journal of Optimization Theory and Applications, 169 (2016), 902-924.  doi: 10.1007/s10957-015-0785-x.  Google Scholar

[19]

C. W. Miller and I. Yang, Optimal control of conditional value-at-risk in continuous time, SIAM J. Control Optim., 55 (2017), 856-884.  doi: 10.1137/16M1058492.  Google Scholar

[20]

J. Nocedal and S. J. Wright, Numerical Optimization, Second edition, Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.  Google Scholar

[21]

B. Øksendal, Stochastic Differential Equations: An Introduction with Applications, Hochschultext / Universitext. Springer, 2003. Google Scholar

[22]

J. L. Pedersen and G. Peskir, Optimal mean-variance portfolio selection, Math. Financ. Econ., 11 (2017), 137-160.  doi: 10.1007/s11579-016-0174-8.  Google Scholar

[23]

S. G. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.  Google Scholar

[24]

L. Pfeiffer, Optimality conditions for mean-field type optimal control problems, SFB-report 2015-015, (2015). Google Scholar

[25]

L. Pfeiffer, Numerical methods for mean-field type optimal control problems, Pure and Applied Functional Analysis, 1 (2016), 629-655.   Google Scholar

[26]

L. Pfeiffer, Risk-averse Merton's portfolio problem, IFAC-PapersOnLine, 49 (2016), 266-271.  doi: 10.1016/j.ifacol.2016.07.452.  Google Scholar

[27]

L. Pfeiffer, Two approaches to stochastic optimal control problems with a final time expectation constraint, Appl. Math. Optim., 77 (2018), 377-404.  doi: 10.1007/s00245-016-9378-9.  Google Scholar

[28]

H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Modelling and Applied Probability, 61. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-89500-8.  Google Scholar

[29]

H. Pham and X. L. Wei, Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics, SIAM Journal on Control and Optimization, 55 (2017), 1069-1101.  doi: 10.1137/16M1071390.  Google Scholar

[30]

H. Pham and X. L. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM Control Optim. Calc. Var., 24 (2018), 437-461.  doi: 10.1051/cocv/2017019.  Google Scholar

[31]

A. Shapiro, D. Dentcheva and A. Ruszczyński, Lectures on Stochastic Programming. Modeling and Theory, Second edition, MOS-SIAM Series on Optimization, 9. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, Mathematical Optimization Society, Philadelphia, PA, 2014.  Google Scholar

[32]

X. L. Tan and N. Touzi, Optimal transportation under controlled stochastic dynamics, Ann. Probab., 41 (2013), 3201-3240.  doi: 10.1214/12-AOP797.  Google Scholar

[33]

N. Touzi, Direct characterization of the value of super-replication under stochastic volatility and portfolio constraints, Stochastic Process. Appl., 88 (2000), 305-328.  doi: 10.1016/S0304-4149(00)00007-7.  Google Scholar

[34]

C. Villani, Optimal Transport. Old and New, Grundlehren der Mathematischen Wissenschaften, 338. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-71050-9.  Google Scholar

[35]

J. M. Yong and X. Y. Zhou., Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43. Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

Y. Achdou and M. Laurière, On the system of partial differential equations arising in mean field type control, Discrete and Continuous Dynamical Systems, 35 (2015), 3879-3900.  doi: 10.3934/dcds.2015.35.3879.  Google Scholar

[2]

G. AlbiY.-P. ChoiM. Fornasier and D. Kalise, Mean field control hierarchy, Appl. Math. Optim., 76 (2017), 93-135.  doi: 10.1007/s00245-017-9429-x.  Google Scholar

[3]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[4]

A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics, Springer, New York, 2013. doi: 10.1007/978-1-4614-8508-7.  Google Scholar

[5]

J. F. BonnansL. Pfeiffer and O. S. Serea, Sensitivity analysis for relaxed optimal control problems with final-state constraints, Nonlinear Anal., 89 (2013), 55-80.  doi: 10.1016/j.na.2013.04.013.  Google Scholar

[6]

J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Series in Operations Research, Springer-Verlag, New York, 2000. doi: 10.1007/978-1-4612-1394-9.  Google Scholar

[7]

J. F. Bonnans and F. J. Silva., First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65 (2012), 403-439.  doi: 10.1007/s00245-012-9162-4.  Google Scholar

[8]

B. BouchardR. Elie and N. Touzi, Stochastic target problems with controlled loss, SIAM J. Control Optim., 48 (2009/10), 3123-3150.  doi: 10.1137/08073593X.  Google Scholar

[9]

R. BuckdahnB. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Appl. Math. Optim., 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[10]

P. H. Calamai and J. J. Moré, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), 93-116.  doi: 10.1007/BF02592073.  Google Scholar

[11]

F. Camilli and M. Falcone, An approximation scheme for the optimal control of diffusion processes, RAIRO Modél. Math. Anal. Numér., 29 (1995), 97-122.  doi: 10.1051/m2an/1995290100971.  Google Scholar

[12]

P. Cardaliaguet, Notes on Mean Field Games, 2012. Google Scholar

[13]

R. Carmona and F. Delarue, Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics, Ann. Probab., 43 (2015), 2647-2700.  doi: 10.1214/14-AOP946.  Google Scholar

[14]

A. R. Conn, N. I. M. Gould and P. L. Toint, LANCELOT. A Fortran Package for Large-Scale Nonlinear Optimization (Release A), Springer Series in Computational Mathematics, 17. Springer-Verlag, Berlin, 1992. doi: 10.1007/978-3-662-12211-2.  Google Scholar

[15]

A. Fleig and R. Guglielmi, Optimal control of the Fokker-Planck equation with space-dependent controls, Journal of Optimization Theory and Applications, 174 (2017), 408-427.  doi: 10.1007/s10957-017-1120-5.  Google Scholar

[16]

W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006.  Google Scholar

[17]

N. V. Krylov, Controlled Diffusion Processes, Applications of Mathematics, 14. Springer-Verlag, New York-Berlin, 1980.  Google Scholar

[18]

M. Laurière and O. Pironneau, Dynamic programming for mean-field type control, Journal of Optimization Theory and Applications, 169 (2016), 902-924.  doi: 10.1007/s10957-015-0785-x.  Google Scholar

[19]

C. W. Miller and I. Yang, Optimal control of conditional value-at-risk in continuous time, SIAM J. Control Optim., 55 (2017), 856-884.  doi: 10.1137/16M1058492.  Google Scholar

[20]

J. Nocedal and S. J. Wright, Numerical Optimization, Second edition, Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.  Google Scholar

[21]

B. Øksendal, Stochastic Differential Equations: An Introduction with Applications, Hochschultext / Universitext. Springer, 2003. Google Scholar

[22]

J. L. Pedersen and G. Peskir, Optimal mean-variance portfolio selection, Math. Financ. Econ., 11 (2017), 137-160.  doi: 10.1007/s11579-016-0174-8.  Google Scholar

[23]

S. G. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979.  doi: 10.1137/0328054.  Google Scholar

[24]

L. Pfeiffer, Optimality conditions for mean-field type optimal control problems, SFB-report 2015-015, (2015). Google Scholar

[25]

L. Pfeiffer, Numerical methods for mean-field type optimal control problems, Pure and Applied Functional Analysis, 1 (2016), 629-655.   Google Scholar

[26]

L. Pfeiffer, Risk-averse Merton's portfolio problem, IFAC-PapersOnLine, 49 (2016), 266-271.  doi: 10.1016/j.ifacol.2016.07.452.  Google Scholar

[27]

L. Pfeiffer, Two approaches to stochastic optimal control problems with a final time expectation constraint, Appl. Math. Optim., 77 (2018), 377-404.  doi: 10.1007/s00245-016-9378-9.  Google Scholar

[28]

H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Modelling and Applied Probability, 61. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-89500-8.  Google Scholar

[29]

H. Pham and X. L. Wei, Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics, SIAM Journal on Control and Optimization, 55 (2017), 1069-1101.  doi: 10.1137/16M1071390.  Google Scholar

[30]

H. Pham and X. L. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM Control Optim. Calc. Var., 24 (2018), 437-461.  doi: 10.1051/cocv/2017019.  Google Scholar

[31]

A. Shapiro, D. Dentcheva and A. Ruszczyński, Lectures on Stochastic Programming. Modeling and Theory, Second edition, MOS-SIAM Series on Optimization, 9. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, Mathematical Optimization Society, Philadelphia, PA, 2014.  Google Scholar

[32]

X. L. Tan and N. Touzi, Optimal transportation under controlled stochastic dynamics, Ann. Probab., 41 (2013), 3201-3240.  doi: 10.1214/12-AOP797.  Google Scholar

[33]

N. Touzi, Direct characterization of the value of super-replication under stochastic volatility and portfolio constraints, Stochastic Process. Appl., 88 (2000), 305-328.  doi: 10.1016/S0304-4149(00)00007-7.  Google Scholar

[34]

C. Villani, Optimal Transport. Old and New, Grundlehren der Mathematischen Wissenschaften, 338. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-71050-9.  Google Scholar

[35]

J. M. Yong and X. Y. Zhou., Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43. Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

Figure 1.  Convergence results for the Test Case 1
Figure 2.  Numerical results
Figure 3.  Convergence results for the Test Case 2
[1]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077

[2]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046

[3]

Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347

[4]

Bernard Bonnard, Jérémy Rouot. Geometric optimal techniques to control the muscular force response to functional electrical stimulation using a non-isometric force-fatigue model. Journal of Geometric Mechanics, 2020  doi: 10.3934/jgm.2020032

[5]

Manil T. Mohan. First order necessary conditions of optimality for the two dimensional tidal dynamics system. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020045

[6]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

[7]

Mengni Li. Global regularity for a class of Monge-Ampère type equations with nonzero boundary conditions. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020267

[8]

Pengyu Chen. Non-autonomous stochastic evolution equations with nonlinear noise and nonlocal conditions governed by noncompact evolution families. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020383

[9]

Jie Li, Xiangdong Ye, Tao Yu. Mean equicontinuity, complexity and applications. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 359-393. doi: 10.3934/dcds.2020167

[10]

Knut Hüper, Irina Markina, Fátima Silva Leite. A Lagrangian approach to extremal curves on Stiefel manifolds. Journal of Geometric Mechanics, 2020  doi: 10.3934/jgm.2020031

[11]

José Madrid, João P. G. Ramos. On optimal autocorrelation inequalities on the real line. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020271

[12]

Adrian Constantin, Darren G. Crowdy, Vikas S. Krishnamurthy, Miles H. Wheeler. Stuart-type polar vortices on a rotating sphere. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 201-215. doi: 10.3934/dcds.2020263

[13]

Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020276

[14]

Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020076

[15]

João Marcos do Ó, Bruno Ribeiro, Bernhard Ruf. Hamiltonian elliptic systems in dimension two with arbitrary and double exponential growth conditions. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 277-296. doi: 10.3934/dcds.2020138

[16]

Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of a Sobolev type impulsive functional evolution system in Banach spaces. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020049

[17]

Meng Chen, Yong Hu, Matteo Penegini. On projective threefolds of general type with small positive geometric genus. Electronic Research Archive, , () : -. doi: 10.3934/era.2020117

[18]

Zhenzhen Wang, Tianshou Zhou. Asymptotic behaviors and stochastic traveling waves in stochastic Fisher-KPP equations. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020323

[19]

Tommi Brander, Joonas Ilmavirta, Petteri Piiroinen, Teemu Tyni. Optimal recovery of a radiating source with multiple frequencies along one line. Inverse Problems & Imaging, 2020, 14 (6) : 967-983. doi: 10.3934/ipi.2020044

[20]

Li-Bin Liu, Ying Liang, Jian Zhang, Xiaobing Bao. A robust adaptive grid method for singularly perturbed Burger-Huxley equations. Electronic Research Archive, 2020, 28 (4) : 1439-1457. doi: 10.3934/era.2020076

2019 Impact Factor: 0.857

Metrics

  • PDF downloads (97)
  • HTML views (367)
  • Cited by (0)

Other articles
by authors

[Back to Top]