Optimality conditions in variational form for non-linear constrained stochastic control problems

Pfeiffer Laurent

doi:10.3934/mcrf.2020008

Article Contents

2020, Volume 10, Issue 3: 493-526. Doi: 10.3934/mcrf.2020008

This issue Previous Article Sparse optimal control for the heat equation with mixed control-state constraints Next Article State-constrained semilinear elliptic optimization problems with unrestricted sparse controls

Optimality conditions in variational form for non-linear constrained stochastic control problems

Pfeiffer Laurent

Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria

Dedicated to Prof. Dr. Fréderic Bonnans on the occasion of his 60th birthday

Received: January 31, 2018

Revised: November 30, 2018

Early access: December 2019

Published: July 2020

Abstract Full Text(HTML) Figure(3) Related Papers Cited by

Abstract

Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

Keywords:

Mathematics Subject Classification: Primary: 90C15, 93E20; Secondary: 49J53, 49K99.

Citation:

Full Text(HTML)

Figure 1. Convergence results for the Test Case 1

Download: Full-size image PowerPoint slide

Figure 2. Numerical results

Download: Full-size image PowerPoint slide

Figure 3. Convergence results for the Test Case 2

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	Y. Achdou and M. Laurière, On the system of partial differential equations arising in mean field type control, Discrete and Continuous Dynamical Systems, 35 (2015), 3879-3900. doi: 10.3934/dcds.2015.35.3879.
[2]	G. Albi, Y.-P. Choi, M. Fornasier and D. Kalise, Mean field control hierarchy, Appl. Math. Optim., 76 (2017), 93-135. doi: 10.1007/s00245-017-9429-x.
[3]	D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), 341-356. doi: 10.1007/s00245-010-9123-8.
[4]	A. Bensoussan, J. Frehse and P. Yam, Mean Field Games and Mean Field Type Control Theory, SpringerBriefs in Mathematics, Springer, New York, 2013. doi: 10.1007/978-1-4614-8508-7.
[5]	J. F. Bonnans, L. Pfeiffer and O. S. Serea, Sensitivity analysis for relaxed optimal control problems with final-state constraints, Nonlinear Anal., 89 (2013), 55-80. doi: 10.1016/j.na.2013.04.013.
[6]	J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Series in Operations Research, Springer-Verlag, New York, 2000. doi: 10.1007/978-1-4612-1394-9.
[7]	J. F. Bonnans and F. J. Silva., First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65 (2012), 403-439. doi: 10.1007/s00245-012-9162-4.
[8]	B. Bouchard, R. Elie and N. Touzi, Stochastic target problems with controlled loss, SIAM J. Control Optim., 48 (2009/10), 3123-3150. doi: 10.1137/08073593X.
[9]	R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Appl. Math. Optim., 64 (2011), 197-216. doi: 10.1007/s00245-011-9136-y.
[10]	P. H. Calamai and J. J. Moré, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), 93-116. doi: 10.1007/BF02592073.
[11]	F. Camilli and M. Falcone, An approximation scheme for the optimal control of diffusion processes, RAIRO Modél. Math. Anal. Numér., 29 (1995), 97-122. doi: 10.1051/m2an/1995290100971.
[12]	P. Cardaliaguet, Notes on Mean Field Games, 2012.
[13]	R. Carmona and F. Delarue, Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics, Ann. Probab., 43 (2015), 2647-2700. doi: 10.1214/14-AOP946.
[14]	A. R. Conn, N. I. M. Gould and P. L. Toint, LANCELOT. A Fortran Package for Large-Scale Nonlinear Optimization (Release A), Springer Series in Computational Mathematics, 17. Springer-Verlag, Berlin, 1992. doi: 10.1007/978-3-662-12211-2.
[15]	A. Fleig and R. Guglielmi, Optimal control of the Fokker-Planck equation with space-dependent controls, Journal of Optimization Theory and Applications, 174 (2017), 408-427. doi: 10.1007/s10957-017-1120-5.
[16]	W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second edition, Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006.
[17]	N. V. Krylov, Controlled Diffusion Processes, Applications of Mathematics, 14. Springer-Verlag, New York-Berlin, 1980.
[18]	M. Laurière and O. Pironneau, Dynamic programming for mean-field type control, Journal of Optimization Theory and Applications, 169 (2016), 902-924. doi: 10.1007/s10957-015-0785-x.
[19]	C. W. Miller and I. Yang, Optimal control of conditional value-at-risk in continuous time, SIAM J. Control Optim., 55 (2017), 856-884. doi: 10.1137/16M1058492.
[20]	J. Nocedal and S. J. Wright, Numerical Optimization, Second edition, Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.
[21]	B. Øksendal, Stochastic Differential Equations: An Introduction with Applications, Hochschultext / Universitext. Springer, 2003.
[22]	J. L. Pedersen and G. Peskir, Optimal mean-variance portfolio selection, Math. Financ. Econ., 11 (2017), 137-160. doi: 10.1007/s11579-016-0174-8.
[23]	S. G. Peng, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), 966-979. doi: 10.1137/0328054.
[24]	L. Pfeiffer, Optimality conditions for mean-field type optimal control problems, SFB-report 2015-015, (2015).
[25]	L. Pfeiffer, Numerical methods for mean-field type optimal control problems, Pure and Applied Functional Analysis, 1 (2016), 629-655.
[26]	L. Pfeiffer, Risk-averse Merton's portfolio problem, IFAC-PapersOnLine, 49 (2016), 266-271. doi: 10.1016/j.ifacol.2016.07.452.
[27]	L. Pfeiffer, Two approaches to stochastic optimal control problems with a final time expectation constraint, Appl. Math. Optim., 77 (2018), 377-404. doi: 10.1007/s00245-016-9378-9.
[28]	H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Modelling and Applied Probability, 61. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-89500-8.
[29]	H. Pham and X. L. Wei, Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics, SIAM Journal on Control and Optimization, 55 (2017), 1069-1101. doi: 10.1137/16M1071390.
[30]	H. Pham and X. L. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM Control Optim. Calc. Var., 24 (2018), 437-461. doi: 10.1051/cocv/2017019.
[31]	A. Shapiro, D. Dentcheva and A. Ruszczyński, Lectures on Stochastic Programming. Modeling and Theory, Second edition, MOS-SIAM Series on Optimization, 9. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, Mathematical Optimization Society, Philadelphia, PA, 2014.
[32]	X. L. Tan and N. Touzi, Optimal transportation under controlled stochastic dynamics, Ann. Probab., 41 (2013), 3201-3240. doi: 10.1214/12-AOP797.
[33]	N. Touzi, Direct characterization of the value of super-replication under stochastic volatility and portfolio constraints, Stochastic Process. Appl., 88 (2000), 305-328. doi: 10.1016/S0304-4149(00)00007-7.
[34]	C. Villani, Optimal Transport. Old and New, Grundlehren der Mathematischen Wissenschaften, 338. Springer-Verlag, Berlin, 2009. doi: 10.1007/978-3-540-71050-9.
[35]	J. M. Yong and X. Y. Zhou., Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics (New York), 43. Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.