American Institute of Mathematical Sciences

September  2020, 10(3): 493-526. doi: 10.3934/mcrf.2020008

Optimality conditions in variational form for non-linear constrained stochastic control problems

 Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria

Dedicated to Prof. Dr. Fréderic Bonnans on the occasion of his 60th birthday

Received  February 2018 Revised  December 2018 Published  December 2019

Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

Citation: Laurent Pfeiffer. Optimality conditions in variational form for non-linear constrained stochastic control problems. Mathematical Control & Related Fields, 2020, 10 (3) : 493-526. doi: 10.3934/mcrf.2020008
References:

show all references

References:
Convergence results for the Test Case 1
Numerical results
Convergence results for the Test Case 2
 [1] Heinz Schättler, Urszula Ledzewicz, Helmut Maurer. Sufficient conditions for strong local optimality in optimal control problems with $L_{2}$-type objectives and control constraints. Discrete & Continuous Dynamical Systems - B, 2014, 19 (8) : 2657-2679. doi: 10.3934/dcdsb.2014.19.2657 [2] Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97 [3] Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020026 [4] Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3 [5] Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control & Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026 [6] Lucas Bonifacius, Ira Neitzel. Second order optimality conditions for optimal control of quasilinear parabolic equations. Mathematical Control & Related Fields, 2018, 8 (1) : 1-34. doi: 10.3934/mcrf.2018001 [7] Sofia O. Lopes, Fernando A. C. C. Fontes, Maria do Rosário de Pinho. On constraint qualifications for nondegenerate necessary conditions of optimality applied to optimal control problems. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 559-575. doi: 10.3934/dcds.2011.29.559 [8] Adel Chala, Dahbia Hafayed. On stochastic maximum principle for risk-sensitive of fully coupled forward-backward stochastic control of mean-field type with application. Evolution Equations & Control Theory, 2020, 9 (3) : 817-843. doi: 10.3934/eect.2020035 [9] Wei Zhu, Xue-Cheng Tai, Tony Chan. Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Problems & Imaging, 2013, 7 (4) : 1409-1432. doi: 10.3934/ipi.2013.7.1409 [10] Jianxiong Ye, An Li. Necessary optimality conditions for nonautonomous optimal control problems and its applications to bilevel optimal control. Journal of Industrial & Management Optimization, 2019, 15 (3) : 1399-1419. doi: 10.3934/jimo.2018101 [11] Yves Achdou, Mathieu Laurière. On the system of partial differential equations arising in mean field type control. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3879-3900. doi: 10.3934/dcds.2015.35.3879 [12] Shahlar F. Maharramov. Necessary optimality conditions for switching control problems. Journal of Industrial & Management Optimization, 2010, 6 (1) : 47-55. doi: 10.3934/jimo.2010.6.47 [13] Ciro D'Apice, Olha P. Kupenko, Rosanna Manzo. On boundary optimal control problem for an arterial system: First-order optimality conditions. Networks & Heterogeneous Media, 2018, 13 (4) : 585-607. doi: 10.3934/nhm.2018027 [14] Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017 [15] Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time. Discrete & Continuous Dynamical Systems - B, 2019, 24 (4) : 1743-1767. doi: 10.3934/dcdsb.2018235 [16] Wei Zhu. A numerical study of a mean curvature denoising model using a novel augmented Lagrangian method. Inverse Problems & Imaging, 2017, 11 (6) : 975-996. doi: 10.3934/ipi.2017045 [17] Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018 [18] Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks & Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699 [19] Qun Lin, Ryan Loxton, Kok Lay Teo. The control parameterization method for nonlinear optimal control: A survey. Journal of Industrial & Management Optimization, 2014, 10 (1) : 275-309. doi: 10.3934/jimo.2014.10.275 [20] Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020019

2019 Impact Factor: 0.857