A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection

Maximilien Germain; Huyên Pham; Xavier Warin

doi:10.3934/naco.2022033

Article Contents

2023, Volume 13, Issue 3&4: 555-582. Doi: 10.3934/naco.2022033

This issue Previous Article Static Markowitz mean-variance portfolio selection model with long-term bonds Next Article G-Gaussian processes under sublinear expectations and

$ q $

-Brownian motion in quantum mechanics

A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection

1.
EDF R & D, Université Paris Cité and Sorbonne Université, CNRS, Laboratoire de Probabilités, Statistique et Modélisation, F-75013 Paris, France
2.
Université Paris Cité and Sorbonne Université, CNRS, Laboratoire de Probabilités, Statistique et Modélisation, CREST-ENSAE & FiME, F-75013 Paris, France
3.
EDF R & D & FiME, 91120 Palaiseau, France

^*Corresponding author: Huyên Pham
^*Corresponding author: Huyên Pham

Received: November 2021

Revised: October 2022

Early access: November 2022

Published: September 2023

This work was supported by FiME (Finance for Energy Market Research Centre) and the "Finance et Développement Durable - Approches Quantitatives" EDF - CACIB Chair

Abstract / Introduction Full Text(HTML) Figure(8) / Table(1) Related Papers Cited by

Abstract

We consider the control of McKean-Vlasov dynamics (or mean-field control) with probabilistic state constraints. We rely on a level-set approach which provides a representation of the constrained problem in terms of an unconstrained one with exact penalization and running maximum or integral cost. The method is then extended to the common noise setting. Our work extends (Bokanowski, Picarelli, and Zidani, SIAM J. Control Optim. 54.5 (2016), pp. 2568–2593) and (Bokanowski, Picarelli, and Zidani, Appl. Math. Optim. 71 (2015), pp. 125–163) to a mean-field setting.

The reformulation as an unconstrained problem is particularly suitable for the numerical resolution of the problem, that is achieved from an extension of a machine learning algorithm from (Carmona, Laurière, arXiv:1908.01613 to appear in Ann. Appl. Prob., 2019). A first application concerns the storage of renewable electricity in the presence of mean-field price impact and another one focuses on a mean-variance portfolio selection problem with probabilistic constraints on the wealth. We also illustrate our approach for a direct numerical resolution of the primal Markowitz continuous-time problem without relying on duality.

Keywords:

Mathematics Subject Classification: Primary: 49N80, 49M99; Secondary: 68T07, 93E20.

Citation:

FullText(HTML)

Figure 1. Sample path of the controlled process $ X_t^\alpha $, with the analytical optimal control (for the unconstrained case) and the computed control. On the left figure we don't have the true control but plot the unconstrained one for comparison. Here $ \Lambda = 100 $

Download: Full-size image PowerPoint slide

Figure 2. Histogram of $ X_T^\alpha $ for 50000 samples. Here $ \Lambda = 100 $

Download: Full-size image PowerPoint slide

Figure 3. Auxiliary value function $ {\mathcal{Y}}_{\Lambda}(z) $ for several values of $ \Lambda $ in the constrained case, auxiliary value function $ {\mathcal{Y}}(z) $ in the unconstrained case

Download: Full-size image PowerPoint slide

Figure 4. Conditional expectation $ E[X_t^\alpha\ |\ X_t^\alpha\leq 0.9] $ estimated with 50000 samples. The black line corresponds to $ \delta = 0.8 $. Here $ \Lambda = 100 $

Download: Full-size image PowerPoint slide

Figure 5. Sample trajectory of the controlled process $ X^\alpha_t $ and the control for problem (28) (left). Variance $ \mathrm{Var}(X_t) $ estimated with 50000 samples for problem (28) (right) with $ \Lambda = 10 $

Download: Full-size image PowerPoint slide

Figure 6. Auxiliary value function $ {\mathcal{U}}_{\Lambda}(z) $ for several values of $ \Lambda $

Download: Full-size image PowerPoint slide

Figure 7. $ \hat w $ function value for the storage problem

Download: Full-size image PowerPoint slide

Figure 8. Storage trajectories with the Level Set and Dynamic Programming method

Download: Full-size image PowerPoint slide

Table 1. Estimate of the solution with maturity $ T = 1. $ Average and standard deviation observed over 10 independent runs are reported, with the relative error (in $ \% $). We also report the terminal expectation and variance of the approximated optimally controlled process for a single run. '$ \rm Not avail. $ ' means that we don't have a reference value and 'Tr.' means true. For problem (28), we take $ \Lambda = 10 $ and for problem (27) we illustrate the values obtained for $ \Lambda \in \{1., 10., 100\} $

Problem	Average	Std	Tr. val.	Error
(27) $ \Lambda = 1. $	-1.044	0.0010	$ \rm Not avail. $	$ \rm Not avail. $
(27) $ \Lambda = 10. $	-1.044	0.0005	$ \rm Not avail. $	$ \rm Not avail. $
27 $ \Lambda = 100. $	-1.045	0.0005	$ \rm Not avail. $	$ \rm Not avail. $
(28) $ \Lambda = 10. $	-1.048	0.0017	-1.050	0.22
(25)	-1.050	0.0009	-1.050	0.07
(25) [17]	-1.052	0.0022	-1.050	0.13
Problem	$\mathbb{E}[X_T^{\alpha^*}]$	Tr. $\mathbb{E}[X_T^{\alpha^*}]$	$\mathrm{Var}(X_T^{\alpha^*})$	Tr. $\mathrm{Var}(X_T^{\alpha^*})$
(27) $\Lambda = 1.$	1.07	Not avail.	0.026	Not avail.
(27) $\Lambda = 10.$	1.07	Not avail.	0.026	Not avail.
27 $\Lambda = 100.$	1.07	Not avail.	0.027	Not avail.
(28) $\Lambda = 10.$	1.10	1.10	0.049	0.050
(25)	1.10	1.10	0.050	0.050
(25)[17]	1.10	1.10	0.053	0.050

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	C. Alasseur, I. Ben Taher and A. Matoussi, An extended mean field game for storage in smart grids, Journal of Optimization Theory and Applications, 184 (2020), 644-670. doi: 10.1007/s10957-019-01619-3.
[2]	A. Altarovici, O. Bokanowski and H. Zidani, A general hamilton-jacobi framework for non-linear state-constrained control problems, ESAIM: COCV, 19 (2013), 337-357. doi: 10.1051/cocv/2012011.
[3]	S. S. Arjmand and G. Mazanti, Nonsmooth mean field games with state constraints, arXiv: 2110.15713
[4]	A. Balata, M. Ludkovski, A. Maheshwari and J. Palczewski, Statistical learning for probability-constrained stochastic optimal control, European Journal of Operational Research, 290 (2021), 640-656. doi: 10.1016/j.ejor.2020.08.041.
[5]	O. Bokanowski, N. Gammoudi and H. Zidani, Optimistic planning algorithms for state-constrained optimal control problems, Computers and Mathematics with applications, 109 (2022), 158-179. doi: 10.1016/j.camwa.2022.01.016.
[6]	O. Bokanowski, A. Picarelli and H. Zidani, Dynamic programming and error estimates for stochastic control problems with maximum cost, Appl. Math. Optim., 74 (2014), 125-163. doi: 10.1007/s00245-014-9255-3.
[7]	O. Bokanowski, A. Picarelli and H. Zidani, State-constrained stochastic optimal control problems via reachability approach, SIAM Journal on Control and Optimization, 54 (2016), 2568-2593. doi: 10.1137/15M1023737.
[8]	B. Bonnet, A pontryagin maximum principle in wasserstein spaces for constrained optimal control problems, ESAIM: COCV, 25 (2019), 52. doi: 10.1051/cocv/2019044.
[9]	B. Bonnet and H. Frankowska, Necessary optimality conditions for optimal control problems in wasserstein spaces, Appl. Math. Optim., 84 (2021), 1281-1330. doi: 10.1007/s00245-021-09772-w.
[10]	B. Bouchard, B. Djehiche and I. Kharroubi, Quenched mass transport of particles toward a target, Journal of Optimization Theory and Applications, 186 (2020), 345-374. doi: 10.1007/s10957-020-01704-y.
[11]	B. Bouchard, R. Elie and C. Imbert, Optimal control under stochastic target constraints, SIAM Journal on Control and Optimization, 48 (2010), 3501-3531. doi: 10.1137/090757629.
[12]	P. Cannarsa and R. Capuani, Existence and uniqueness for mean field games with state constraints, in PDE Models for Multi-Agent Phenomena (eds. C. Pierre, P. A. and S. F.), vol. 28 of Springer INdAM Series
[13]	P. Cannarsa, R. Capuani and P. Cardaliaguet, Mean field games with state constraints: from mild to pointwise solutions of the pde system, Calculus of Variations and Partial Differential Equations, 60 (2021), Article Number 108. doi: 10.1007/s00526-021-01936-4.
[14]	R. Carmona and F. Delarue, Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics, The Annals of Probability, 43 (2015), 2647-2700. doi: 10.1214/14-AOP946.
[15]	R. Carmona and F. Delarue, Probabilistic Theory of Mean Field Games: vol. I, Mean Field FBSDEs, Control, and Games, Springer, 2018. doi: 10.1007/s00245-016-9396-7.
[16]	R. Carmona and F. Delarue, Probabilistic Theory of Mean Field Games: vol. II, Mean Field Game with Common Noise and Master Equations, Springer, 2018.
[17]	R. Carmona and M. Laurière, Convergence analysis of machine learning algorithms for the numerical solution of mean-field control and games: II the finite horizon case, arXiv: 1908.01613, to appear in The Annals of Applied Probability.
[18]	L. Chen and J. Wang, Maximum principle for delayed stochastic mean-field control problem with state constraint, Advances in Difference Equations, (2019), Article Number 348. doi: 10.1186/s13662-019-2283-1.
[19]	Y.-L. Chow, X. Yu and C. Zhou, On dynamic programming principle for stochastic control under expectation constraints, Journal of Optimization Theory and Applications, 185 (2020), 803-818. doi: 10.1007/s10957-020-01673-2.
[20]	A. Cosso, F. Gozzi, I. Kharroubi, H. Pham and M. Rosestolato, Optimal control of path-dependent mckean-vlasov sdes in infinite dimension, arXiv: 2012.14772, to appear in Annals of Applied Probability.
[21]	A. Cosso and H. Pham, Zero-sum stochastic differential games of generalized mckean-vlasov type, Journal de Mathématiques Pures et Appliquées, 129 (2019), 180-212. doi: 10.1016/j.matpur.2018.12.005.
[22]	N. Curin, M. Kettler, X. Kleisinger-Yu, V. Komaric, T. Krabichler, J. Teichmann and H. Wutte, A deep learning model for gas storage optimization, arXiv: 2102.01980. doi: 10.1007/s10203-021-00363-6.
[23]	S. Daudin, Optimal control of diffusion processes with terminal constraint in law, arXiv: 2012.10707. doi: 10.1007/s10957-022-02053-8.
[24]	S. Daudin, Optimal control of the fokker-planck equation under state constraints in the wasserstein space, arXiv: 2109.14978.
[25]	M. F. Djete, D. Possama and X. Tan, Mckean-Vlasov optimal control: The dynamic programming principle, Annals of Probability, 50 (2022), 791-833. doi: 10.1214/21-aop1548.
[26]	G. Fu and U. Horst, Mean-field leader-follower games with terminal state constraint, SIAM Journal on Control and Optimization, 58 (2020), 2078-2113. doi: 10.1137/19M1241878.
[27]	A. Galichon, P. Henry-Labordère and N. Touzi, A stochastic control approach to no-arbitrage bounds given marginals, with an application to lookback options, The Annals of Applied Probability, 24 (2014), 312-336. doi: 10.1214/13-AAP925.
[28]	A. Geletu, M. Klppel, H. Zhang and P. Li, Advances and applications of chance-constrained approaches to systems optimisation under uncertainty, International Journal of Systems Science, 44 (2013), 1209-1232. doi: 10.1080/00207721.2012.670310.
[29]	H. Gevret, N. Langrené, J. Lelong, R. Lobato, T. Ouillon, X. Warin and A. Maheshwari, Stochastic Optimization library in c++, 2018, URL https://hal.archives-ouvertes.fr/hal-01361291.
[30]	J. Graber and S. Mayorga, A note on mean field games of controls with state constraints: existence of mild solutions, arXiv: 2109.11655.
[31]	D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, 3rd International Conference for Learning Representations, 2014.
[32]	M. Laurière and O. Pironneau, Dynamic programming for mean-field type control, Comptes Rendus Mathematique, 352 (2014), 707-713. doi: 10.1016/j.crma.2014.07.008.
[33]	M. Germain, M. Laurière, H. Pham and X. Warin, DeepSets and derivative networks for solving symmetric PDEs, Journal of Scientific Computing, 91 (2022), Article Number 63. doi: 10.1007/s10915-022-01796-w.
[34]	W. Lefebvre, G. Loeper and H. Pham, Mean-variance portfolio selection with tracking error penalization, Mathematics, 8 (2020).
[35]	S. M. Pesenti and S. Jaimungal, Portfolio optimisation within a wasserstein ball, Available at SSRN: https://ssrn.com/abstract=3744994.
[36]	L. Pfeiffer, X. Tan and Y.-L. Zhou, Duality and approximation of stochastic optimal control problems under expectation constraints, SIAM Journal on Control and Optimization, 59 (2021), 3231-3260. doi: 10.1137/20M1349886.
[37]	H. Pham and X. Wei, Dynamic programming for optimal control of stochastic mckean-vlasov dynamics, SIAM Journal on Control and Optimization, 55 (2017), 1069-1101. doi: 10.1137/16M1071390.
[38]	H. Pham and X. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM: COCV, 24 (2018), 437-461. doi: 10.1051/cocv/2017019.
[39]	A. Picarelli and T. Vargiolu, Optimal management of pumped hydroelectric production with state constrained optimal control, Journal of Economic Dynamics and Control, 126 (2021), 103940. doi: 10.1016/j.jedc.2020.103940.
[40]	T. Rockafellar, Convex Analysis Princeton University Press, 1970.
[41]	H. M. Soner and N. Touzi, Stochastic target problems, dynamic programming, and viscosity solutions, SIAM Journal on Control and Optimization, 41 (2002), 404-424. doi: 10.1137/S0363012900378863.
[42]	X. Warin, Deep learning for efficient frontier calculation in finance, arXiv: 2101.02044.
[43]	X. Warin, Reservoir optimization and machine learning methods, arXiv: 2106.08097.