American Institute of Mathematical Sciences

September & December  2018, 8(3&4): 557-582. doi: 10.3934/mcrf.2018023

Numerical study of polynomial feedback laws for a bilinear control problem

 1 Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria 2 RICAM Institute, Austrian Academy of Sciences, Altenbergerstrasse 69, 4040 Linz, Austria

* Corresponding author: Karl Kunisch

Received  September 2017 Revised  July 2018 Published  September 2018

An infinite-dimensional bilinear optimal control problem with infinite-time horizon is considered. The associated value function can be expanded in a Taylor series around the equilibrium, the Taylor series involving multilinear forms which are uniquely characterized by generalized Lyapunov equations. A numerical method for solving these equations is proposed. It is based on a generalization of the balanced truncation model reduction method and some techniques of tensor calculus, in order to attenuate the curse of dimensionality. Polynomial feedback laws are derived from the Taylor expansion and are numerically investigated for a control problem of the Fokker-Planck equation. Their efficiency is demonstrated for initial values which are sufficiently close to the equilibrium.

Citation: Tobias Breiten, Karl Kunisch, Laurent Pfeiffer. Numerical study of polynomial feedback laws for a bilinear control problem. Mathematical Control & Related Fields, 2018, 8 (3&4) : 557-582. doi: 10.3934/mcrf.2018023
References:

show all references

References:
1D Fokker-Planck equation
Comparison of the original and reduced models, for $n = 100$, $r = 25$, and $\beta = 10^{-4}$
Comparison of the original and reduced models, for $n = 100$, $r = 25$, and $\beta = 10^{-4}$
">Figure 4.  Comparison of the reduced models with $r = 21$ and $r = 9$, derived from a finer discretization (with $n = 1000$), with the setup of Figure 3
Convergence of the control laws for $\beta = 10^{-4}, n = 1000$ and $r = 9.$
Convergence of the control laws for $\beta = 10^{-4}, n = 1000$ and $r = 9.$
Initial condition and controls for the test case 2
Initial condition and controls for the test case 3
2D Fokker-Planck equation
Control shape functions α1 and α2
Singular value decay for n = 2500
Initial condition and controls for the test case 4
Initial condition and controls for the test case 5
Convergence results for the test case 1
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.038 0.038 0.038 0.038 0.038 0.038 1e$^{-4}$ 0.034 0.033 0.033 0.033 0.033 0.032 1e$^{-5}$ 0.037 0.031 0.031 0.031 0.031 0.030 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 0.228 0.026 0.024 0.024 0.024 1e$^{-4}$ 4.26 1.19 0.82 0.61 0.61 1e$^{-5}$ 29.8 10.3 7.91 4.70 4.05 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.038 0.038 0.038 0.038 0.038 0.038 1e$^{-4}$ 0.034 0.033 0.033 0.033 0.033 0.032 1e$^{-5}$ 0.037 0.031 0.031 0.031 0.031 0.030 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 0.228 0.026 0.024 0.024 0.024 1e$^{-4}$ 4.26 1.19 0.82 0.61 0.61 1e$^{-5}$ 29.8 10.3 7.91 4.70 4.05 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
Convergence results for the test case 2
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.156 0.155 0.155 0.155 0.155 0.154 5e$^{-4}$ 0.147 0.145 0.145 0.145 0.145 0.144 1e$^{-4}$ 0.138 0.122 0.120 0.120 0.120 0.119 5e$^{-5}$ 0.190 0.114 0.111 0.112 0.111 0.110 1e$^{-5}$ 0.205 0.194 0.104 0.111 0.113 0.095 (A) Cost of the controls up. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 1.149 0.169 0.119 0.034 0.031 5e$^{-4}$ 2.583 0.737 0.171 0.336 0.219 1e$^{-4}$ 18.50 7.02 3.16 4.01 1.52 5e$^{-5}$ 46.87 13.18 8.40 8.17 2.65 1e$^{-5}$ 90.5 78.0 39.0 42.6 34.3 (B) L2-distance between the controls up and the optimal control uopt.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.156 0.155 0.155 0.155 0.155 0.154 5e$^{-4}$ 0.147 0.145 0.145 0.145 0.145 0.144 1e$^{-4}$ 0.138 0.122 0.120 0.120 0.120 0.119 5e$^{-5}$ 0.190 0.114 0.111 0.112 0.111 0.110 1e$^{-5}$ 0.205 0.194 0.104 0.111 0.113 0.095 (A) Cost of the controls up. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 1.149 0.169 0.119 0.034 0.031 5e$^{-4}$ 2.583 0.737 0.171 0.336 0.219 1e$^{-4}$ 18.50 7.02 3.16 4.01 1.52 5e$^{-5}$ 46.87 13.18 8.40 8.17 2.65 1e$^{-5}$ 90.5 78.0 39.0 42.6 34.3 (B) L2-distance between the controls up and the optimal control uopt.
Convergence results for the test case 3
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.525 0.511 0.511 0.512 0.510 0.507 5e$^{-4}$ 0.451 0.417 0.431 0.459 0.446 0.408 1e$^{-4}$ 0.381 0.368 2.689 $\infty$ $\infty$ 0.246 5e$^{-5}$ 0.381 0.432 $\infty$ $\infty$ $\infty$ 0.206 1e$^{-5}$ 0.365 $\infty$ $\infty$ $\infty$ $\infty$ 0.147 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 4.88 1.50 1.77 2.31 1.52 5e$^{-4}$ 11.26 5.03 7.11 11.89 11.99 1e$^{-4}$ 46.34 35.36 57.08 $\infty$ $\infty$ 5e$^{-5}$ 74.79 60.86 $\infty$ $\infty$ $\infty$ 1e$^{-5}$ 172.3 $\infty$ $\infty$ $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.525 0.511 0.511 0.512 0.510 0.507 5e$^{-4}$ 0.451 0.417 0.431 0.459 0.446 0.408 1e$^{-4}$ 0.381 0.368 2.689 $\infty$ $\infty$ 0.246 5e$^{-5}$ 0.381 0.432 $\infty$ $\infty$ $\infty$ 0.206 1e$^{-5}$ 0.365 $\infty$ $\infty$ $\infty$ $\infty$ 0.147 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 4.88 1.50 1.77 2.31 1.52 5e$^{-4}$ 11.26 5.03 7.11 11.89 11.99 1e$^{-4}$ 46.34 35.36 57.08 $\infty$ $\infty$ 5e$^{-5}$ 74.79 60.86 $\infty$ $\infty$ $\infty$ 1e$^{-5}$ 172.3 $\infty$ $\infty$ $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
Convergence results for the test case 4
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.247 0.235 0.234 0.234 0.232 5e$^{-4}$ 0.232 0.207 0.205 0.205 0.203 1e$^{-4}$ 0.252 0.180 0.174 0.174 0.171 5e$^{-5}$ 0.279 0.179 0.168 0.168 0.165 1e$^{-5}$ 0.524 0.182 20.696 0.164 0.158 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-3}$ 3.53 0.80 0.19 0.14 5e$^{-4}$ 6.73 1.42 0.37 0.24 1e$^{-4}$ 27.40 5.78 1.83 1.24 5e$^{-5}$ 52.50 11.06 3.69 2.40 1e$^{-5}$ 257.01 63.97 84.31 10.61 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.247 0.235 0.234 0.234 0.232 5e$^{-4}$ 0.232 0.207 0.205 0.205 0.203 1e$^{-4}$ 0.252 0.180 0.174 0.174 0.171 5e$^{-5}$ 0.279 0.179 0.168 0.168 0.165 1e$^{-5}$ 0.524 0.182 20.696 0.164 0.158 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-3}$ 3.53 0.80 0.19 0.14 5e$^{-4}$ 6.73 1.42 0.37 0.24 1e$^{-4}$ 27.40 5.78 1.83 1.24 5e$^{-5}$ 52.50 11.06 3.69 2.40 1e$^{-5}$ 257.01 63.97 84.31 10.61 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
Convergence results for the test case 5
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-1}$ 7.58 7.57 7.57 7.57 7.52 5e$^{-2}$ 6.41 6.39 6.40 6.39 6.35 1e$^{-2}$ 3.70 3.34 3.09 3.32 3.00 5e$^{-3}$ 3.07 2.68 2.28 2.96 2.05 1e$^{-3}$ 2.45 2.41 $\infty$ $\infty$ 0.93 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-1}$ 0.70 0.61 0.62 0.60 5e$^{-2}$ 1.10 0.69 0.80 0.63 1e$^{-2}$ 13.02 11.10 4.08 9.01 5e$^{-3}$ 21.59 19.80 9.66 20.06 1e$^{-3}$ 47.34 55.69 $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-1}$ 7.58 7.57 7.57 7.57 7.52 5e$^{-2}$ 6.41 6.39 6.40 6.39 6.35 1e$^{-2}$ 3.70 3.34 3.09 3.32 3.00 5e$^{-3}$ 3.07 2.68 2.28 2.96 2.05 1e$^{-3}$ 2.45 2.41 $\infty$ $\infty$ 0.93 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-1}$ 0.70 0.61 0.62 0.60 5e$^{-2}$ 1.10 0.69 0.80 0.63 1e$^{-2}$ 13.02 11.10 4.08 9.01 5e$^{-3}$ 21.59 19.80 9.66 20.06 1e$^{-3}$ 47.34 55.69 $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 [1] Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369 [2] Bian-Xia Yang, Shanshan Gu, Guowei Dai. Existence and multiplicity for Hamilton-Jacobi-Bellman equation. Communications on Pure & Applied Analysis, 2021, 20 (11) : 3767-3793. doi: 10.3934/cpaa.2021130 [3] Sylvain De Moor, Luis Miguel Rodrigues, Julien Vovelle. Invariant measures for a stochastic Fokker-Planck equation. Kinetic & Related Models, 2018, 11 (2) : 357-395. doi: 10.3934/krm.2018017 [4] Marco Torregrossa, Giuseppe Toscani. On a Fokker-Planck equation for wealth distribution. Kinetic & Related Models, 2018, 11 (2) : 337-355. doi: 10.3934/krm.2018016 [5] Michael Herty, Christian Jörres, Albert N. Sandjo. Optimization of a model Fokker-Planck equation. Kinetic & Related Models, 2012, 5 (3) : 485-503. doi: 10.3934/krm.2012.5.485 [6] José Antonio Alcántara, Simone Calogero. On a relativistic Fokker-Planck equation in kinetic theory. Kinetic & Related Models, 2011, 4 (2) : 401-426. doi: 10.3934/krm.2011.4.401 [7] Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161 [8] Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation. Electronic Research Archive, 2021, 29 (5) : 3429-3447. doi: 10.3934/era.2021046 [9] Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251 [10] Fabio Camilli, Serikbolsyn Duisembay, Qing Tang. Approximation of an optimal control problem for the time-fractional Fokker-Planck equation. Journal of Dynamics & Games, 2021, 8 (4) : 381-402. doi: 10.3934/jdg.2021013 [11] Helge Dietert, Josephine Evans, Thomas Holding. Contraction in the Wasserstein metric for the kinetic Fokker-Planck equation on the torus. Kinetic & Related Models, 2018, 11 (6) : 1427-1441. doi: 10.3934/krm.2018056 [12] Andreas Denner, Oliver Junge, Daniel Matthes. Computing coherent sets using the Fokker-Planck equation. Journal of Computational Dynamics, 2016, 3 (2) : 163-177. doi: 10.3934/jcd.2016008 [13] Ioannis Markou. Hydrodynamic limit for a Fokker-Planck equation with coefficients in Sobolev spaces. Networks & Heterogeneous Media, 2017, 12 (4) : 683-705. doi: 10.3934/nhm.2017028 [14] Manh Hong Duong, Yulong Lu. An operator splitting scheme for the fractional kinetic Fokker-Planck equation. Discrete & Continuous Dynamical Systems, 2019, 39 (10) : 5707-5727. doi: 10.3934/dcds.2019250 [15] Giuseppe Toscani. A Rosenau-type approach to the approximation of the linear Fokker-Planck equation. Kinetic & Related Models, 2018, 11 (4) : 697-714. doi: 10.3934/krm.2018028 [16] Larry M. Bates, Francesco Fassò, Nicola Sansonetto. The Hamilton-Jacobi equation, integrability, and nonholonomic systems. Journal of Geometric Mechanics, 2014, 6 (4) : 441-449. doi: 10.3934/jgm.2014.6.441 [17] Xuhui Wang, Lei Hu. A new method to solve the Hamilton-Jacobi-Bellman equation for a stochastic portfolio optimization model with boundary memory. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021137 [18] Ludovic Dan Lemle. $L^1(R^d,dx)$-uniqueness of weak solutions for the Fokker-Planck equation associated with a class of Dirichlet operators. Electronic Research Announcements, 2008, 15: 65-70. doi: 10.3934/era.2008.15.65 [19] Joseph G. Conlon, André Schlichting. A non-local problem for the Fokker-Planck equation related to the Becker-Döring model. Discrete & Continuous Dynamical Systems, 2019, 39 (4) : 1821-1889. doi: 10.3934/dcds.2019079 [20] Simon Plazotta. A BDF2-approach for the non-linear Fokker-Planck equation. Discrete & Continuous Dynamical Systems, 2019, 39 (5) : 2893-2913. doi: 10.3934/dcds.2019120

2020 Impact Factor: 1.284