# American Institute of Mathematical Sciences

September  2018, 8(3&4): 557-582. doi: 10.3934/mcrf.2018023

## Numerical study of polynomial feedback laws for a bilinear control problem

 1 Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria 2 RICAM Institute, Austrian Academy of Sciences, Altenbergerstrasse 69, 4040 Linz, Austria

* Corresponding author: Karl Kunisch

Received  September 2017 Revised  July 2018 Published  September 2018

An infinite-dimensional bilinear optimal control problem with infinite-time horizon is considered. The associated value function can be expanded in a Taylor series around the equilibrium, the Taylor series involving multilinear forms which are uniquely characterized by generalized Lyapunov equations. A numerical method for solving these equations is proposed. It is based on a generalization of the balanced truncation model reduction method and some techniques of tensor calculus, in order to attenuate the curse of dimensionality. Polynomial feedback laws are derived from the Taylor expansion and are numerically investigated for a control problem of the Fokker-Planck equation. Their efficiency is demonstrated for initial values which are sufficiently close to the equilibrium.

Citation: Tobias Breiten, Karl Kunisch, Laurent Pfeiffer. Numerical study of polynomial feedback laws for a bilinear control problem. Mathematical Control & Related Fields, 2018, 8 (3&4) : 557-582. doi: 10.3934/mcrf.2018023
##### References:

show all references

##### References:
1D Fokker-Planck equation
Comparison of the original and reduced models, for $n = 100$, $r = 25$, and $\beta = 10^{-4}$
Comparison of the original and reduced models, for $n = 100$, $r = 25$, and $\beta = 10^{-4}$
Comparison of the reduced models with $r = 21$ and $r = 9$, derived from a finer discretization (with $n = 1000$), with the setup of Figure 3
Convergence of the control laws for $\beta = 10^{-4}, n = 1000$ and $r = 9.$
Convergence of the control laws for $\beta = 10^{-4}, n = 1000$ and $r = 9.$
Initial condition and controls for the test case 2
Initial condition and controls for the test case 3
2D Fokker-Planck equation
Control shape functions α1 and α2
Singular value decay for n = 2500
Initial condition and controls for the test case 4
Initial condition and controls for the test case 5
Convergence results for the test case 1
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.038 0.038 0.038 0.038 0.038 0.038 1e$^{-4}$ 0.034 0.033 0.033 0.033 0.033 0.032 1e$^{-5}$ 0.037 0.031 0.031 0.031 0.031 0.030 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 0.228 0.026 0.024 0.024 0.024 1e$^{-4}$ 4.26 1.19 0.82 0.61 0.61 1e$^{-5}$ 29.8 10.3 7.91 4.70 4.05 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.038 0.038 0.038 0.038 0.038 0.038 1e$^{-4}$ 0.034 0.033 0.033 0.033 0.033 0.032 1e$^{-5}$ 0.037 0.031 0.031 0.031 0.031 0.030 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 0.228 0.026 0.024 0.024 0.024 1e$^{-4}$ 4.26 1.19 0.82 0.61 0.61 1e$^{-5}$ 29.8 10.3 7.91 4.70 4.05 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
Convergence results for the test case 2
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.156 0.155 0.155 0.155 0.155 0.154 5e$^{-4}$ 0.147 0.145 0.145 0.145 0.145 0.144 1e$^{-4}$ 0.138 0.122 0.120 0.120 0.120 0.119 5e$^{-5}$ 0.190 0.114 0.111 0.112 0.111 0.110 1e$^{-5}$ 0.205 0.194 0.104 0.111 0.113 0.095 (A) Cost of the controls up. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 1.149 0.169 0.119 0.034 0.031 5e$^{-4}$ 2.583 0.737 0.171 0.336 0.219 1e$^{-4}$ 18.50 7.02 3.16 4.01 1.52 5e$^{-5}$ 46.87 13.18 8.40 8.17 2.65 1e$^{-5}$ 90.5 78.0 39.0 42.6 34.3 (B) L2-distance between the controls up and the optimal control uopt.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.156 0.155 0.155 0.155 0.155 0.154 5e$^{-4}$ 0.147 0.145 0.145 0.145 0.145 0.144 1e$^{-4}$ 0.138 0.122 0.120 0.120 0.120 0.119 5e$^{-5}$ 0.190 0.114 0.111 0.112 0.111 0.110 1e$^{-5}$ 0.205 0.194 0.104 0.111 0.113 0.095 (A) Cost of the controls up. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 1.149 0.169 0.119 0.034 0.031 5e$^{-4}$ 2.583 0.737 0.171 0.336 0.219 1e$^{-4}$ 18.50 7.02 3.16 4.01 1.52 5e$^{-5}$ 46.87 13.18 8.40 8.17 2.65 1e$^{-5}$ 90.5 78.0 39.0 42.6 34.3 (B) L2-distance between the controls up and the optimal control uopt.
Convergence results for the test case 3
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.525 0.511 0.511 0.512 0.510 0.507 5e$^{-4}$ 0.451 0.417 0.431 0.459 0.446 0.408 1e$^{-4}$ 0.381 0.368 2.689 $\infty$ $\infty$ 0.246 5e$^{-5}$ 0.381 0.432 $\infty$ $\infty$ $\infty$ 0.206 1e$^{-5}$ 0.365 $\infty$ $\infty$ $\infty$ $\infty$ 0.147 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 4.88 1.50 1.77 2.31 1.52 5e$^{-4}$ 11.26 5.03 7.11 11.89 11.99 1e$^{-4}$ 46.34 35.36 57.08 $\infty$ $\infty$ 5e$^{-5}$ 74.79 60.86 $\infty$ $\infty$ $\infty$ 1e$^{-5}$ 172.3 $\infty$ $\infty$ $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_6)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.525 0.511 0.511 0.512 0.510 0.507 5e$^{-4}$ 0.451 0.417 0.431 0.459 0.446 0.408 1e$^{-4}$ 0.381 0.368 2.689 $\infty$ $\infty$ 0.246 5e$^{-5}$ 0.381 0.432 $\infty$ $\infty$ $\infty$ 0.206 1e$^{-5}$ 0.365 $\infty$ $\infty$ $\infty$ $\infty$ 0.147 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ $p=6$ 1e$^{-3}$ 4.88 1.50 1.77 2.31 1.52 5e$^{-4}$ 11.26 5.03 7.11 11.89 11.99 1e$^{-4}$ 46.34 35.36 57.08 $\infty$ $\infty$ 5e$^{-5}$ 74.79 60.86 $\infty$ $\infty$ $\infty$ 1e$^{-5}$ 172.3 $\infty$ $\infty$ $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
Convergence results for the test case 4
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.247 0.235 0.234 0.234 0.232 5e$^{-4}$ 0.232 0.207 0.205 0.205 0.203 1e$^{-4}$ 0.252 0.180 0.174 0.174 0.171 5e$^{-5}$ 0.279 0.179 0.168 0.168 0.165 1e$^{-5}$ 0.524 0.182 20.696 0.164 0.158 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-3}$ 3.53 0.80 0.19 0.14 5e$^{-4}$ 6.73 1.42 0.37 0.24 1e$^{-4}$ 27.40 5.78 1.83 1.24 5e$^{-5}$ 52.50 11.06 3.69 2.40 1e$^{-5}$ 257.01 63.97 84.31 10.61 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-3}$ 0.247 0.235 0.234 0.234 0.232 5e$^{-4}$ 0.232 0.207 0.205 0.205 0.203 1e$^{-4}$ 0.252 0.180 0.174 0.174 0.171 5e$^{-5}$ 0.279 0.179 0.168 0.168 0.165 1e$^{-5}$ 0.524 0.182 20.696 0.164 0.158 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-3}$ 3.53 0.80 0.19 0.14 5e$^{-4}$ 6.73 1.42 0.37 0.24 1e$^{-4}$ 27.40 5.78 1.83 1.24 5e$^{-5}$ 52.50 11.06 3.69 2.40 1e$^{-5}$ 257.01 63.97 84.31 10.61 (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
Convergence results for the test case 5
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-1}$ 7.58 7.57 7.57 7.57 7.52 5e$^{-2}$ 6.41 6.39 6.40 6.39 6.35 1e$^{-2}$ 3.70 3.34 3.09 3.32 3.00 5e$^{-3}$ 3.07 2.68 2.28 2.96 2.05 1e$^{-3}$ 2.45 2.41 $\infty$ $\infty$ 0.93 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-1}$ 0.70 0.61 0.62 0.60 5e$^{-2}$ 1.10 0.69 0.80 0.63 1e$^{-2}$ 13.02 11.10 4.08 9.01 5e$^{-3}$ 21.59 19.80 9.66 20.06 1e$^{-3}$ 47.34 55.69 $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 $\beta$ $J(u_2)$ $J(u_3)$ $J(u_4)$ $J(u_5)$ $J(u_{\text{opt}})$ 1e$^{-1}$ 7.58 7.57 7.57 7.57 7.52 5e$^{-2}$ 6.41 6.39 6.40 6.39 6.35 1e$^{-2}$ 3.70 3.34 3.09 3.32 3.00 5e$^{-3}$ 3.07 2.68 2.28 2.96 2.05 1e$^{-3}$ 2.45 2.41 $\infty$ $\infty$ 0.93 (A) Cost of the controls $u_p$. $\beta$ $\| u_p-u_{\text{opt}} \|_{L^2(0, T)}$ $p=2$ $p=3$ $p=4$ $p=5$ 1e$^{-1}$ 0.70 0.61 0.62 0.60 5e$^{-2}$ 1.10 0.69 0.80 0.63 1e$^{-2}$ 13.02 11.10 4.08 9.01 5e$^{-3}$ 21.59 19.80 9.66 20.06 1e$^{-3}$ 47.34 55.69 $\infty$ $\infty$ (B) $L^2$-distance between the controls $u_p$ and the optimal control $u_{\text{opt}}$.
 [1] Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369 [2] Sylvain De Moor, Luis Miguel Rodrigues, Julien Vovelle. Invariant measures for a stochastic Fokker-Planck equation. Kinetic & Related Models, 2018, 11 (2) : 357-395. doi: 10.3934/krm.2018017 [3] Michael Herty, Christian Jörres, Albert N. Sandjo. Optimization of a model Fokker-Planck equation. Kinetic & Related Models, 2012, 5 (3) : 485-503. doi: 10.3934/krm.2012.5.485 [4] Marco Torregrossa, Giuseppe Toscani. On a Fokker-Planck equation for wealth distribution. Kinetic & Related Models, 2018, 11 (2) : 337-355. doi: 10.3934/krm.2018016 [5] José Antonio Alcántara, Simone Calogero. On a relativistic Fokker-Planck equation in kinetic theory. Kinetic & Related Models, 2011, 4 (2) : 401-426. doi: 10.3934/krm.2011.4.401 [6] Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161 [7] Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251 [8] Helge Dietert, Josephine Evans, Thomas Holding. Contraction in the Wasserstein metric for the kinetic Fokker-Planck equation on the torus. Kinetic & Related Models, 2018, 11 (6) : 1427-1441. doi: 10.3934/krm.2018056 [9] Andreas Denner, Oliver Junge, Daniel Matthes. Computing coherent sets using the Fokker-Planck equation. Journal of Computational Dynamics, 2016, 3 (2) : 163-177. doi: 10.3934/jcd.2016008 [10] Ioannis Markou. Hydrodynamic limit for a Fokker-Planck equation with coefficients in Sobolev spaces. Networks & Heterogeneous Media, 2017, 12 (4) : 683-705. doi: 10.3934/nhm.2017028 [11] Giuseppe Toscani. A Rosenau-type approach to the approximation of the linear Fokker-Planck equation. Kinetic & Related Models, 2018, 11 (4) : 697-714. doi: 10.3934/krm.2018028 [12] Manh Hong Duong, Yulong Lu. An operator splitting scheme for the fractional kinetic Fokker-Planck equation. Discrete & Continuous Dynamical Systems - A, 2019, 39 (10) : 5707-5727. doi: 10.3934/dcds.2019250 [13] Larry M. Bates, Francesco Fassò, Nicola Sansonetto. The Hamilton-Jacobi equation, integrability, and nonholonomic systems. Journal of Geometric Mechanics, 2014, 6 (4) : 441-449. doi: 10.3934/jgm.2014.6.441 [14] Ludovic Dan Lemle. $L^1(R^d,dx)$-uniqueness of weak solutions for the Fokker-Planck equation associated with a class of Dirichlet operators. Electronic Research Announcements, 2008, 15: 65-70. doi: 10.3934/era.2008.15.65 [15] Joseph G. Conlon, André Schlichting. A non-local problem for the Fokker-Planck equation related to the Becker-Döring model. Discrete & Continuous Dynamical Systems - A, 2019, 39 (4) : 1821-1889. doi: 10.3934/dcds.2019079 [16] Simon Plazotta. A BDF2-approach for the non-linear Fokker-Planck equation. Discrete & Continuous Dynamical Systems - A, 2019, 39 (5) : 2893-2913. doi: 10.3934/dcds.2019120 [17] Patrick Cattiaux, Elissar Nasreddine, Marjolaine Puel. Diffusion limit for kinetic Fokker-Planck equation with heavy tails equilibria: The critical case. Kinetic & Related Models, 2019, 12 (4) : 727-748. doi: 10.3934/krm.2019028 [18] Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295 [19] Florian Schneider, Andreas Roth, Jochen Kall. First-order quarter-and mixed-moment realizability theory and Kershaw closures for a Fokker-Planck equation in two space dimensions. Kinetic & Related Models, 2017, 10 (4) : 1127-1161. doi: 10.3934/krm.2017044 [20] Shui-Nee Chow, Wuchen Li, Haomin Zhou. Entropy dissipation of Fokker-Planck equations on graphs. Discrete & Continuous Dynamical Systems - A, 2018, 38 (10) : 4929-4950. doi: 10.3934/dcds.2018215

2018 Impact Factor: 1.292