# American Institute of Mathematical Sciences

doi: 10.3934/mcrf.2021003
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

## Learning nonlocal regularization operators

 1 Institute of Mathematics and Scientific Computing, University of Graz, Heinrichstrasse 36, 8010 Graz, Austria 2 Radon Institute, Austrian Academy of Sciences, RICAM Linz, Altenbergerstrasse 69, 4040 Linz, Austria

* Corresponding author: Gernot Holler

Received  February 2020 Revised  September 2020 Early access January 2021

A learning approach for determining which operator from a class of nonlocal operators is optimal for the regularization of an inverse problem is investigated. The considered class of nonlocal operators is motivated by the use of squared fractional order Sobolev seminorms as regularization operators. First fundamental results from the theory of regularization with local operators are extended to the nonlocal case. Then a framework based on a bilevel optimization strategy is developed which allows to choose nonlocal regularization operators from a given class which i) are optimal with respect to a suitable performance measure on a training set, and ii) enjoy particularly favorable properties. Results from numerical experiments are also provided.

Citation: Gernot Holler, Karl Kunisch. Learning nonlocal regularization operators. Mathematical Control & Related Fields, doi: 10.3934/mcrf.2021003
##### References:

show all references

##### References:
Optimal weights for linear state equation in case (A) (first row) and case (B) (second row) using $\beta = 0$. The training set consisted of 512 data vectors given in a single batch
Ground truth and reconstructed controls for one data vector from the validation set for the linear state equation in case (B) using $\beta = 0$. The training set consisted of 512 data vectors given in a single batch
Optimal weights for $s = 0.1$ and various $L^1$ regularization parameters $\beta$ in case (B). The training and validation set both consisted of 512 data vectors given in a single batch. The average validation errors were $1.42 \times {10^{ - 2}}$ ($\beta = 10^{-2}$), $1.18\times {10^{ - 2}}$ ($\beta = 10^{-3}$), and $1.03\times {10^{ - 2}}$ ($\beta = 10^{-4}$)
Average training and validation error for optimal regularization parameter $\nu^*$ (second and third column) and optimal weight $\sigma^*$ (fourth and fifth column) in case (A) using $\beta = 0$. The training and validation set both consisted of 512 data vectors
 (a) s = 0:1 batchsize train error reg val error reg train error weight val error weight 8 1.81 × 10−2 2.19 × 10−2 1.42 × 10−2 2.04 × 10−2 64 1.88 × 10−2 2.11 × 10−2 1.61 × 10−2 1.86 × 10−2 512 1.90 × 10−2 2.09 × 10−2 1.65 × 10−2 1.82 × 10−2 (b) s = 0:9 8 1.53 × 10−2 1.98 × 10−2 1.51 × 10−2 1.97 × 10−2 64 1.63 × 10−2 1.85 × 10−2 1.61 × 10−2 1.84 × 10−2 512 1.65 × 10−2 1.82 × 10−2 1.64 × 10−2 1.80 × 10−2 (c) L2 and H1 regularization with optimal ν* batchsize train error L2 val error L2 train error H1 val error H1 8 1.93 × 10−2 2.30 × 10−2 1.51 × 10−2 1.97 × 10−2 64 1.99 × 10−2 2.22 × 10−2 1.61 × 10−2 1.84 × 10−2 512 2.01 × 10−2 2.20 × 10−2 1.64 × 10−2 1.80 × 10−2
 (a) s = 0:1 batchsize train error reg val error reg train error weight val error weight 8 1.81 × 10−2 2.19 × 10−2 1.42 × 10−2 2.04 × 10−2 64 1.88 × 10−2 2.11 × 10−2 1.61 × 10−2 1.86 × 10−2 512 1.90 × 10−2 2.09 × 10−2 1.65 × 10−2 1.82 × 10−2 (b) s = 0:9 8 1.53 × 10−2 1.98 × 10−2 1.51 × 10−2 1.97 × 10−2 64 1.63 × 10−2 1.85 × 10−2 1.61 × 10−2 1.84 × 10−2 512 1.65 × 10−2 1.82 × 10−2 1.64 × 10−2 1.80 × 10−2 (c) L2 and H1 regularization with optimal ν* batchsize train error L2 val error L2 train error H1 val error H1 8 1.93 × 10−2 2.30 × 10−2 1.51 × 10−2 1.97 × 10−2 64 1.99 × 10−2 2.22 × 10−2 1.61 × 10−2 1.84 × 10−2 512 2.01 × 10−2 2.20 × 10−2 1.64 × 10−2 1.80 × 10−2
Average training and validation error for optimal regularization parameter $\nu^*$ (second and third column) and optimal weight $\sigma^*$ (fourth and fifth column) in case (B) using $\beta = 0$. The training and validation set both consisted of 512 data vectors
 (a) s = 0.1 batchsize train error reg val error reg train error weight val error weight 8 2.03 × 10−2 2.12 × 10−2 1.02 × 10−2 1.09 × 10−2 64 2.05 × 10−2 2.09 × 10−2 1.02 × 10−2 1.04 × 10−2 512 2.06 × 10−2 2.08 × 10−2 1.14 × 10−2 1.15 × 10−2 (b) s = 0.9 8 1.37 × 10−2 1.44 × 10−2 1.33 × 10−2 1.41 × 10−2 64 1.38 × 10−2 1.42 × 10−2 1.35 × 10−2 1.38 × 10−2 512 1.39 × 10−2 1.41 × 10−2 1.35 × 10−2 1.38 × 10−2 (c) L2 and H1 regularization with optimal ν* 8 2.48 × 10−2 2.60 × 10−2 1.33 × 10−2 1.40 × 10−2 64 2.51 × 10−2 2.56 × 10−2 1.35 × 10−2 1.38 × 10−2 512 2.52 × 10−2 2.55 × 10−2 1.35 × 10−2 1.38 × 10−2
 (a) s = 0.1 batchsize train error reg val error reg train error weight val error weight 8 2.03 × 10−2 2.12 × 10−2 1.02 × 10−2 1.09 × 10−2 64 2.05 × 10−2 2.09 × 10−2 1.02 × 10−2 1.04 × 10−2 512 2.06 × 10−2 2.08 × 10−2 1.14 × 10−2 1.15 × 10−2 (b) s = 0.9 8 1.37 × 10−2 1.44 × 10−2 1.33 × 10−2 1.41 × 10−2 64 1.38 × 10−2 1.42 × 10−2 1.35 × 10−2 1.38 × 10−2 512 1.39 × 10−2 1.41 × 10−2 1.35 × 10−2 1.38 × 10−2 (c) L2 and H1 regularization with optimal ν* 8 2.48 × 10−2 2.60 × 10−2 1.33 × 10−2 1.40 × 10−2 64 2.51 × 10−2 2.56 × 10−2 1.35 × 10−2 1.38 × 10−2 512 2.52 × 10−2 2.55 × 10−2 1.35 × 10−2 1.38 × 10−2
 [1] Shakoor Pooseh, Ricardo Almeida, Delfim F. M. Torres. Fractional order optimal control problems with free terminal time. Journal of Industrial & Management Optimization, 2014, 10 (2) : 363-381. doi: 10.3934/jimo.2014.10.363 [2] Jianxiong Ye, An Li. Necessary optimality conditions for nonautonomous optimal control problems and its applications to bilevel optimal control. Journal of Industrial & Management Optimization, 2019, 15 (3) : 1399-1419. doi: 10.3934/jimo.2018101 [3] Tuğba Akman Yıldız, Amin Jajarmi, Burak Yıldız, Dumitru Baleanu. New aspects of time fractional optimal control problems within operators with nonsingular kernel. Discrete & Continuous Dynamical Systems - S, 2020, 13 (3) : 407-428. doi: 10.3934/dcdss.2020023 [4] Enkhbat Rentsen, J. Zhou, K. L. Teo. A global optimization approach to fractional optimal control. Journal of Industrial & Management Optimization, 2016, 12 (1) : 73-82. doi: 10.3934/jimo.2016.12.73 [5] Giuseppe Buttazzo, Lorenzo Freddi. Optimal control problems with weakly converging input operators. Discrete & Continuous Dynamical Systems, 1995, 1 (3) : 401-420. doi: 10.3934/dcds.1995.1.401 [6] Martin Benning, Elena Celledoni, Matthias J. Ehrhardt, Brynjulf Owren, Carola-Bibiane Schönlieb. Deep learning as optimal control problems: Models and numerical methods. Journal of Computational Dynamics, 2019, 6 (2) : 171-198. doi: 10.3934/jcd.2019009 [7] Sören Bartels, Nico Weber. Parameter learning and fractional differential operators: Applications in regularized image denoising and decomposition problems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021048 [8] Ru-Yu Lai, Laurel Ohm. Inverse problems for the fractional Laplace equation with lower order nonlinear perturbations. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021051 [9] Barbara Kaltenbacher, William Rundell. Regularization of a backwards parabolic equation by fractional operators. Inverse Problems & Imaging, 2019, 13 (2) : 401-430. doi: 10.3934/ipi.2019020 [10] Sebastian Albrecht, Marion Leibold, Michael Ulbrich. A bilevel optimization approach to obtain optimal cost functions for human arm movements. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 105-127. doi: 10.3934/naco.2012.2.105 [11] Paul B. Hermanns, Nguyen Van Thoai. Global optimization algorithm for solving bilevel programming problems with quadratic lower levels. Journal of Industrial & Management Optimization, 2010, 6 (1) : 177-196. doi: 10.3934/jimo.2010.6.177 [12] Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems & Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511 [13] Philipp Hungerländer, Barbara Kaltenbacher, Franz Rendl. Regularization of inverse problems via box constrained minimization. Inverse Problems & Imaging, 2020, 14 (3) : 437-461. doi: 10.3934/ipi.2020021 [14] Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399 [15] Frank Pörner, Daniel Wachsmuth. Tikhonov regularization of optimal control problems governed by semi-linear partial differential equations. Mathematical Control & Related Fields, 2018, 8 (1) : 315-335. doi: 10.3934/mcrf.2018013 [16] Omid S. Fard, Javad Soolaki, Delfim F. M. Torres. A necessary condition of Pontryagin type for fuzzy fractional optimal control problems. Discrete & Continuous Dynamical Systems - S, 2018, 11 (1) : 59-76. doi: 10.3934/dcdss.2018004 [17] Vaibhav Mehandiratta, Mani Mehra, Günter Leugering. Fractional optimal control problems on a star graph: Optimality system and numerical solution. Mathematical Control & Related Fields, 2021, 11 (1) : 189-209. doi: 10.3934/mcrf.2020033 [18] Leonardo Colombo, David Martín de Diego. Second-order variational problems on Lie groupoids and optimal control applications. Discrete & Continuous Dynamical Systems, 2016, 36 (11) : 6023-6064. doi: 10.3934/dcds.2016064 [19] Leonardo Colombo, David Martín de Diego. Higher-order variational problems on lie groups and optimal control applications. Journal of Geometric Mechanics, 2014, 6 (4) : 451-478. doi: 10.3934/jgm.2014.6.451 [20] Pierluigi Colli, Gianni Gilardi, Jürgen Sprekels. Deep quench approximation and optimal control of general Cahn–Hilliard systems with fractional operators and double obstacle potentials. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 243-271. doi: 10.3934/dcdss.2020213

2020 Impact Factor: 1.284