We look at continuum solutions in optimisation problems associated to linear inverse problems $ y = Ax $ with non-negativity constraint $ x \geq 0 $. We focus on the case where the noise model leads to maximum likelihood estimation through general divergences, which cover a wide range of common noise statistics such as Gaussian and Poisson. Considering $ x $ as a Radon measure over the domain on which the reconstruction is taking place, we show a general singularity result. In the high noise regime corresponding to $ y \notin\{A x \mid x \geq 0\} $ and under a key assumption on the divergence as well as on the operator $ A $, any optimiser has a singular part with respect to the Lebesgue measure. We hence provide an explanation as to why any possible algorithm successfully solving the optimisation problem will lead to undesirably spiky-looking images when the image resolution gets finer, a phenomenon well documented in the literature. We illustrate these results with several numerical examples inspired by medical imaging.
| Citation: |
Figure 1. An example of map $v \mapsto d(1, v) $ which fulfills the assumptions above. In particular, the derivative tends to −∞ when v approaches zero, and the divergence is zero only at the point v = 1. Note that this is a plot of the function $v \mapsto d_\beta(1, v) $ (see subsubsection 2.3.1) for β = 1.2
Figure 3. Two views of the functions described in subsection 4.2: here the two detectors are the line segment $ [u,v] $ with $ u = 1+2i $, $ v \sim -0.7 + 1.4i $, and the symmetric line segment $ [\bar u, \bar v] $ with respect to the real axis. We only plot the function on half of its domain since it is symmetric with respect to the real axis. As can be seen, the functions cannot be continuously extended up to the vertices
Figure 4. Example of PET functions for the regular polygonk $ P_N $, with $ N = 12 $. Outside of the pale blue region, $ a_{7,8} $ vanishes. Outside of the pale green region (a trapezium whose diagonals intersect at the point denoted $ c = c_{0,4} $), the function $ a_{0,4} $ vanishes. At the point $ z $, $ a_{0,4}(z) $ is given by $ \frac{\alpha}{\pi} $
Figure 7. The singular solutions of the problem (6) with two detectors, a1 = 1, a2(x) = x on the interval K = [0, 1]. There are three distinct regions outside the cone $A \mathcal{M}_{+} $, but only one which intersects the first quadrant $ y \in {\mathbb R}_+^2 $, where the optimal solution is a Dirac, given by ξδ1 for some $ \xi \geq 0 $
Figure 8. The solutions for $ y = (0,1) $ of the problem in Figure 7 with total variation regularisation. Each curve corresponds to a different regularisation parameter, labelled by its base-10 logarithm. We see that when the regularisation parameter goes to zero, the computed solution converges to the expected exact solution, which, we see from Figure 7, is µ = .5δ1, depicted by a blue circle
| [1] |
J. Adler, H. Kohr and O. Öktem, ODL-a Python framework for rapid prototyping in inverse problems, Royal Institute of Technology, (2017).
|
| [2] |
C. Boyer, A. Chambolle, Y. D. Castro, V. Duval, F. De Gournay and P. Weiss, On representer theorems and convex regularization, SIAM Journal on Optimization, 29 (2019), 1260-1281.
doi: 10.1137/18M1200750.
|
| [3] |
K. Bredies and M. Carioni, Sparsity of solutions for variational inverse problems with finite-dimensional data, Calculus of Variations and Partial Differential Equations, 59 (2020), Paper No. 14, 26 pp.
doi: 10.1007/s00526-019-1658-1.
|
| [4] |
H. Brezis, Functional Aanalysis, Sobolev Spaces and Partial Differential Equations, Springer Science & Business Media, 2010.
|
| [5] |
Y. C. Cavalcanti, T. Oberlin, N. Dobigeon, C. Févotte, S. Stute, M.-J. Ribeiro and C. Tauber, Factor analysis of dynamic PET images: Beyond Gaussian noise, IEEE Transactions on Medical Imaging, (2019).
|
| [6] |
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2010), 120-145.
doi: 10.1007/s10851-010-0251-1.
|
| [7] |
A. Cichocki and S.-i. Amari, Families of alpha-beta-and gamma-divergences: Flexible and robust measures of similarities, Entropy, 12 (2010), 1532-1568.
doi: 10.3390/e12061532.
|
| [8] |
C. Clason, B. Kaltenbacher and E. Resmerita, Regularization of ill-posed problems with non-negative solutions, Splitting Algorithms, Modern Operator Theory, and Applications, (2019), 113-135.
|
| [9] |
C. Clason and A. Schiela, Optimal control of elliptic equations with positive measures, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 217-240.
doi: 10.1051/cocv/2015046.
|
| [10] |
T. Debarre, Q. Denoyelle and J. Fageot, On the uniqueness of solutions for the basis pursuit in the continuum, Inverse Problems, 38 (2022), 125005, 19 pp, arXiv: 2009.11855.
doi: 10.1088/1361-6420/ac9998.
|
| [11] |
Q. Denoyelle, V. Duval and G. Peyré, Support recovery for sparse super-resolution of positive measures, Journal of Fourier Analysis and Applications, 23 (2017), 1153-1194.
doi: 10.1007/s00041-016-9502-x.
|
| [12] |
A. R. De Pierro, On the relation between the ISRA and the EM algorithm for positron emission tomography, IEEE Transactions on Medical Imaging, 12 (1993), 328-333.
doi: 10.1109/42.232263.
|
| [13] |
C. Févotte and J. Idier, Algorithms for nonnegative matrix factorization with the $\beta$-divergence, Neural Computation, 23 (2011), 2421-2456.
doi: 10.1162/NECO_a_00168.
|
| [14] |
T. T. Georgiou, Solution of the general moment problem via a one-parameter imbedding, IEEE Transactions on Automatic Control, 50 (2005), 811-826.
doi: 10.1109/TAC.2005.849212.
|
| [15] |
S. Henrot, C. Soussen and D. Brie, Fast positive deconvolution of hyperspectral images, IEEE Transactions on Image Processing, 22 (2012), 828-833.
doi: 10.1109/TIP.2012.2216280.
|
| [16] |
S. G. Krantz and H. R. Parks, A Primer of Real Analytic Functions, Springer Science & Business Media, 2002.
doi: 10.1007/978-0-8176-8134-0.
|
| [17] |
D. D. Lee and H. S. Seung, Algorithms for non-negative matrix factorization, Advances in Neural Information Processing Systems, (2001), 556-562.
|
| [18] |
J. Lohéac, E. Trélat and E. Zuazua, Minimal controllability time for the heat equation under unilateral state or control constraints, Mathematical Models and Methods in Applied Sciences, 27 (2017), 1587-1644.
doi: 10.1142/S0218202517500270.
|
| [19] |
B. Mair, M. Rao and J. Anderson, Positron emission tomography, Borel measures and weak convergence, Inverse Problems, 12 (1996), 965-976.
doi: 10.1088/0266-5611/12/6/011.
|
| [20] |
O. Öktem, C. Pouchol and O. Verdier, Spatiotemporal PET reconstruction using ML-EM with learned diffeomorphic deformation, International Workshop on Machine Learning for Medical Image Reconstruction, (2019), 151-162.
|
| [21] |
W. J. Palenstijn, Astra toolbox, Jan. 2012.
|
| [22] |
C. Pouchol and O. Verdier, The ML-EM algorithm in continuum: Sparse measure solutions, Inverse Problems, 36 (2020), 035013, 26 pp.
doi: 10.1088/1361-6420/ab6d55.
|
| [23] |
C. Pouchol and O. Verdier, Statistical model and ML-EM algorithm for emission tomography with known movement, Journal of Mathematical Imaging and Vision, 63 (2021), 650-663.
doi: 10.1007/s10851-021-01021-7.
|
| [24] |
E. Resmerita, H. W. Engl and A. N. Iusem, The expectation-maximization algorithm for ill-posed integral equations: A convergence analysis, Inverse Problems, 23 (2007), 2575-2588.
doi: 10.1088/0266-5611/23/6/019.
|
| [25] |
R. T. Rockafellar, Extension of Fenchel's duality theorem for convex functions, Duke Mathematical Journal, 33 (1966), 81-89.
|
| [26] |
L. A. Shepp and Y. Vardi, Maximum likelihood reconstruction for emission tomography, IEEE Transactions on Medical Imaging, 1 (1982), 113-122.
|
| [27] |
U. Simsekli, A. T. Cemgil and Y. K. Yilmaz, Learning the beta-divergence in tweedie compound poisson matrix factorization models, International Conference on Machine Learning, (2013), 1409-1417.
|
| [28] |
Y. Vardi, L. Shepp and L. Kaufman, A statistical model for positron emission tomography, Journal of the American statistical Association, 80 (1985), 8-37.
doi: 10.1080/01621459.1985.10477119.
|
| [29] |
Z. Yang and E. Oja, Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization, IEEE Transactions on Neural Networks, 22 (2011), 1878-1891.
|
An example of map
Example of shifts with
Two views of the functions described in subsection 4.2: here the two detectors are the line segment
Example of PET functions for the regular polygonk
Case
Case
The singular solutions of the problem (6) with two detectors, a1 = 1, a2(x) = x on the interval K = [0, 1]. There are three distinct regions outside the cone
The solutions for
Relation between regularisation parameter and the reconstruction of the Shepp--Logan phantom