Hyperpriors for Matérn fields with applications in Bayesian inversion

Lassi Roininen; Mark Girolami; Sari Lasanen; Markku Markkanen

doi:10.3934/ipi.2019001

Article Contents

2019, Volume 13, Issue 1: 1-29. Doi: 10.3934/ipi.2019001

This issue Previous Article Next Article Inverse problems for the heat equation with memory

Hyperpriors for Matérn fields with applications in Bayesian inversion

1.
LUT University, School of Engineering Science, P.O.Box 20, FI-53851 Lappeenranta, Finland
2.
Department of Mathematics, Imperial College London, and Alan Turing Institute, London, United Kingdom
3.
LUT University, School of Engineering Science, Finland
4.
Eigenor Corporation, Lompolontie 1, FI-99600 Sodankylä, Finland

Received: December 2016

Revised: April 2017

Published: December 2018

Abstract / Introduction Full Text(HTML) Figure(9) Related Papers Cited by

Abstract

We introduce non-stationary Matérn field priors with stochastic partial differential equations, and construct correlation length-scaling with hyperpriors. We model both the hyperprior and the Matérn prior as continuous-parameter random fields. As hypermodels, we use Cauchy and Gaussian random fields, which we map suitably to a desired correlation length-scaling range. For computations, we discretise the models with finite difference methods. We consider the convergence of the discretised prior and posterior to the discretisation limit. We apply the developed methodology to certain interpolation, numerical differentiation and deconvolution problems, and show numerically that we can make Bayesian inversion which promotes competing constraints of smoothness and edge-preservation. For computing the conditional mean estimator of the posterior distribution, we use a combination of Gibbs and Metropolis-within-Gibbs sampling algorithms.

Keywords:

Mathematics Subject Classification: Primary: 15A29, 60G15; Secondary: 62M05.

Citation:

Full Text(HTML)

Figure 1. Examples of constructing non-stationary Matérn realisations with hypermodels. Top panel - from left to right: Realisation $\ell^N_\omega$ given Cauchy walk as $u^N_\omega$, resulting covariance matrix, and four realisations. Bottom panel: Same as above, but with a Gaussian process hyperprior

Download: Full-size image PowerPoint slide

Figure 2. Non-stationary structures obtained by starting from a constant-parameter or inhomogeneous Matérn field realisation (upper panel), after which have been mapped to correlation length-scaling fields (middle). In the bottom panel, we have corresponding realisations with isotropic and anisotropic structures. This kind of structure can detect regions within which the behaviour of the random field is smooth, but the regions are distinct

Download: Full-size image PowerPoint slide

Figure 3. Top panel: 81 noisy measurements and estimated $\ell^N$ ($N = 161$) with Cauchy noise (B) and Gaussian hyperprior (C). (D, G, J) are conditional mean estimates of $v^N$ ($N = 161$) with long length-scaling (D), $\ell^N$ minimising MAE (G), $\ell^N$ minimising RMSE (J). (E, H, K) and (F, I, L) are CM-estimates of $v^N$ on different meshes with Cauchy hypermodel and Gaussian hypermodels, respectively

Download: Full-size image PowerPoint slide

Figure 4. Estimates of $\ell^N$ and $v^N$ with a Cauchy walk hypermodel $u^N$ on different lattices with $81$ measurements, with the number of unknowns varying as in figures. Bottom four subfigures (G-J) are chains and cumulative means of certain $\ell^N$ and $v^N$ elements

Download: Full-size image PowerPoint slide

Figure 5. Estimates of $\ell^N$ and $v^N$ with a Gaussian hyperprior $u^N$ on different lattices with $81$ measurements, with the number of unknowns varying as in figures. Bottom four subfigures (G-J) are chains and cumulative means of certain $\ell^N$ and $v^N$ elements

Download: Full-size image PowerPoint slide

Figure 6. Numerical differentiation of a noisy signal with the developed Gaussian hypermodel. We plot $v^N$ on different meshes for seeing the discretisation-invariance of the estimates

Download: Full-size image PowerPoint slide

Figure 7. Two-dimensional interpolation of block-shaped and Gaussian-shaped structures from noisy observations. A Matérn hyperprior is used in the analysis

Download: Full-size image PowerPoint slide

Figure 8. Two-dimensional interpolations with sparse data of block-shaped and Gaussian-shaped structures from noisy observations. A Matérn hyperprior is used in the analysis

Download: Full-size image PowerPoint slide

Figure 9. Two-dimensional deconvolution of block-shaped and Gaussian-shaped structures from noisy observations. A Matérn hyperprior is used in the analysis

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	J. M. Bardsley, Gaussian Markov random field priors for inverse problems, Inverse Problems and Imaging, 7 (2013), 397-416. doi: 10.3934/ipi.2013.7.397.
[2]	V. I. Bogachev, Gaussian Measures, American Mathematical Society, Providence RI, 1998. doi: 10.1090/surv/062.
[3]	V. I. Bogachev, Measure Theory, Springer-Verlag, Berlin, 2007. doi: 10.1007/978-3-540-34514-5.
[4]	D. Bolin, Spatial matérn fields driven by non-gaussian noise, Scandinavian Journal of Statistics, 41 (2014), 557-579. doi: 10.1111/sjos.12046.
[5]	R. Buckdahn and E. Pardoux, Monotonicity methods for white noise driven quasi-linear SPDEs, Diffusion Processes and Related Problems in Analysis, Vol. I (Evanston, IL, 1989), Birkhäuser Boston, Boston, MA, 22 (1990), 219–233. doi: 10.1007/978-1-4684-0564-4_13.
[6]	D. Calvetti and E. Somersalo, Introduction to Bayesian Scientific Computing – Ten Lectures on Subjective Computing, Springer, New York, 2007.
[7]	D. Calvetti and E. Somersalo, A Gaussian hypermodel to recover blocky objects, Inverse Problems, 23 (2007), 733-754. doi: 10.1088/0266-5611/23/2/016.
[8]	D. Calvetti and E. Somersalo, Hypermodels in the Bayesian imaging framework, Inverse Problems, 24 (2008), 034013, 20pp. doi: 10.1088/0266-5611/24/3/034013.
[9]	N. K. Chada, M. A. Iglesias, K. Roininen and A. M. Stuart, Parameterizations for Ensemble Kalman Inversion, arXiv: 1709.01781, (2016).
[10]	J. Charrier, Strong and weak error estimates for elliptic partial differential equations with random coefficients, SIAM Journal on Numerical Analysis, 50 (2012), 216-246. doi: 10.1137/100800531.
[11]	S. L. Cotter, G. O. Roberts, A. M. Stuart and D. White, MCMC Methods for Functions: Modifying old algorithms to make them faster, Statistical Science, 28 (2013), 424-446. doi: 10.1214/13-STS421.
[12]	T. A. Davis, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization, ACM Trans. Math. Software, 38 (2011), Art. 8, 22 pp. doi: 10.1145/2049662.2049670.
[13]	M. Dunlop, Analysis and Computation for Bayesian Inverse Problems, PhD thesis, University of Warwick, 2016.
[14]	M. M. Dunlop, M Girolami, A. M. Stuart and A. L. Teckentrup, How Deep Are Deep Gaussian Processes?, arXiv: 1711.11280 (2017).
[15]	M. Filippone and M. Girolami, Pseudo-Marginal Bayesian Inference for Gaussian Processes, IEEE Transactions Pattern Analysis and Machine Intelligence, 36 (2014), 2214-2226.
[16]	L. V. Foster and T. A. Davis, Reliable calculation of numerical rank, null space bases, pseudoinverse solutions and basic solutions using SuiteSparseQR, ACM Trans. Math. Software, 40 (2013), Art. 7, 23 pp. doi: 10.1145/2513109.2513116.
[17]	G.-A. Fuglstad, D. Simpson, F. Lindgren and H. Rue, Does non-stationary spatial data always require non-stationary random fields?, Spatial Statistics, 14 (2015), 505-531. doi: 10.1016/j.spasta.2015.10.001.
[18]	T. Helin and M. Lassas, Hierarchical models in statistical inverse problems and the Mumford-Shah functional, Inverse Problems, 27 (2011), 015008, 32pp. doi: 10.1088/0266-5611/27/1/015008.
[19]	B. S. Jovanović and E. Süli, Analysis of Finite Difference Schemes, Springer, London, 2014. doi: 10.1007/978-1-4471-5460-0.
[20]	J. P. Kahane, Some Random Series of Functions, Cambridge University Press, Cambridge, 1985.
[21]	J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Springer-Verlag, New York, 2005.
[22]	S. Lasanen, Non-Gaussian statistical inverse problems. Part Ⅰ: Posterior distributions, Inverse Probl. Imaging, 6 (2012), 215-266. doi: 10.3934/ipi.2012.6.215.
[23]	M. Lassas, E. Saksman and S. Siltanen, Discretization invariant Bayesian inversion and Besov space priors, Inverse Problems and Imaging, 3 (2009), 87-122. doi: 10.3934/ipi.2009.3.87.
[24]	M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?, Inverse Problems, 20 (2004), 1537-1563. doi: 10.1088/0266-5611/20/5/013.
[25]	F. Lindgren, H. Rue and J. Lindström, An explicit link between Gaussian Markov random fields: The stochastic partial differential equation approach, Journal of the Royal Statistical Society: Series B, 73 (2011), 423-498. doi: 10.1111/j.1467-9868.2011.00777.x.
[26]	M. Markkanen, L. Roininen, J. M. J. Huttunen and S. Lasanen, Cauchy difference priors for edge-preserving Bayesian inversion with an application to x-ray tomography, arXiv: 1603.06135, (2016).
[27]	D. Mitrea, M. Mitrea and L. Yan, Boundary value problems for the Laplacian in convex and semiconvex domains, J. Funct. Anal., 258 (2010), 2507-2585. doi: 10.1016/j.jfa.2010.01.012.
[28]	J. Norberg, L. Roininen, J. Vierinen, O. Amm, D. McKay-Bukowski and M.S Lehtinen, Ionospheric tomography in Bayesian framework with Gaussian Markov random field priors, Radio Science, 50 (2015), 138-152.
[29]	C. J. Paciorek, Nonstationary Gaussian Processes for Regression and Spatial Modelling, PhD thesis, Carnegie Mellon University, 2003.
[30]	C. J. Paciorek and M. J. Schervish, Spatial modelling using a new class of nonstationary covariance functions, Environmetrics, 17 (2006), 483-506. doi: 10.1002/env.785.
[31]	O. Papaspiliopoulos, G. O. Roberts and M. Sköld, A general framework for the parametrization of hierarchical models, Statistical Science, 22 (2007), 59-73. doi: 10.1214/088342307000000014.
[32]	L. Roininen, J. Huttunen and S. Lasanen, Whittle-Matérn priors for Bayesian statistical inversion with applications in electrical impedance tomography, Inverse Problems and Imaging, 8 (2014), 561-586. doi: 10.3934/ipi.2014.8.561.
[33]	L. Roininen, P. Piiroinen and M. Lehtinen, Constructing continuous stationary covariances as limits of the second-order stochastic difference equations, Inverse Problems and Imaging, 7 (2013), 611-647. doi: 10.3934/ipi.2013.7.611.
[34]	J. A. Rozanov, Markovian random fields, and stochastic partial differential equations, Mat. Sb. (N.S.), 103 (1977), 590-613.
[35]	H. Rue and L. Held, Gaussian Markov Random Fields: Theory and Applications, Chapman & Hall/CRC, Boca Raton, FL, 2005. doi: 10.1201/9780203492024.
[36]	H. Rue, S. Martino and N. Chopin, Approximate Bayesian inference for latent Gaussian Models by using integrated nested Laplace approximations, Journal of the Royal Statistical Society: Series b (Statistical Methodology), 71 (2009), 319-392. doi: 10.1111/j.1467-9868.2008.00700.x.
[37]	A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numerica, 19 (2010), 451-559. doi: 10.1017/S0962492910000061.
[38]	T. J. Sullivan, Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors, Inverse Problems and Imaging, 11 (2017), 857-874. doi: 10.3934/ipi.2017040.
[39]	J. Vanhatalo and A. Vehtari, Sparse log gaussian processes via MCMC for spatial epidemiology, JMLR Workshop and Conference Proceedings, 1 (2007), 73-89.
[40]	J. Wallin and D. Bolin, Geostatistical modelling using non-Gaussian Matérn fields, Scand. J. Statist., 42 (2015), 872-890. doi: 10.1111/sjos.12141.
[41]	Y. R. Yue, D. Simpson, F. Lindgren and H. Rue, Bayesian adaptive smoothing splines using stochastic differential equations, Bayesian Analysis, 9 (2014), 397-423. doi: 10.1214/13-BA866.