Estimating and using information in inverse problems

Wolfgang Bangerth; Chris R. Johnson; Dennis K. Njeru; Bart van Bloemen Waanders

doi:10.3934/ipi.2026003

Article Contents

2026, Volume 24: 1-33. Doi: 10.3934/ipi.2026003

This volume Previous Article Determination of DN map from scattering relation for simple surfaces in low regularity Next Article A local Fourier extension method for function approximation

Estimating and using information in inverse problems

1.
Department of Mathematics, Department of Geosciences, Colorado State University, Fort Collins, CO 80523-1874, USA
2.
Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT 84112, USA
3.
Sandia National Laboratories, MS 0370, PO Box 5800, Albuquerque, NM 87185, USA

^*Corresponding author: Wolfgang Bangerth
^*Corresponding author: Wolfgang Bangerth

Received: February 2025

Revised: September 2025

Early access: November 26, 2025

Published online: November 26, 2025

Abstract / Introduction Full Text(HTML) Figure(5) Related Papers Cited by

Abstract

In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts of the function can be inferred accurately and which cannot. For example, it is generally understood that we can identify system parameters accurately only close to detectors, or along ray paths between sources and detectors, because we have "the most information" for these places.

Although referenced in many publications, the "information" that is invoked in such contexts is not a well understood and clearly defined quantity. Herein, we present a definition of information density that is based on the variance of coefficients as derived from a Bayesian reformulation of the inverse problem. We then discuss three areas in which this information density can be useful in practical algorithms for the solution of inverse problems, and illustrate the usefulness in one of these areas – how to choose the discretization mesh for the function to be reconstructed – using numerical experiments.

Keywords:

Information,
inverse problems.

Mathematics Subject Classification: Primary: 65N21, 35R30, 94A17.

Citation:

Full Text(HTML)

Figure 1. Left: The solution $ u^\ast({\mathbf x}) $ of the forward problem from which we generate "synthetic" measurements $ z_\ell $ via (32). Right: The source term $ q^\ast({\mathbf x}) $ from which we compute synthetic measurements is constant and nonzero only in the solid red circle offset from the center; the detector locations $ \xi_\ell, \ell = 1, \ldots, L = 100 $ are marked by dots.

Download: Full-size image PowerPoint slide

Figure 2. The solution of problem (33), computed on a very fine finite element mesh. Top left: The primal variable $ u({\mathbf x}) $, shown with the same scale for color and isocontours as in Fig. 1. Top right: The recovered sources $ q({\mathbf x}) $, i.e., the solution of the inverse problem. The red circle indicates the location of the source term used in generating the synthetic data. Bottom left: The adjoint variable $ \lambda({\mathbf x}) $. Bottom right: The information density $ j({\mathbf x}) $ associated with this problem, as defined in (29).

Download: Full-size image PowerPoint slide

Figure 3. Reconstructions (top) on a sequence of meshes (bottom) refined based on the information content of each cell of the mesh.

Download: Full-size image PowerPoint slide

Figure 4. Sequences of meshes generated by different mesh refinement criteria. Top: Mesh refinement is driven by an a posteriori error indicator. Bottom: Mesh refinement is driven by a a smoothness indicator.

Download: Full-size image PowerPoint slide

Figure 5. Comparison of the eigenvalues of the resolution matrix $ H $ for different ways of refining meshes. Left: After three refinement cycles, yielding problems with approximately 1,000 parameters to identify for the adaptive refinement criteria, and 16,384 for global refinement. For the global refinement case, only the first 1,100 eigenvalues are shown; the remaining ones have nearly the same value. Right: After six refinement cycles, resulting in problems with approximately 4,000 parameters for the adaptive refinement criteria. (Global refinement would have resulted in more than one million parameters; the eigenvalues of this matrix could not be computed.).

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	V. Akçelik, G. Biros, O. Ghattas, J. Hill, D. Keyes and B. van Bloemen Waanders, Parallel algorithms for PDE-constrained optimization, chapter 16, SIAM, 2006, 291-322. doi: 10.1137/1.9780898718133.ch16.
[2]	A. Alexanderian, N. Petra, G. Stadler and O. Ghattas, A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized $\ell_0$-sparsification, SIAM Journal on Scientific Computing, 36 (2014), A2122-A2148. doi: 10.1137/130933381.
[3]	I. Ambartsumyan, W. Boukaram, T. Bui-Thanh, O. Ghattas, D. Keyes, G. Stadler, G. Turkiyyah and S. Zampini, Hierarchical matrix approximations of Hessians arising in inverse problems governed by PDEs, SIAM J. Sci. Comput., 42 (2020), A3397-A3426. doi: 10.1137/19M1270367.
[4]	D. Arndt, W. Bangerth, B. Blais, M. Fehling, R. Gassmöller, T. Heister, L. Heltai, U. Köcher, M. Kronbichler, M. Maier, P. Munch, J.-P. Pelteret, S. Proell, K. Simon, B. Turcksin, D. Wells and J. Zhang, The deal.Ⅱ library, version 9.3, Journal of Numerical Mathematics, 29 (2021), 171-186. doi: 10.1515/jnma-2021-0081.
[5]	D. Arndt, W. Bangerth, D. Davydov, T. Heister, L. Heltai, M. Kronbichler, M. Maier, J.-P. Pelteret, B. Turcksin and D. Wells, The deal.Ⅱ finite element library: Design, features, and insights, Computers & Mathematics with Applications, 81 (2021), 407-422, https://arXiv.org/abs/1910.13247. doi: 10.1016/j.camwa.2020.02.022.
[6]	S. R. Arridge and M. Schweiger, Inverse methods for optical tomography, in Information Processing in Medical Imaging (eds. H. H. Barrett and A. F. Gmitro), Springer Berlin Heidelberg, Berlin, Heidelberg, 1993, 259-277. doi: 10.1007/BFb0013793.
[7]	A. C. Atkinson and A. N. Donev, Optimum Experimental Design, Clarendon Press, Oxford, 1992.
[8]	L. Auer, L. Boschi, T. W. Becker, T. Nissen‐Meyer and D. Giardini, Savani: A variable resolution whole‐mantle model of anisotropic shear velocity variations based on multiple data sets, J. Geophys. Res. Solid Earth, 119 (2014), 3006-3034. doi: 10.1002/2013JB010773.
[9]	W. Bangerth, Adaptive Finite Element Methods for the Identification of Distributed Parameters in Partial Differential Equations, PhD thesis, University of Heidelberg, 2002.
[10]	W. Bangerth, A framework for the adaptive finite element solution of large inverse problems, SIAM J. Sc. Comput., 30 (2008), 2965-2989. doi: 10.1137/070690560.
[11]	W. Bangerth and A. Joshi, Adaptive finite element methods for the solution of inverse problems in optical tomography, Inverse Problems, 24 (2008), 034011, 22 pp. doi: 10.1088/0266-5611/24/3/034011.
[12]	W. Bangerth and R. Rannacher, Adaptive Finite Element Methods for Differential Equations, Birkhäuser Verlag, 2003. doi: 10.1007/978-3-0348-7605-6.
[13]	H. T. Banks, K. Holm and F. Kappel, Comparison of optimal design methods in inverse problems, Inverse Problems, 27 (2011), 075002, 31 pp. doi: 10.1088/0266-5611/27/7/075002.
[14]	A. J. Berglund, Statistics of camera-based single-particle tracking, Physical Review E, 82 (2010). doi: 10.1103/PhysRevE.82.011917.
[15]	J. G. Berryman, Analysis of approximate inverses in tomography in tomography Ⅰ. Resolution analysis of common inverses, Optimization and Engineering, 1 (2000), 87-115. doi: 10.1023/A:1010098523281.
[16]	D. Bigoni, Y. Chen, N. Garcia Trillos, Y. Marzouk and D. Sanz-Alonso, Data-driven forward discretizations for Bayesian inversion, Inverse Problems, 36 (2020), 105008, 27 pp. doi: 10.1088/1361-6420/abb2fa.
[17]	R. Bonadio, S. Lebedev, T. Meier, P. Arroucau, A. J. Schaeffer, A. Licciardi, M. R. Agius, C. Horan, L. Collins, B. M. O'Reilly, P. W. Readman and I reland Array Working Group, Optimal resolution tomography with error tracking and the structure of the crust and upper mantle beneath Ireland and Britain, Geophysical Journal International, 226 (2021), 2158-2188. doi: 10.1093/gji/ggab169.
[18]	L. Borcea, V. Druskin and A. V. Mamonov, Circular resistor networks for electrical impedance tomography with partial boundary measurements, Inverse Problems, 26 (2010), 045010, 30 pp. doi: 10.1088/0266-5611/26/4/045010.
[19]	E. Bozdağ, D. Peter, M. Lefebvre, D. Komatitsch, J. Tromp, J. Hill, N. Podhorszki and D. Pugmire, Global adjoint tomography: First-generation model, Geophysical Journal International, 207 (2016), 1739-1766. doi: 10.1093/gji/ggw356.
[20]	D. Braess, Finite Elements, Cambridge University Press, 1997. doi: 10.1007/978-3-662-07233-2.
[21]	S. C. Brenner and R. L. Scott, The Mathematical Theory of Finite Elements, 2nd edition, Springer, Berlin-Heidelberg-New York, 2002.
[22]	T. Bui-Thanh, C. Burstedde, O. Ghattas, J. Martin, G. Stadler and L. C. Wilcox, Extreme-scale UQ for Bayesian inverse problems governed by PDEs, in Proceedings of the Supercomputing 2012 conference, ACM/IEEE SC Conference Series, 2012. doi: 10.1109/SC.2012.56.
[23]	T. Bui-Thanh, O. Ghattas, J. Martin and G. Stadler, A computational framework for infinite-dimensional bayesian inverse problems part i: The linearized case, with application to global seismic inversion, SIAM J. Sci. Comput., 35 (2013), A2494-A2523. doi: 10.1137/12089586X.
[24]	C. Cohen-Bacrie, Y. Goussard and R. Guardo, Regularized reconstruction in electrical impedance tomography using a variance uniformization constraint, IEEE Trans. Med. Imag., 16 (1997), 562-571. doi: 10.1109/42.640745.
[25]	T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, 2006.
[26]	D. R. Cox and D. V. Hinkley, Theoretical Statistics, Chapman, 1974.
[27]	F. A. Dahlen, S.-H. Hung and G. Nolet, Fréchet kernels for finite-frequency traveltimes—Ⅰ. Theory, Geophys. J. Int., 141 (2000), 157-174. doi: 10.1046/j.1365-246X.2000.00070.x.
[28]	S. Das, Efficient Calculation of Fisher Information Matrix: Monte Carlo Approach Using Prior Information, PhD thesis, Johns Hopkins University, 2007.
[29]	S. Das, J. C. Spall and R. Ghanem, Efficient monte carlo computation of fisher information matrix using prior information, Computational Statistics & Data Analysis, 54 (2010), 272-289. doi: 10.1016/j.csda.2009.09.018.
[30]	A. Datta-Gupta, J. Xie, N. Gupta, M. J. King and W. J. Lee, Radius of investigation and its generalization to unconventional reservoirs, J. Petroleum Techn., 63 (2011), 52-55. doi: 10.2118/0711-0052-JPT.
[31]	M. H. DeGroot, Optimal Statistical Decisions, Wiley-Interscience, 1970.
[32]	A. F. Emery and A. V. Nenarokomov, Optimal experiment design, Measurement Science and Technology, 9 (1998), 864-876. doi: 10.1088/0957-0233/9/6/003.
[33]	H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers, Dordrecht, 1996.
[34]	Y. Favennec, Hessian and Fisher matrices for error analysis in inverse heat conduction problems, Numerical Heat Transfer, Part B: Fundamentals, 52 (2007), 323-340. doi: 10.1080/10407790701443958.
[35]	A. Griesbaum, B. Kaltenbacher and B. Vexler, Efficient computation of the Tikhonov regularization parameter by goal-oriented adaptive discretization, Inverse Problems, 24 (2008), 025025, 20 pp. doi: 10.1088/0266-5611/24/2/025025.
[36]	S.-H. Hung, F. A. Dahlen and G. Nolet, Fréchet kernels for finite-frequency traveltimes—Ⅱ. Examples, Geophys. J. Int., 141 (2000), 175-203. doi: 10.1046/j.1365-246X.2000.00072.x.
[37]	A. Jamshidi, J. M. V. Samani, H. M. V. Samani, A. Zanini, M. G. Tanda and M. Mazaheri, Solving inverse problems of unknown contaminant source in groundwater-river integrated systems using a surrogate transport model based optimization, Water, 12 (2020), 2415. doi: 10.3390/w12092415.
[38]	J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Springer, 2006.
[39]	S. M. Kay, Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory, Prentice Hall, 1993.
[40]	F. J. Kuchuk, Radius of investigation for reserve estimation from pressure transient well tests, SPE, 120515, 1-22.
[41]	W. Lei, Y. Ruan, E. Bozdağ, D. Peter, M. Lefebvre, D. Komatitsch, J. Tromp, J. Hill, N. Podhorszki and D. Pugmire, Global adjoint tomography—model GLAD-M25, Geophysical Journal International, 223 (2020), 1-21. doi: 10.1093/gji/ggaa253.
[42]	Q. Liu and Y. J. Gu, Seismic imaging: From classical to adjoint tomography, Tectonophysics, 566-567 (2012), 31-66. doi: 10.1016/j.tecto.2012.07.006.
[43]	R. Montelli, G. Nolet, F. A. Dahlen and G. Masters, A catalogue of deep mantle plumes: New results from finite-frequency tomography, Geochemistry, Geophysics, Geosystems, 7 (2006). doi: 10.1029/2006gc001248.
[44]	R. M. Neupauer, B. Borchers and J. L. Wilson, Comparison of inverse methods for reconstructing the release history of a groundwater contamination source, Water Resources Research, 36 (2000), 2469-2475. doi: 10.1029/2000WR900176.
[45]	J. Nocedal and S. J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer, New York, 1999. doi: 10.1007/b98874.
[46]	S. Nordebo and M. Gustavsson, A priori modeling for gradient based inverse scattering algorithms, Progress in Electromagnetics Research B, 16 (2009), 407-432. doi: 10.2528/PIERB09060805.
[47]	N. Petra, J. Martin, G. Stadler and O. Ghattas, A computational framework for infinite-dimensional bayesian inverse problems, part ⅱ: Stochastic newton mcmc with application to ice sheet flow inverse problems, SIAM J. Sci. Comput., 36 (2014), A1525-A155. doi: 10.1137/130934805.
[48]	B. W. Pogue, T. O. McBride, J. Prewitt, U. L. Österberg and K. D. Paulsen, Spatially variant regularization improves optical tomography, Appl. Optics, 38 (1999), 2950-2961. doi: 10.1364/AO.38.002950.
[49]	N. Rawlinson, A. Fichtner, M. Sambridge and M. K. Young, Seismic tomography and the assessment of uncertainty, in Advances in Geophysics, Elsevier, 55 (2014), 1-76. doi: 10.1016/bs.agph.2014.08.001.
[50]	J. Ritsema, A. Deuss, H. J. van Heijst and J. H. Woodhouse, S40RTS: A degree-40 shear-velocity model for the mantle from new rayleigh wave dispersion, teleseismic traveltime and normal-mode splitting function measurements, Geophysical Journal International, 184 (2010), 1223-1236. doi: 10.1111/j.1365-246X.2010.04884.x.
[51]	E. Romera, P. Sánchez-Moreno and J. S. Dehesa, The Fisher information of single-particle systems with a central potential, Chemical Physics Letters, 414 (2005), 468-472. doi: 10.1016/j.cplett.2005.08.032.
[52]	M. Sambridge and N. Rawlinson, Seismic tomography with irregular meshes, in Seismic Earth: Array Analysis of Broadband Seismograms (eds. A. Levander and G. Nolet), American Geophysical Union, 157 (2005), 49-65. doi: 10.1029/157GM04.
[53]	T. H. Skaggs and Z. J. Kabala, Recovering the release history of a groundwater contaminant, Water Resources Research, 30 (1994), 71-79. doi: 10.1029/93WR02656.
[54]	A. Spantini, A. Solonen, T. Cui, J. Martin, L. Tenorio and Y. Marzouk, Optimal low-rank approximations of Bayesian linear inverse problems, SIAM J. Sci. Comput., 37 (2015), A2451-A2487. doi: 10.1137/140977308.
[55]	A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numerica, 19 (2010), 451-559. doi: 10.1017/S0962492910000061.
[56]	L. Tang, G. Hamarneh and R. Abugharbieh, Reliability-driven, spatially-adaptive regularization for deformable registration, in Biomedical Image Registration, 4th International Workshop, WBIR 2010, Lübeck, Germany, July 11-13, 2010. Proceedings (eds. B. Fischer, B. M. Dawant and C. Lorenz), vol. 6204 of Lecture Notes in Computer Science, Springer, 2010, 173-185. doi: 10.1007/978-3-642-14366-3_16.
[57]	A. Tarantola, Inverse Problem Theory, Elsevier, Amsterdam, New York, 1987.
[58]	A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation, SIAM, 2004.
[59]	D. W. Vasco, H. Keers and K. Karasaki, Estimation of reservoir properties using transient pressure data: An asymptotic approach, Water Resources Res., 36 (2000), 3447-3465. doi: 10.1029/2000WR900179.
[60]	R. Verfürth, A Review of A Posteriori Error Estimation and Adaptive Mesh Refinement Techniques, Wiley/Teubner, New York, Stuttgart, 1996.
[61]	J. A. Vrugt, P. H. Stauffer, T. Wöhling, B. A. Robinson and V. V. Vesselinov, Inverse modeling of subsurface flow and transport properties: A review with new developments, Vadose Zone Journal, 7 (2008), 843-864. doi: 10.2136/vzj2007.0078.
[62]	M. W. Woolrich, T. E. J. Behrens, C. F. Beckmann and S. M. Smith, Mixture models with adaptive spatial regularization for segmentation with an application to MRI data, IEEE Trans. Med. Imag., 24 (2005), 1-11. doi: 10.1109/TMI.2004.836545.
[63]	Y. Yin, E. A. Hoffman and C.-L. Lin, Lung lobar slippage assessed with the aid of image registration, Med. Image. Comput. Comput. Assist. Interv., 13 (2010), 578-585. doi: 10.1007/978-3-642-15745-5_71.
[64]	H. Zhang and C. H. Thurber, Estimating the model resolution matrix for large seismic tomography problems based on lanczos bidiagonalization with partial reorthogonalization, Geophysical Journal International, 170 (2007), 337-345. doi: 10.1111/j.1365-246X.2007.03418.x.
[65]	L. Zhao, T. H. Jordan, K. B. Olsen and P. Chen, Fréchet kernels for imaging regional earth structure based on three-dimensional reference models, Bulletin of the Seismological Society of America, 95 (2005), 2066-2080. doi: 10.1785/0120050081.