We propose alternatives to Bayesian prior distributions that are frequently used in the study of inverse problems. Our aim is to construct priors that have similar good edge-preserving properties as total variation or Mumford-Shah priors but correspond to well-defined infinite-dimensional random variables, and can be approximated by finite-dimensional random variables. We introduce a new wavelet-based model, where the non-zero coefficients are chosen in a systematic way so that prior draws have certain fractal behaviour. We show that realisations of this new prior take values in Besov spaces and have singularities only on a small set $ \tau $ with a certain Hausdorff dimension. We also introduce an efficient algorithm for calculating the MAP estimator, arising from the the new prior, in the denoising problem.
Citation: |
Table Algorithm 1.
Pseudocode for finding the minimising
[1] | F. Abramovich, T. Sapatinas and B. W. Silverman, Wavelet thresholding via a Bayesian approach, J. R. Stat. Soc. Ser. B Stat. Methodol., 60 (1998), 725-749. doi: 10.1111/1467-9868.00151. |
[2] | S. Agapiou, M. Dashti and T. Helin, Rates of contraction of posterior distributions based on p-exponential priors, Bernoulli, 27 (2021), 1616-1642. doi: 10.3150/20-bej1285. |
[3] | A. Alman, L. Johnson, D. Calverley, G. Grunwald, D. Lezotte and J. Hokanson, Diagnostic capabilities of fractal dimension and mandibular cortical width to identify men and women with decreased bone mineral density, Osteoporosis International, 23 (2012), 1631-1636. |
[4] | K. B. Athreya and P. E. Ney, Branching Processes, Dover Publications, Inc., Mineola, NY, 2004. |
[5] | R. M. Brown, Global uniqueness in the impedance imaging problem for less regular conductivities, SIAM Journal on Mathematical Analysis, 27 (1996), 1049-1056. doi: 10.1137/S0036141094271132. |
[6] | C. B. Caldwell, S. J. Stapleton, D. W. Holdsworth, R. A. Jong, W. J. Weiser, G. Cooke and M. J. Yaffe, Characterisation of mammographic parenchymal pattern by fractal dimension, Physics in Medicine & Biology, 35 (1990), 235. |
[7] | D. Calvetti and E. Somersalo, A Gaussian hypermodel to recover blocky objects, Inverse Problems, 23 (2007), 733-754. doi: 10.1088/0266-5611/23/2/016. |
[8] | D. Calvetti and E. Somersalo, Hypermodels in the Bayesian imaging framework, Inverse Problems, 24 (2008), 034013, 20 pp. doi: 10.1088/0266-5611/24/3/034013. |
[9] | I. Castillo, Pólya tree posterior distributions on densities, Ann. Inst. Henri Poincaré Probab. Stat., 53 (2017), 2074-2102. doi: 10.1214/16-AIHP784. |
[10] | I. Castillo and R. Mismer, Spike and slab pólya tree posterior densities: Adaptive inference, Ann. Inst. Henri Poincaré Probab. Stat., 57 (2021), 1521-1548. doi: 10.1214/20-aihp1132. |
[11] | I. Castillo and T. Randrianarisoa, Optional polya trees: Posterior rates and uncertainty quantification, (2021), arXiv preprint, arXiv: 2110.05265. |
[12] | I. Castillo and V. Rockova, Multiscale Analysis of Bayesian CART, University of Chicago, Becker Friedman Institute for Economics Working Paper, 2019. |
[13] | A. Chambolle, R. A. DeVore, N. yong Lee and B. J. Lucier, Nonlinear wavelet image processing: Variational problems, compression, and noise removal through wavelet shrinkage, IEEE Transactions on Image Processing, 7 (1998), 319-335. doi: 10.1109/83.661182. |
[14] | A. Chan and J. A. Tuszynski, Automatic prediction of tumour malignancy in breast cancer with fractal dimension, Royal Society Open Science, 3 (2016), 160558. |
[15] | S. G. Chang, B. Yu and M. Vetterli, Adaptive wavelet thresholding for image denoising and compression, IEEE Trans. Image Process., 9 (2000), 1532-1546. doi: 10.1109/83.862633. |
[16] | M. Dashti, S. Harris and A. Stuart, Besov priors for Bayesian inverse problems, Inverse Probl. Imaging, 6 (2012), 183-200. doi: 10.3934/ipi.2012.6.183. |
[17] | M. Dashti, K. J. Law, A. M. Stuart and J. Voss, MAP estimators and their consistency in Bayesian nonparametric inverse problems, Inverse Problems, 29 (2013), 095017, 27 pp. doi: 10.1088/0266-5611/29/9/095017. |
[18] | M. Dashti and A. M. Stuart, The Bayesian approach to inverse problems, Handbook of Uncertainty Quantification, Springer, Cham, 1, 2, 3 (2017), 311-428. |
[19] | I. Daubechies, Ten Lectures on Wavelets (Ninth printing, 2006), BMS-NSF Regional Conference Series in Applied Mathematics, 61. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992. doi: 10.1137/1.9781611970104. |
[20] | I. Daubechies, M. Defrise and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math., 57 (2004), 1413-1457. doi: 10.1002/cpa.20042. |
[21] | M. Davis and H. Li, Evaluation of fractal dimension for mixing and combustion by the schlieren method, Experiments in Fluids, 21 (1996), 248-258. |
[22] | D. L. Donoho and I. M. Johnstone, Ideal spatial adaptation via wavelet shrinkage, Biometrika, 81 (1994), 425-455. doi: 10.1093/biomet/81.3.425. |
[23] | A. V. Dryakhlov and A. A. Tempelman, On Hausdorff dimension of random fractals, New York J. Math., 7 (2001), 99-115. |
[24] | M. Giordano and K. Ray, Nonparametric {B}ayesian inference for reversible multidimensional diffusions, Ann. Statist., 50 (2022), 2872-2898. doi: 10.1214/22-aos2213. |
[25] | C. Gómez, Á. Mediavilla, R. Hornero, D. Abásolo and A. Fernández, Use of the Higuchi's fractal dimension for the analysis of MEG recordings from Alzheimer's disease patients, Medical Engineering & Physics, 31 (2009), 306-313. |
[26] | H. Haario, M. Laine, M. Lehtinen, E. Saksman and J. Tamminen, Markov chain monte carlo methods for high dimensional inversion in remote sensing, J. R. Stat. Soc. Ser. B Stat. Methodol., 66 (2004), 591-607. doi: 10.1111/j.1467-9868.2004.02053.x. |
[27] | T. Helin, On infinite-dimensional hierarchical probability models in statistical inverse problems, Inverse Probl. Imaging, 3 (2009), 567-597. doi: 10.3934/ipi.2009.3.567. |
[28] | T. Helin and M. Lassas, Hierarchical models in statistical inverse problems and the mumford–shah functional, Inverse Problems, 27 (2011), 015008, 32 pp. doi: 10.1088/0266-5611/27/1/015008. |
[29] | M. Jansen, Noise Reduction by Wavelet Thresholding, Lecture Notes in Statistics, 161. Springer-Verlag, New York, 2001. doi: 10.1007/978-1-4613-0145-5. |
[30] | S. Karlin and H. E. Taylor, A First Course in Stochastic Processes, Second edition, Academic Press, New York-London, 1975. |
[31] | S. Lasanen, Discretizations of generalized random variables with applications to inverse problems, Ann. Acad. Sci. Fenn. Math. Diss., (2002), 64 pp. |
[32] | S. Lasanen, Measurements and infinite-dimensional statistical inverse theory, PAMM: Proceedings in Applied Mathematics and Mechanics, Wiley Online Library, 7 (2007), 1080101-1080102. |
[33] | M. Lassas, E. Saksman and S. Siltanen, Discretization-invariant Bayesian inversion and Besov space priors, Inverse Problems and Imaging, 3 (2009), 87-122. doi: 10.3934/ipi.2009.3.87. |
[34] | M. Lassas and S. Siltanen, Can one use total variation prior for edge-preserving Bayesian inversion?, Inverse Problems, 20 (2004), 1537-1563. doi: 10.1088/0266-5611/20/5/013. |
[35] | M. Lehtinen, L. Päivärinta and E. Somersalo, Linear inverse problems for generalised random variables, Inverse Problems, 5 (1989), 599-612. doi: 10.1088/0266-5611/5/4/011. |
[36] | A. Lejay and P. Pigato, A threshold model for local volatility: Evidence of leverage and mean reversion effects on historical data, Int. J. Theor. Appl. Finance, 22 (2019), 1950017, 24 pp. doi: 10.1142/S0219024919500171. |
[37] | H. Li, M. L. Giger, O. I. Olopade and L. Lan, Fractal analysis of mammographic parenchymal patterns in breast cancer risk assessment, Academic Radiology, 14 (2007), 513-521. |
[38] | B. B. Mandelbrot, The inescapable need for fractal tools in finance, Annals of Finance, 1 (2005), 193-195. |
[39] | ——, Parallel cartoons of fractal models of finance, Annals of Finance, 1 (2005), 179-192. |
[40] | R. D. Mauldin and S. C. Williams, Random recursive constructions: Asymptotic geometric and topological properties, Trans. Amer. Math. Soc., 295 (1986), 325-346. doi: 10.1090/S0002-9947-1986-0831202-5. |
[41] | Y. Meyer, Wavelets and Operators, Cambridge Studies in Advanced Mathematics, 37. Cambridge University Press, Cambridge, 1992. |
[42] | P. Pigato, Extreme at-the-money skew in a local volatility model, Finance and Stochastics, 23 (2019), 827-859. doi: 10.1007/s00780-019-00406-2. |
[43] | M. Reiß, Asymptotic equivalence for nonparametric regression with multivariate and random design, Annals of Statistics, 36 (2008), 1957-1982. doi: 10.1214/07-AOS525. |
[44] | E. Ribak, C. Schwartz and G. Baum, Fractal wave fronts: Simulation and prediction for adaptive optics, European Southern Observatory Conference and Workshop Proceedings, 48 (1994), 205. |
[45] | L. Sendur and I. W. Selesnick, Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency, IEEE Transactions on Signal Processing, 50 (2002), 2744-2756. |
[46] | A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numerica, 19 (2010), 451-559. doi: 10.1017/S0962492910000061. |
[47] | G. N. Thomas, S.-Y. Ong, Y. C. Tham, W. Hsu, M. L. Lee, Q. P. Lau, W. Tay, J. AlessiCalandro, L. Hodgson, R. Kawasaki, et al., Measurement of macular fractal dimension using a computer-assisted program, Investigative Ophthalmology & Visual Science, 55 (2014), 2237- 2243. |
[48] | H. Triebel, Theory of Function Spaces, Monographs in Mathematics, 78. Birkhäuser Verlag, Basel, 1983. doi: 10.1007/978-3-0346-0416-1. |
[49] | H. Triebel, Theory of Function Spaces. II, Monographs in Mathematics, 84. Birkhäuser Verlag, Basel, 1992. doi: 10.1007/978-3-0346-0419-2. |
[50] | H. Triebel, Function Spaces and Wavelets on Domains, EMS Tracts in Mathematics, 7. European Mathematical Society (EMS), Zürich, 2008. doi: 10.4171/019. |
[51] | H. Triebel, Theory of Function Spaces IV, Monographs in Mathematics, 107. Birkhäuser/Springer, Cham, 2020. doi: 10.1007/978-3-030-35891-4. |
[52] | B. J. West, Fractal Physiology and Chaos in Medicine, Studies of Nonlinear Phenomena in Life Science, 16. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2013. doi: 10.1142/8577. |
(a) Photograph featuring fractal-like structures such as boundaries of clouds. (b) The three-dimensional structure of human lungs follows a self-similar rule leading to a fractal dimension. Also shown is an X-ray tomography slice of lungs. Images courtesy of Wikimedia Commons
On the left: all the nodes where
From left to right; Weierstrass function with Hausdorff dimension 1.5, sixth iteration of the space filling Hilbert curve with Hausdorff dimension 2, and Menger sponge with Hausdorff dimension
Original blocks signal and the observed noisy signal with signal to noise ratio 3
The denoised signal using pruning algorithm on top left and the denoised signal that was attained using hard thresholding on top right. Below them are the wavelet trees corresponding to the estimators
Prior draws from the 1-dimensional semi-Gaussian random tree Besov prior with Haar wavelets. The wavelet densities are
Measured accelerometer data on left and the denoised signal on right
Original sharp image and image with Gaussian noise with variance
From left to right; Denoised images and close-ups attained using pruning, tree enforced soft thresholding and soft thresholding algorithms. From the close-ups we can clearly see that the tree based methods denoise smooth areas better than soft thresholding
Prior draws from the 2-dimensional semi-Gaussian random tree Besov prior with Daubechies 2 wavelets. The wavelet densities are