February  2013, 7(1): 81-105. doi: 10.3934/ipi.2013.7.81

Bayesian inverse problems with Monte Carlo forward models

1. 

Department of Applied Physics and Applied Mathematics, Columbia University, 200 S. W. Mudd Building, MC 4701, 500 W. 120th Street, New York, NY 10027, United States, United States

2. 

Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, United States

Received  December 2011 Revised  December 2012 Published  February 2013

The full application of Bayesian inference to inverse problems requires exploration of a posterior distribution that typically does not possess a standard form. In this context, Markov chain Monte Carlo (MCMC) methods are often used. These methods require many evaluations of a computationally intensive forward model to produce the equivalent of one independent sample from the posterior. We consider applications in which approximate forward models at multiple resolution levels are available, each endowed with a probabilistic error estimate. These situations occur, for example, when the forward model involves Monte Carlo integration. We present a novel MCMC method called $MC^3$ that uses low-resolution forward models to approximate draws from a posterior distribution built with the high-resolution forward model. The acceptance ratio is estimated with some statistical error; then a confidence interval for the true acceptance ratio is found, and acceptance is performed correctly with some confidence. The high-resolution models are rarely run and a significant speed up is achieved.
    Our multiple-resolution forward models themselves are built around a new importance sampling scheme that allows Monte Carlo forward models to be used efficiently in inverse problems. The method is used to solve an inverse transport problem that finds applications in atmospheric remote sensing. We present a path-recycling methodology to efficiently vary parameters in the transport equation. The forward transport equation is solved by a Monte Carlo method that is amenable to the use of $MC^3$ to solve the inverse transport problem using a Bayesian formalism.
Citation: Guillaume Bal, Ian Langmore, Youssef Marzouk. Bayesian inverse problems with Monte Carlo forward models. Inverse Problems & Imaging, 2013, 7 (1) : 81-105. doi: 10.3934/ipi.2013.7.81
References:
[1]

Simon Arridge, et al., Approximation errors and model reduction with an application in optical diffusion tomography,, Inverse Problems, 22 (2006).   Google Scholar

[2]

Guillaume Bal, Anthony Davis and Ian Langmore, A hybrid (Monte Carlo/deterministic) approach for multi-dimensional radiation transport,, J. Computational Physics, 230 (2011), 7723.   Google Scholar

[3]

George Casella and Robert Berger, "Statistical Inference,", Duxbury, (2002).   Google Scholar

[4]

Jin Chen and Xavier Intes, Time-gated perturbation Monte Carlo for whole body functional imaging in small animals,, Optics Express, 17 (2009).   Google Scholar

[5]

J. Andrés Christen and Colin Fox, Markov chain Monte Carlo using an approximation,, Journal of Computational and Graphical Statistics, 14 (2005), 795.  doi: 10.1198/106186005X76983.  Google Scholar

[6]

Rick Durrett, "Probability: Theory and Examples,", third edition, (2005).   Google Scholar

[7]

Yalchin Efendiev, Thomas Hou and W. Luo, Preconditioning Markov chain Monte Carlo simulations using coarse-scale models,, SIAM J. Sci. Comput., 28 (2006), 776.  doi: 10.1137/050628568.  Google Scholar

[8]

Charles J. Geyer, Practical Markov chain Monte Carlo,, Statistical Science, 7 (1992), 473.   Google Scholar

[9]

Carole K. Hayakawa and Jerome Spanier, Perturbation Monte Carlo methods for the solution of inverse problems,, in, (2004), 227.   Google Scholar

[10]

Carole K. Hayakawa, Jerome Spanier, and Frédéric Bevilacqua, et al., Perturbation Monte Carlo methods to solve inverse photon migration problems in heterogeneous tissues,, Optics Letters, 26 (2001), 1333.   Google Scholar

[11]

Jari P. Kaipio and Erkki Somersalo, "Statistical and Computational Inverse Problems,", Applied Mathematical Sciences, 160 (2005).   Google Scholar

[12]

Jari P. Kaipio and Erkki Somersalo, Statistical inverse problems: Discretization, model reduction, and inverse crimes,, Journal of Computational and Applied Mathematics, 198 (2007), 493.  doi: 10.1016/j.cam.2005.09.027.  Google Scholar

[13]

Ian Langmore, Anthony Davis and Guillaume Bal, Multi-pixel retrieval of structural and optical parameters in a 2D scene with a path-recycling Monte Carlo forward model and a new Bayesian inference engine,, IEEE TGRS, (2012).   Google Scholar

[14]

Jun S. Liu, "Monte Carlo Strategies in Scientific Computing,", Springer Series in Statistics, (2008).   Google Scholar

[15]

David Moulton, Colin Fox and Daniil Svyatskiy, Multilevel approximations in sample-based inversion from the Dirichlet-to-Neumann map,, Journal of Physics: Conference Series, (2008).   Google Scholar

[16]

Hanna K. Pikkarainen, State estimation approach to nonstationary inverse problems: Discretization error and filtering problem,, Inverse Problems, 22 (2006), 365.  doi: 10.1088/0266-5611/22/1/020.  Google Scholar

[17]

Christian Robert and George Casella, "Monte Carlo Statistical Methods,", Second edition, (2004).   Google Scholar

[18]

Luke Tierney, Markov chains for exploring posterior distributions,, The Annals of Statistics, 22 (1994), 1701.  doi: 10.1214/aos/1176325750.  Google Scholar

show all references

References:
[1]

Simon Arridge, et al., Approximation errors and model reduction with an application in optical diffusion tomography,, Inverse Problems, 22 (2006).   Google Scholar

[2]

Guillaume Bal, Anthony Davis and Ian Langmore, A hybrid (Monte Carlo/deterministic) approach for multi-dimensional radiation transport,, J. Computational Physics, 230 (2011), 7723.   Google Scholar

[3]

George Casella and Robert Berger, "Statistical Inference,", Duxbury, (2002).   Google Scholar

[4]

Jin Chen and Xavier Intes, Time-gated perturbation Monte Carlo for whole body functional imaging in small animals,, Optics Express, 17 (2009).   Google Scholar

[5]

J. Andrés Christen and Colin Fox, Markov chain Monte Carlo using an approximation,, Journal of Computational and Graphical Statistics, 14 (2005), 795.  doi: 10.1198/106186005X76983.  Google Scholar

[6]

Rick Durrett, "Probability: Theory and Examples,", third edition, (2005).   Google Scholar

[7]

Yalchin Efendiev, Thomas Hou and W. Luo, Preconditioning Markov chain Monte Carlo simulations using coarse-scale models,, SIAM J. Sci. Comput., 28 (2006), 776.  doi: 10.1137/050628568.  Google Scholar

[8]

Charles J. Geyer, Practical Markov chain Monte Carlo,, Statistical Science, 7 (1992), 473.   Google Scholar

[9]

Carole K. Hayakawa and Jerome Spanier, Perturbation Monte Carlo methods for the solution of inverse problems,, in, (2004), 227.   Google Scholar

[10]

Carole K. Hayakawa, Jerome Spanier, and Frédéric Bevilacqua, et al., Perturbation Monte Carlo methods to solve inverse photon migration problems in heterogeneous tissues,, Optics Letters, 26 (2001), 1333.   Google Scholar

[11]

Jari P. Kaipio and Erkki Somersalo, "Statistical and Computational Inverse Problems,", Applied Mathematical Sciences, 160 (2005).   Google Scholar

[12]

Jari P. Kaipio and Erkki Somersalo, Statistical inverse problems: Discretization, model reduction, and inverse crimes,, Journal of Computational and Applied Mathematics, 198 (2007), 493.  doi: 10.1016/j.cam.2005.09.027.  Google Scholar

[13]

Ian Langmore, Anthony Davis and Guillaume Bal, Multi-pixel retrieval of structural and optical parameters in a 2D scene with a path-recycling Monte Carlo forward model and a new Bayesian inference engine,, IEEE TGRS, (2012).   Google Scholar

[14]

Jun S. Liu, "Monte Carlo Strategies in Scientific Computing,", Springer Series in Statistics, (2008).   Google Scholar

[15]

David Moulton, Colin Fox and Daniil Svyatskiy, Multilevel approximations in sample-based inversion from the Dirichlet-to-Neumann map,, Journal of Physics: Conference Series, (2008).   Google Scholar

[16]

Hanna K. Pikkarainen, State estimation approach to nonstationary inverse problems: Discretization error and filtering problem,, Inverse Problems, 22 (2006), 365.  doi: 10.1088/0266-5611/22/1/020.  Google Scholar

[17]

Christian Robert and George Casella, "Monte Carlo Statistical Methods,", Second edition, (2004).   Google Scholar

[18]

Luke Tierney, Markov chains for exploring posterior distributions,, The Annals of Statistics, 22 (1994), 1701.  doi: 10.1214/aos/1176325750.  Google Scholar

[1]

Gunther Uhlmann, Jian Zhai. Inverse problems for nonlinear hyperbolic equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 455-469. doi: 10.3934/dcds.2020380

[2]

Yi-Hsuan Lin, Gen Nakamura, Roland Potthast, Haibing Wang. Duality between range and no-response tests and its application for inverse problems. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020072

[3]

Kha Van Huynh, Barbara Kaltenbacher. Some application examples of minimization based formulations of inverse problems and their regularization. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020074

[4]

Feifei Cheng, Ji Li. Geometric singular perturbation analysis of Degasperis-Procesi equation with distributed delay. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 967-985. doi: 10.3934/dcds.2020305

[5]

Sushil Kumar Dey, Bibhas C. Giri. Coordination of a sustainable reverse supply chain with revenue sharing contract. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020165

[6]

Zonghong Cao, Jie Min. Selection and impact of decision mode of encroachment and retail service in a dual-channel supply chain. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020167

[7]

Kien Trung Nguyen, Vo Nguyen Minh Hieu, Van Huy Pham. Inverse group 1-median problem on trees. Journal of Industrial & Management Optimization, 2021, 17 (1) : 221-232. doi: 10.3934/jimo.2019108

[8]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

[9]

Touria Karite, Ali Boutoulout. Global and regional constrained controllability for distributed parabolic linear systems: RHUM approach. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020055

[10]

Noufel Frikha, Valentin Konakov, Stéphane Menozzi. Well-posedness of some non-linear stable driven SDEs. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 849-898. doi: 10.3934/dcds.2020302

[11]

Monia Capanna, Jean C. Nakasato, Marcone C. Pereira, Julio D. Rossi. Homogenization for nonlocal problems with smooth kernels. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020385

[12]

Yongge Tian, Pengyang Xie. Simultaneous optimal predictions under two seemingly unrelated linear random-effects models. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020168

[13]

Chao Wang, Qihuai Liu, Zhiguo Wang. Periodic bouncing solutions for Hill's type sub-linear oscillators with obstacles. Communications on Pure & Applied Analysis, 2021, 20 (1) : 281-300. doi: 10.3934/cpaa.2020266

[14]

Vieri Benci, Sunra Mosconi, Marco Squassina. Preface: Applications of mathematical analysis to problems in theoretical physics. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020446

[15]

Wenbin Li, Jianliang Qian. Simultaneously recovering both domain and varying density in inverse gravimetry by efficient level-set methods. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020073

[16]

Noriyoshi Fukaya. Uniqueness and nondegeneracy of ground states for nonlinear Schrödinger equations with attractive inverse-power potential. Communications on Pure & Applied Analysis, 2021, 20 (1) : 121-143. doi: 10.3934/cpaa.2020260

[17]

Kai Yang. Scattering of the focusing energy-critical NLS with inverse square potential in the radial case. Communications on Pure & Applied Analysis, 2021, 20 (1) : 77-99. doi: 10.3934/cpaa.2020258

[18]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046

[19]

Shiqiu Fu, Kanishka Perera. On a class of semipositone problems with singular Trudinger-Moser nonlinearities. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020452

[20]

Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, 2020  doi: 10.3934/fods.2020018

2019 Impact Factor: 1.373

Metrics

  • PDF downloads (55)
  • HTML views (0)
  • Cited by (10)

Other articles
by authors

[Back to Top]