American Institute of Mathematical Sciences

Advanced
2015, 2015(special): 75-84. doi: 10.3934/proc.2015.0075

Vectorized and parallel particle filter SMC parameter estimation for stiff ODEs

Andrea Arnold 1, , Daniela Calvetti 2, and  Erkki Somersalo 3,

1. 

Department of Mathematics, North Carolina State University, Campus Box 8205, 2311 Stinson Drive, 2108 SAS Hall, Raleigh, NC 27695-8205, United States
2. 

Case Western Reserve University, Department of Mathematics and Center for Modelling Integrated Metabolic Systems, 10900 Euclid Avenue, Cleveland, OH 44106
3. 

Case Western Reserve University, Department of Mathematics, Applied Mathematics, and Statistics, Cleveland, OH 44106

Received  August 2014 Revised  September 2015 Published  November 2015

Particle filter (PF) sequential Monte Carlo (SMC) methods are very attractive for estimating parameters of time-dependent systems where the data is either not all available at once, or the range of time constants is wide enough to create problems in the numerical time propagation of the states. The need to evolve (and hence integrate) a large number of particles makes PF-based methods computationally challenging, and parallelization is often advocated to speed up computing time. While careful parallelization may indeed improve performance, vectorization of the algorithm on a single processor may result in even larger speedups for certain problems. In this paper we demonstrate how the PF-SMC class of algorithms proposed in [2] can be implemented in both parallel and vectorized computing environments, illustrating the performance with computed examples in MATLAB. In particular, two stiff test problems with different features show that both the size and structure of the problem affect which version of the algorithm is more efficient.
Keywords: vectorization, Markov chain Monte Carlo, linear multistep methods., Parallel computing, particle lters.
Mathematics Subject Classification: Primary: 65Y05, 65Y10; Secondary: 62M20, 65L06, 62M0.
Citation: Andrea Arnold, Daniela Calvetti, Erkki Somersalo. Vectorized and parallel particle filter SMC parameter estimation for stiff ODEs. Conference Publications, 2015, 2015 (special) : 75-84. doi: 10.3934/proc.2015.0075
References:
[1]

A. Arnold, Sequential Monte Carlo Parameter Estimation for Differential Equations,, Ph.D thesis, (2014).   Google Scholar

[2]

A. Arnold, D. Calvetti and E. Somersalo, Linear multistep methods, particle filtering and sequential Monte Carlo,, Inverse Problems, 29 (2013).   Google Scholar

[3]

O. Brun, V. Teuliere and J.-M. Garcia, Parallel particle filtering,, Journal of Parallel and Distributed Computing, 62 (2002), 1186.   Google Scholar

[4]

B. Calderhead, M. Girolami and N. D. Lawrence, Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes,, Adv. Neural Inf. Process. Syst., 21 (2009), 217.   Google Scholar

[5]

D. Calvetti and E. Somersalo, Large scale statistical parameter estimation in complex systems with an application to metabolic models,, Multiscale Model. Simul., 5 (2006), 1333.   Google Scholar

[6]

N. Chopin, P. E. Jacob and O. Papaspiliopoulos, SMC$^2$: an efficient algorithm for sequential analysis of state space models,, J. R. Stat. Soc. Ser. B Stat. Methodol., 75 (2013), 397.   Google Scholar

[7]

A. Golightly and D. J. Wilkinson, Bayesian parameter inference for stochastic biochemical network models using particle MCMC,, J. R. Soc. Interface Focus, 1 (2011), 807.   Google Scholar

[8]

A. Iserles, A First Course in the Numerical Analysis of Differential Equations,, $2^{nd}$ edition, (2009).   Google Scholar

[9]

A. Lee, C. Yau, M. B. Giles, A. Doucet and C. C. Holmes, On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods,, J. Comput. Graph. Statist., 19 (2010), 769.   Google Scholar

[10]

R. J. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations,, SIAM, (2007).   Google Scholar

[11]

C. Lieberman and K. Willcox, Goal-oriented inference: approach, linear theory, and application to advection diffusion,, SIAM Review, 55 (2013), 493.   Google Scholar

[12]

J. Liu and M. West, Combined parameter and state estimation in simulation-based filtering,, in Sequential Monte Carlo Methods in Practice (eds. A. Doucet, (2001), 197.   Google Scholar

[13]

S. Maskell, B. Alun-Jones and M. Macleod, A single instruction multiple data particle filter,, Nonlinear Statistical Signal Processing Workshop 2006 IEEE, (2006), 51.   Google Scholar

[14]

L. M. Murray, Bayesian state-space modelling on high-performance hardware using LibBi, preprint,, , ().   Google Scholar

[15]

M. Pitt and N. Shephard, Filtering via simulation: auxiliary particle filters,, J. Amer. Statist. Assoc., 94 (1999), 590.   Google Scholar

[16]

R. Ren and G. Orkoulas, Parallel Markov chain Monte Carlo simulations,, The Journal of Chemical Physics, 126 (2007).   Google Scholar

[17]

L. R. Scott, T. Clark and B. Bagheri, Scientific Parallel Computing,, Princeton, (2005).   Google Scholar

[18]

M. West, Approximating posterior distributions by mixtures,, J. R. Stat. Soc. Ser. B Stat. Methodol., 55 (1993), 409.   Google Scholar

[19]

M. West, Mixture models, Monte Carlo, Bayesian updating and dynamic models,, in Computing Science and Statistics: Proceedings of the 24th Symposium on the Interface (ed. J. H. Newton), (1993), 325.   Google Scholar

[20]

D. J. Wilkinson, Parallel Bayesian computation,, in Handbook of Parallel Computing and Statistics (ed. E. J. Kontoghiorghes), (2005), 477.   Google Scholar

[21]

Y. Zhou, vSMC: Parallel sequential Monte Carlo in C++, preprint,, , ().   Google Scholar

show all references

References:
[1]

A. Arnold, Sequential Monte Carlo Parameter Estimation for Differential Equations,, Ph.D thesis, (2014).   Google Scholar

[2]

A. Arnold, D. Calvetti and E. Somersalo, Linear multistep methods, particle filtering and sequential Monte Carlo,, Inverse Problems, 29 (2013).   Google Scholar

[3]

O. Brun, V. Teuliere and J.-M. Garcia, Parallel particle filtering,, Journal of Parallel and Distributed Computing, 62 (2002), 1186.   Google Scholar

[4]

B. Calderhead, M. Girolami and N. D. Lawrence, Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes,, Adv. Neural Inf. Process. Syst., 21 (2009), 217.   Google Scholar

[5]

D. Calvetti and E. Somersalo, Large scale statistical parameter estimation in complex systems with an application to metabolic models,, Multiscale Model. Simul., 5 (2006), 1333.   Google Scholar

[6]

N. Chopin, P. E. Jacob and O. Papaspiliopoulos, SMC$^2$: an efficient algorithm for sequential analysis of state space models,, J. R. Stat. Soc. Ser. B Stat. Methodol., 75 (2013), 397.   Google Scholar

[7]

A. Golightly and D. J. Wilkinson, Bayesian parameter inference for stochastic biochemical network models using particle MCMC,, J. R. Soc. Interface Focus, 1 (2011), 807.   Google Scholar

[8]

A. Iserles, A First Course in the Numerical Analysis of Differential Equations,, $2^{nd}$ edition, (2009).   Google Scholar

[9]

A. Lee, C. Yau, M. B. Giles, A. Doucet and C. C. Holmes, On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods,, J. Comput. Graph. Statist., 19 (2010), 769.   Google Scholar

[10]

R. J. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations,, SIAM, (2007).   Google Scholar

[11]

C. Lieberman and K. Willcox, Goal-oriented inference: approach, linear theory, and application to advection diffusion,, SIAM Review, 55 (2013), 493.   Google Scholar

[12]

J. Liu and M. West, Combined parameter and state estimation in simulation-based filtering,, in Sequential Monte Carlo Methods in Practice (eds. A. Doucet, (2001), 197.   Google Scholar

[13]

S. Maskell, B. Alun-Jones and M. Macleod, A single instruction multiple data particle filter,, Nonlinear Statistical Signal Processing Workshop 2006 IEEE, (2006), 51.   Google Scholar

[14]

L. M. Murray, Bayesian state-space modelling on high-performance hardware using LibBi, preprint,, , ().   Google Scholar

[15]

M. Pitt and N. Shephard, Filtering via simulation: auxiliary particle filters,, J. Amer. Statist. Assoc., 94 (1999), 590.   Google Scholar

[16]

R. Ren and G. Orkoulas, Parallel Markov chain Monte Carlo simulations,, The Journal of Chemical Physics, 126 (2007).   Google Scholar

[17]

L. R. Scott, T. Clark and B. Bagheri, Scientific Parallel Computing,, Princeton, (2005).   Google Scholar

[18]

M. West, Approximating posterior distributions by mixtures,, J. R. Stat. Soc. Ser. B Stat. Methodol., 55 (1993), 409.   Google Scholar

[19]

M. West, Mixture models, Monte Carlo, Bayesian updating and dynamic models,, in Computing Science and Statistics: Proceedings of the 24th Symposium on the Interface (ed. J. H. Newton), (1993), 325.   Google Scholar

[20]

D. J. Wilkinson, Parallel Bayesian computation,, in Handbook of Parallel Computing and Statistics (ed. E. J. Kontoghiorghes), (2005), 477.   Google Scholar

[21]

Y. Zhou, vSMC: Parallel sequential Monte Carlo in C++, preprint,, , ().   Google Scholar

[1]

Vieri Benci, Marco Cococcioni. The algorithmic numbers in non-archimedean numerical computing environments. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020449
[2]

Sushil Kumar Dey, Bibhas C. Giri. Coordination of a sustainable reverse supply chain with revenue sharing contract. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020165
[3]

Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115
[4]

Zonghong Cao, Jie Min. Selection and impact of decision mode of encroachment and retail service in a dual-channel supply chain. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020167
[5]

Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079
[6]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444
[7]

Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026
[8]

Shun Zhang, Jianlin Jiang, Su Zhang, Yibing Lv, Yuzhen Guo. ADMM-type methods for generalized multi-facility Weber problem. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020171
[9]

Touria Karite, Ali Boutoulout. Global and regional constrained controllability for distributed parabolic linear systems: RHUM approach. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020055
[10]

Noufel Frikha, Valentin Konakov, Stéphane Menozzi. Well-posedness of some non-linear stable driven SDEs. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 849-898. doi: 10.3934/dcds.2020302
[11]

Yongge Tian, Pengyang Xie. Simultaneous optimal predictions under two seemingly unrelated linear random-effects models. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020168
[12]

Wenbin Li, Jianliang Qian. Simultaneously recovering both domain and varying density in inverse gravimetry by efficient level-set methods. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020073
[13]

Chao Wang, Qihuai Liu, Zhiguo Wang. Periodic bouncing solutions for Hill's type sub-linear oscillators with obstacles. Communications on Pure & Applied Analysis, 2021, 20 (1) : 281-300. doi: 10.3934/cpaa.2020266

 Impact Factor: 

Tools

Metrics

  • PDF downloads (38)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences