American Institute of Mathematical Sciences

Advanced
doi: 10.3934/dcdss.2022054
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

Comparison of simulation-based algorithms for parameter estimation and state reconstruction in nonlinear state-space models

Thi Tuyet Trang Chau 1,, , Pierre Ailliot 2, , Valérie Monbet 3, and  Pierre Tandeo 4,

1. 

Univ Rennes, IRMAR-UMR CNRS 6625, F-35000 Rennes, France
2. 

Univ Brest, CNRS UMR 6205, Laboratoire de Mathematiques de Bretagne Atlantique, France
3. 

Univ Rennes, INRIA/SIMSMART, CNRS, IRMAR-UMR 6625, F-35000 Rennes, France
4. 

IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238 Brest, France

* Corresponding author: Thi Tuyet Trang Chau

Present address: Laboratoire des Sciences du Climat et de l'Environnement (LSCE/IPSL UMR CEA-CNRS-UVSQ), F-91191 Gif-Sur-Yvette Cedex, France.

Received  July 2021 Early access March 2022

This study aims at comparing simulation-based approaches for estimating both the state and unknown parameters in nonlinear state-space models. Numerical results on different toy models show that the combination of a Conditional Particle Filter (CPF) with Backward Simulation (BS) smoother and a Stochastic Expectation-Maximization (SEM) algorithm is a promising approach. The CPFBS smoother run with a small number of particles allows to explore efficiently the state-space and simulate relevant trajectories of the state conditionally to the observations. When combined with the SEM algorithm, this algorithm provides accurate estimates of the state and the parameters in nonlinear models, where the application of EM algorithms combined with a standard particle smoother or an ensemble Kalman smoother is limited.

Keywords: EM algorithms, conditional particle filtering, backward simulation, nonlinear models, statistical inference.
Mathematics Subject Classification: Primary: 62M05, 62F10, 62F15, 62F86.
Citation: Thi Tuyet Trang Chau, Pierre Ailliot, Valérie Monbet, Pierre Tandeo. Comparison of simulation-based algorithms for parameter estimation and state reconstruction in nonlinear state-space models. Discrete and Continuous Dynamical Systems - S, doi: 10.3934/dcdss.2022054
References:
[1]

C. Andrieu, A. Doucet and R. Holenstein, Particle markov chain monte Carlo methods, J. R. Stat. Soc. Ser. B Stat. Methodol., 72 (2010), 269-342.  doi: 10.1111/j.1467-9868.2009.00736.x.

[2]

M. Aoki, State Space Modeling of Time Series, Springer-Verlag, Berlin, 1987. doi: 10.1007/978-3-642-96985-0.

[3] D. Barber, A. T. Cemgil and S. Chiappa, Bayesian Time Series Models, Cambridge University Press, 2011.  doi: 10.1017/CBO9780511984679.
[4]

T. Berry and T. Sauer, Adaptive ensemble Kalman filtering of non-linear systems, Tellus A: Dynamic Meteorology and Oceanography, 65 (2013), 20331.  doi: 10.3402/tellusa.v65i0.20331.

[5]

M. Bocquet and P. Sakov, Joint state and parameter estimation with an iterative ensemble Kalman smoother, Nonlin. Processes Geophys., 20 (2013), 803-818.  doi: 10.5194/npg-20-803-2013.

[6]

M. Bocquet and P. Sakov, Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems, Nonlinear Processes in Geophysics, 19 (2012), 383-399.  doi: 10.5194/npg-19-383-2012.

[7]

M. Bocquet and P. Sakov, An iterative ensemble Kalman smoother, Quarterly Journal of the Royal Meteorological Society, 140 (2014), 1521-1535.  doi: 10.1002/qj.2236.

[8]

O. Cappé, S. J. Godsill and E. Moulines, An overview of existing methods and recent advances in sequential monte carlo, Proceedings of the IEEE, 95 (2007), 899-924. 

[9]

A. Carrassi, M. Bocquet, L. Bertino and G. Evensen, Data assimilation in the geosciences: An overview of methods, issues, and perspectives, Wiley Interdisciplinary Reviews: Climate Change, 9 (2018), e535.  doi: 10.1002/wcc.535.

[10]

G. Celeux, D. Chauveau and J. Diebolt, On Stochastic Versions of the EM Algorithm, Research Report RR-2514, INRIA, 1995.

[11]

K. S. Chan and J. Ledolter, Monte Carlo EM estimation for time series models involving counts, J. Amer. Statist. Assoc., 90 (1995), 242-252.  doi: 10.1080/01621459.1995.10476508.

[12]

N. Chopin and S. S. Singh, On particle gibbs sampling, Bernoulli, 21 (2015), 1855-1883.  doi: 10.3150/14-BEJ629.

[13]

B. Delyon, M. Lavielle and E. Moulines, Convergence of a stochastic approximation version of the em algorithm, Ann. Statist., 27 (1999), 94-128.  doi: 10.1214/aos/1018031103.

[14]

A. P. Dempster, N. M. Laird and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Ser. B, 39 (1977), 1-38.  doi: 10.1111/j.2517-6161.1977.tb01600.x.

[15]

R. Douc and O. Cappé, Comparison of resampling schemes for particle filtering, in ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, IEEE, 2005, 64-69. doi: 10.1109/ISPA.2005.195385.

[16]

R. Douc, A. Garivier, E. Moulines and J. Olsson, On the forward filtering backward smoothing particle approximations of the smoothing distribution in general state spaces models, arXiv preprint, arXiv: 0904.0316.

[17]

A. Doucet, N. de Freitas and N. Gordon (eds.), Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science, Springer-Verlag, New York, 2001. doi: 10.1007/978-1-4757-3437-9_1.

[18]

A. Doucet, S. Godsill and C. Andrieu, On Sequential Monte Carlo Sampling Methods for Bayesian Filtering, 1998.

[19]

A. Doucet and A. M. Johansen, A tutorial on particle filtering and smoothing: Fifteen years later, The Oxford Handbook of Nonlinear filtering, 656–704, Oxford Univ. Press, Oxford, 2011.

[20]

D. Dreano, P. Tandeo, M. Pulido, B. Ait-El-Fquih, T. Chonavel and I. Hoteit, Estimating model-error covariances in nonlinear state-space models using kalman smoothing and the expectation-maximization algorithm, Quarterly Journal of the Royal Meteorological Society, 143 (2017), 1877-1885.  doi: 10.1002/qj.3048.

[21] J. Durbin and S. J. Koopman, Time Series Analysis by State Space Methods, Oxford university press, 2012.  doi: 10.1093/acprof:oso/9780199641178.001.0001.
[22]

G. Evensen, The ensemble Kalman filter: Theoretical formulation and practical implementation, Ocean Dynamics, 53 (2003), 343-367.  doi: 10.1007/s10236-003-0036-9.

[23]

G. Evensen and P. J. van Leeuwen, An ensemble Kalman smoother for nonlinear dynamics, Monthly Weather Review, 128 (2000), 1852-1867.  doi: 10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2.

[24]

S. J. Godsill, A. Doucet and M. West, Monte Carlo smoothing for nonlinear time series, J. Amer. Statist. Assoc., 99 (2004), 156-168.  doi: 10.1198/016214504000000151.

[25]

J. D. Hol, T. B. Schon and F. Gustafsson, On resampling algorithms for particle filters, in Nonlinear Statistical Signal Processing Workshop, 2006 IEEE, IEEE, 2006, 79-82. doi: 10.1109/NSSPW.2006.4378824.

[26]

N. Kantas, A. Doucet, S. S. Singh, J. Maciejowski and N. Chopin, On particle methods for parameter estimation in state-space models, Statist. Sci., 30 (2015), 328-351.  doi: 10.1214/14-STS511.

[27]

G. Kitagawa, Monte Carlo filter and smoother for non-Gaussian nonlinear state space models, J. Comput. Graph. Statist., 5 (1996), 1-25.  doi: 10.2307/1390750.

[28]

J. Kokkala, A. Solin and S. Särkkä, Expectation maximization based parameter estimation by sigma-point and particle smoothing, in FUSION, IEEE, 2014, 1-8.

[29]

E. Kuhn and M. Lavielle, Coupling a stochastic approximation version of EM with an MCMC procedure, ESAIM Probab. Stat., 8 (2004), 115-131.  doi: 10.1051/ps:2004007.

[30]

F. Le Gland, V. Monbet and V.-D. Tran, Large sample asymptotics for the ensemble kalman filter, in Handbook on Nonlinear Filtering (eds. D. Crisan and B. Rozovskii), Oxford University Press, Oxford, 2011, chapter 22, 598–631.

[31]

R. Lguensat, P. Tandeo, P. Ailliot, M. Pulido and R. Fablet, The analog data assimilation, Monthly Weather Review, 145 (2017), 4093-4107.  doi: 10.1175/MWR-D-16-0441.1.

[32]

F. Lindsten, An efficient stochastic approximation EM algorithm using conditional particle filters, in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2013, 6274-6278. doi: 10.1109/ICASSP.2013.6638872.

[33]

F. Lindsten, Particle Filters and Markov Chains for Learning of Dynamical Systems, PhD thesis, Linköping University Electronic Press, 2013.

[34]

F. Lindsten, M. I. Jordan and T. B. Schön, Particle Gibbs with ancestor sampling, J. Mach. Learn. Res., 15 (2014), 2145-2184. 

[35]

F. Lindsten and T. B. Schön, On the use of backward simulation in the particle Gibbs sampler, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2012, 3845-3848. doi: 10.1109/ICASSP.2012.6288756.

[36]

F. Lindsten, T. B. Schön, Backward simulation methods for monte Carlo statistical inference, Foundations and Trends® in Machine Learning, 6 (2013), 1-143. doi: 10.1561/9781601986993.

[37]

F. Lindsten, T. Schön and M. I. Jordan, Ancestor sampling for particle Gibbs, in Advances in Neural Information Processing Systems, 2012, 2591-2599.

[38]

E. N. Lorenz, Deterministic nonperiodic flow, J. Atmospheric Sci., 20 (1963), 130-141.  doi: 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

[39]

G. McLachlan and T. Krishnan, The EM Algorithm and Extensions, vol. 382, John Wiley & Sons, 2008. doi: 10.1002/9780470191613.

[40]

M. Netto, L. Gimeno and M. Mendes, On the optimal and suboptimal nonlinear filtering problem for discrete-time systems, IEEE Transactions on Automatic Control, 23 (1978), 1062-1067.  doi: 10.1109/TAC.1978.1101894.

[41]

J. Olsson, O. Cappé, R. Douc and E. Moulines, Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models, Bernoulli, 14 (2008), 155-179.  doi: 10.3150/07-BEJ6150.

[42]

T. B. Schön, A. Wills and B. Ninness, System identification of nonlinear state-space models, Automatica J. IFAC, 47 (2011), 39-49.  doi: 10.1016/j.automatica.2010.10.013.

[43]

R. H. Shumway and D. S. Stoffer, An approach to time series smoothing and forecasting using the em algorithm, Journal of Time Series Analysis, 3 (1982), 253-264. 

[44]

A. Svensson, T. B. Schön and M. Kok, Nonlinear state space smoothing using the conditional particle filter, IFAC-PapersOnLine, 48 (2015), 975-980. 

[45]

P. Tandeo, P. Ailliot, M. Bocquet, A. Carrassi, T. Miyoshi, M. Pulido and Y. Zhen, A review of innovation-based methods to jointly estimate model and observation error covariance matrices in ensemble data assimilation, Monthly Weather Review, 148 (2020), 3973-3994. 

[46]

G. C. G. Wei and M. A. Tanner, A Monte Carlo implementation of the em algorithm and the poor man's data augmentation algorithms, Journal of the American statistical Association, 85 (1990), 699-704.  doi: 10.1080/01621459.1990.10474930.

[47]

N. Whiteley, Discussion on particle markov chain monte carlo methods, Journal of the Royal Statistical Society: Series B, 72 (2010), 306-307. 

show all references

References:
[1]

C. Andrieu, A. Doucet and R. Holenstein, Particle markov chain monte Carlo methods, J. R. Stat. Soc. Ser. B Stat. Methodol., 72 (2010), 269-342.  doi: 10.1111/j.1467-9868.2009.00736.x.

[2]

M. Aoki, State Space Modeling of Time Series, Springer-Verlag, Berlin, 1987. doi: 10.1007/978-3-642-96985-0.

[3] D. Barber, A. T. Cemgil and S. Chiappa, Bayesian Time Series Models, Cambridge University Press, 2011.  doi: 10.1017/CBO9780511984679.
[4]

T. Berry and T. Sauer, Adaptive ensemble Kalman filtering of non-linear systems, Tellus A: Dynamic Meteorology and Oceanography, 65 (2013), 20331.  doi: 10.3402/tellusa.v65i0.20331.

[5]

M. Bocquet and P. Sakov, Joint state and parameter estimation with an iterative ensemble Kalman smoother, Nonlin. Processes Geophys., 20 (2013), 803-818.  doi: 10.5194/npg-20-803-2013.

[6]

M. Bocquet and P. Sakov, Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems, Nonlinear Processes in Geophysics, 19 (2012), 383-399.  doi: 10.5194/npg-19-383-2012.

[7]

M. Bocquet and P. Sakov, An iterative ensemble Kalman smoother, Quarterly Journal of the Royal Meteorological Society, 140 (2014), 1521-1535.  doi: 10.1002/qj.2236.

[8]

O. Cappé, S. J. Godsill and E. Moulines, An overview of existing methods and recent advances in sequential monte carlo, Proceedings of the IEEE, 95 (2007), 899-924. 

[9]

A. Carrassi, M. Bocquet, L. Bertino and G. Evensen, Data assimilation in the geosciences: An overview of methods, issues, and perspectives, Wiley Interdisciplinary Reviews: Climate Change, 9 (2018), e535.  doi: 10.1002/wcc.535.

[10]

G. Celeux, D. Chauveau and J. Diebolt, On Stochastic Versions of the EM Algorithm, Research Report RR-2514, INRIA, 1995.

[11]

K. S. Chan and J. Ledolter, Monte Carlo EM estimation for time series models involving counts, J. Amer. Statist. Assoc., 90 (1995), 242-252.  doi: 10.1080/01621459.1995.10476508.

[12]

N. Chopin and S. S. Singh, On particle gibbs sampling, Bernoulli, 21 (2015), 1855-1883.  doi: 10.3150/14-BEJ629.

[13]

B. Delyon, M. Lavielle and E. Moulines, Convergence of a stochastic approximation version of the em algorithm, Ann. Statist., 27 (1999), 94-128.  doi: 10.1214/aos/1018031103.

[14]

A. P. Dempster, N. M. Laird and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Ser. B, 39 (1977), 1-38.  doi: 10.1111/j.2517-6161.1977.tb01600.x.

[15]

R. Douc and O. Cappé, Comparison of resampling schemes for particle filtering, in ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, IEEE, 2005, 64-69. doi: 10.1109/ISPA.2005.195385.

[16]

R. Douc, A. Garivier, E. Moulines and J. Olsson, On the forward filtering backward smoothing particle approximations of the smoothing distribution in general state spaces models, arXiv preprint, arXiv: 0904.0316.

[17]

A. Doucet, N. de Freitas and N. Gordon (eds.), Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science, Springer-Verlag, New York, 2001. doi: 10.1007/978-1-4757-3437-9_1.

[18]

A. Doucet, S. Godsill and C. Andrieu, On Sequential Monte Carlo Sampling Methods for Bayesian Filtering, 1998.

[19]

A. Doucet and A. M. Johansen, A tutorial on particle filtering and smoothing: Fifteen years later, The Oxford Handbook of Nonlinear filtering, 656–704, Oxford Univ. Press, Oxford, 2011.

[20]

D. Dreano, P. Tandeo, M. Pulido, B. Ait-El-Fquih, T. Chonavel and I. Hoteit, Estimating model-error covariances in nonlinear state-space models using kalman smoothing and the expectation-maximization algorithm, Quarterly Journal of the Royal Meteorological Society, 143 (2017), 1877-1885.  doi: 10.1002/qj.3048.

[21] J. Durbin and S. J. Koopman, Time Series Analysis by State Space Methods, Oxford university press, 2012.  doi: 10.1093/acprof:oso/9780199641178.001.0001.
[22]

G. Evensen, The ensemble Kalman filter: Theoretical formulation and practical implementation, Ocean Dynamics, 53 (2003), 343-367.  doi: 10.1007/s10236-003-0036-9.

[23]

G. Evensen and P. J. van Leeuwen, An ensemble Kalman smoother for nonlinear dynamics, Monthly Weather Review, 128 (2000), 1852-1867.  doi: 10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2.

[24]

S. J. Godsill, A. Doucet and M. West, Monte Carlo smoothing for nonlinear time series, J. Amer. Statist. Assoc., 99 (2004), 156-168.  doi: 10.1198/016214504000000151.

[25]

J. D. Hol, T. B. Schon and F. Gustafsson, On resampling algorithms for particle filters, in Nonlinear Statistical Signal Processing Workshop, 2006 IEEE, IEEE, 2006, 79-82. doi: 10.1109/NSSPW.2006.4378824.

[26]

N. Kantas, A. Doucet, S. S. Singh, J. Maciejowski and N. Chopin, On particle methods for parameter estimation in state-space models, Statist. Sci., 30 (2015), 328-351.  doi: 10.1214/14-STS511.

[27]

G. Kitagawa, Monte Carlo filter and smoother for non-Gaussian nonlinear state space models, J. Comput. Graph. Statist., 5 (1996), 1-25.  doi: 10.2307/1390750.

[28]

J. Kokkala, A. Solin and S. Särkkä, Expectation maximization based parameter estimation by sigma-point and particle smoothing, in FUSION, IEEE, 2014, 1-8.

[29]

E. Kuhn and M. Lavielle, Coupling a stochastic approximation version of EM with an MCMC procedure, ESAIM Probab. Stat., 8 (2004), 115-131.  doi: 10.1051/ps:2004007.

[30]

F. Le Gland, V. Monbet and V.-D. Tran, Large sample asymptotics for the ensemble kalman filter, in Handbook on Nonlinear Filtering (eds. D. Crisan and B. Rozovskii), Oxford University Press, Oxford, 2011, chapter 22, 598–631.

[31]

R. Lguensat, P. Tandeo, P. Ailliot, M. Pulido and R. Fablet, The analog data assimilation, Monthly Weather Review, 145 (2017), 4093-4107.  doi: 10.1175/MWR-D-16-0441.1.

[32]

F. Lindsten, An efficient stochastic approximation EM algorithm using conditional particle filters, in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2013, 6274-6278. doi: 10.1109/ICASSP.2013.6638872.

[33]

F. Lindsten, Particle Filters and Markov Chains for Learning of Dynamical Systems, PhD thesis, Linköping University Electronic Press, 2013.

[34]

F. Lindsten, M. I. Jordan and T. B. Schön, Particle Gibbs with ancestor sampling, J. Mach. Learn. Res., 15 (2014), 2145-2184. 

[35]

F. Lindsten and T. B. Schön, On the use of backward simulation in the particle Gibbs sampler, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2012, 3845-3848. doi: 10.1109/ICASSP.2012.6288756.

[36]

F. Lindsten, T. B. Schön, Backward simulation methods for monte Carlo statistical inference, Foundations and Trends® in Machine Learning, 6 (2013), 1-143. doi: 10.1561/9781601986993.

[37]

F. Lindsten, T. Schön and M. I. Jordan, Ancestor sampling for particle Gibbs, in Advances in Neural Information Processing Systems, 2012, 2591-2599.

[38]

E. N. Lorenz, Deterministic nonperiodic flow, J. Atmospheric Sci., 20 (1963), 130-141.  doi: 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.

[39]

G. McLachlan and T. Krishnan, The EM Algorithm and Extensions, vol. 382, John Wiley & Sons, 2008. doi: 10.1002/9780470191613.

[40]

M. Netto, L. Gimeno and M. Mendes, On the optimal and suboptimal nonlinear filtering problem for discrete-time systems, IEEE Transactions on Automatic Control, 23 (1978), 1062-1067.  doi: 10.1109/TAC.1978.1101894.

[41]

J. Olsson, O. Cappé, R. Douc and E. Moulines, Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models, Bernoulli, 14 (2008), 155-179.  doi: 10.3150/07-BEJ6150.

[42]

T. B. Schön, A. Wills and B. Ninness, System identification of nonlinear state-space models, Automatica J. IFAC, 47 (2011), 39-49.  doi: 10.1016/j.automatica.2010.10.013.

[43]

R. H. Shumway and D. S. Stoffer, An approach to time series smoothing and forecasting using the em algorithm, Journal of Time Series Analysis, 3 (1982), 253-264. 

[44]

A. Svensson, T. B. Schön and M. Kok, Nonlinear state space smoothing using the conditional particle filter, IFAC-PapersOnLine, 48 (2015), 975-980. 

[45]

P. Tandeo, P. Ailliot, M. Bocquet, A. Carrassi, T. Miyoshi, M. Pulido and Y. Zhen, A review of innovation-based methods to jointly estimate model and observation error covariance matrices in ensemble data assimilation, Monthly Weather Review, 148 (2020), 3973-3994. 

[46]

G. C. G. Wei and M. A. Tanner, A Monte Carlo implementation of the em algorithm and the poor man's data augmentation algorithms, Journal of the American statistical Association, 85 (1990), 699-704.  doi: 10.1080/01621459.1990.10474930.

[47]

N. Whiteley, Discussion on particle markov chain monte carlo methods, Journal of the Royal Statistical Society: Series B, 72 (2010), 306-307. 

Figure 1.  Impact of parameter values on smoothing distributions for the Lorenz-63 model (20). The true state (black curve) and observations (black points) have been simulated with $ \theta^* = ({\bf{Q}},{\bf{R}}) = (0.01{\bf{I}}_3,2{\bf{I}}_3) $. The mean of the smoothing distributions (red curve) are computed using a standard particle smoother [16] with $ 100 $ particles. Results obtained with the true parameter value $ \theta^* = (0.01{\bf{I}}_3,2{\bf{I}}_3) $ (left panel) and a wrong parameter value $ \tilde{\theta} = ({\bf{I}}_3,{\bf{I}}_3) $ (right panel) are plotted
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Comparison of one iteration of PF and CPF algorithms using $ N_f = 5 $ particles (light grey points). The differences are highlighted in black : CPF replaces the particle $ {\bf{x}}_t^{(N_f)} $ of the PF with the conditioning particle $ {\bf{x}}_t^* $ (dark grey point)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  Comparisons of PF and CPF algorithms with 10 particles on the Kitagawa model defined in Section 3.2. Grey lines show the ancestors of the particles
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  Example of ancestor tracking based on ancestral links of filtering particles. Particles (grey balls) are obtained using a filtering algorithm with $ N_f = 3 $ particles
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  Comparison of CPF (left), CPFAS (middle), and CPFBS (right). The state (black line) and the observations (black points) have been simulated using the Kitagawa model (19) with $ Q = 1 $ and $ R = 10 $. $ N_f = 10 $ particles (grey points with grey lines showing the genealogy) are used in the three algorithms. The red curves show $ N_s = 10 $ realizations simulated with the algorithms
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  Four iterations of the CPFBS smoother (Algorithm 3). The state (black line) and the observations (black points) have been simulated using the Kitagawa model (19) with $ Q = 1 $ and $ R = 10 $. At the first iteration, the conditioning trajectory (grey dotted line) is initialized with the constant sequence equal to $ 0 $. CPFBS is run with $ N_f = 10 $ particles (grey points) and $ N_s = 10 $ trajectories (red curves)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  Sequence simulated with the linear Gaussian SSM model (18) with $ \theta^* = (0.9, 1, 1) $. The mean of the smoothing distribution (red curve) and $ 95\% $ prediction interval (light red area) are computed based on the smoothing trajectories simulated in the last 10 iterations of CPFBS-SEM algorithm with $ N_f = N_s = 10 $ particles
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  Distribution of the estimates obtained with CPFBS-SEM and CPFAS-SEM algorithms as a function of the number of EM iterations for the linear Gaussian SSM model (18) with $ \theta^* = (0.9,1,1) $, $ T = 100 $, $ N_f = N_s = 10 $. The empirical distributions are computed using $ 100 $ simulated samples. The median (grey dotted line) and $ 95\% $ confidence interval (grey shaded area) are computed using $ 10^3 $ iterations of the KS-EM algorithm
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 9.  Distribution of the estimates obtained with CPFBS-SEM, CPFAS-SEM, PFBS-EM, and EnKS-EM algorithms as a function of the number of particles for the linear Gaussian SSM model (18) with $ \theta^* = (0.9,1,1) $ and $ T = 100 $. Results obtained by running $ 100 $ iterations of the algorithms. The empirical distributions are computed using $ 100 $ simulated samples
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 10.  Sequence simulated with the Kitagawa model (19) with $ \theta^* = (1, 10) $. The mean of the smoothing distribution (red curve) and $ 95\% $ prediction interval (light red area) are computed based on the smoothing trajectories simulated in the last 10 iterations of CPFBS-SEM algorithm with $ N_f = N_s = 10 $ particles
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 11.  Distribution of the estimates obtained with CPFBS-SEM and CPFAS-SEM algorithms as a function of the number of SEM iterations for the Kitagawa model (19) with $ \theta^* = (1,10) $, $ T = 100 $, $ N_f = N_s = 10 $. The empirical distributions are computed using $ 100 $ simulated samples
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 12.  Distribution of the estimates obtained with CPFBS-SEM and CPFAS-SEM algorithms for the Kitagawa model (19) with $ \theta^* = (1,R^*) $, $ R^* \in \{0.1, 1, 5, 10\} $, $ T = 100 $, $ N_f = N_s = 10 $. Results obtained by running $ 100 $ iterations of the SEM algorithms. The empirical distributions are computed using $ 100 $ simulated samples
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 13.  Sequence simulated with the Lorenz-63 model (20) with $ \theta^* = (0.01, 2) $ and time step $ \triangle = 0.15 $. The mean of the smoothing distribution (red curve) and $ 95\% $ prediction interval (light red area) are computed based on the smoothing trajectories simulated in the last 10 iterations of CPFBS-SEM algorithm with $ N_f = N_s = 10 $ particles
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 14.  Distribution of the estimates obtained with CPFBS-SEM and CPFAS-SEM algorithms as a function of the number of EM iterations for the Lorenz-63 models (20) with $ \theta^* = (0.01, 2) $, $ \triangle = 0.15 $ $ T = 100 $, $ N_f = N_s = 20 $. The empirical distributions are computed using $ 100 $ simulated samples
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 15.  Distribution of the estimates obtained with CPFBS-SEM, CPFAS-SEM, and EnKS-EM algorithms as a function of the time step $ \triangle $ for the Lorenz-63 models (20) with $ \theta^* = (0.01, 2) $, $ T = 100 $, $ N_f = N_s = 20 $ and $ 20 $ members for the EnKS algorithm. The empirical distributions are computed using $ 100 $ simulated samples
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Comparison of the reconstruction ability of the CPFBS and CPFAS smoothers using cross-validation on the Lorenz-63 model (20) with $ \triangle = 0.15, \theta^* = (0.01, 2) $. The parameter $ \theta $ is estimated on learning sequences of length $ T = 100 $. Given these estimates, the CPFBS and CPFAS algorithms are run on validation sequences of length $ T' = 100 $. The two scores are computed on only the second component (top) and over all the three components (bottom). Algorithms run with $ N_f = N_s = 20 $ particles/realizations. The median and $ 95\% $ CI of each score are evaluated based on 100 simulated sequences
$2^{\mathrm{nd}}$ component Number of iterations
$10$ $20$ $50$ $100$
CPFBS RMSE $0.4328$
$[ 0.3011 , 0.7473 ]$		 $0.3928$
$[ 0.2771 , 0.6258 ]$		 $0.3772$
$[ 0.2609 , 0.5752 ]$		 $0.3704$
$[ 0.2438 , 0.5737 ]$
CP $89\%$
$[ 72 \% , 97 \% ]$		 $93\%$
$[78\%, 99\%]$		 $96\%$
$[83\%, 100\%]$		 $97\%$
$[87\%, 100\%]$
CPFAS RMSE $0.4351$
$[ 0.2927 , 2.2515 ]$		 $0.4146 $
$[ 0.2532 , 1.216 ]$		 $ 0.3993$
$[ 0.2433 , 0.7047 ]$		 $0.3798$
$[ 0.2315 , 0.7068 ]$
CP $73 \% $
$[ 53 \% , 85 \% ]$		 $85 \%$
$[69\%, 95\%]$		 $92\%$
$[82\%, 99\%]$		 $95\%$
$[86\%, 100\%]$
Three components Number of iterations
$10$ $20$ $50$ $100$
CPFBS RMSE $0.4351$
$[0.2983, 0.7969]$		 $0.3990$
$[0.2771, 0.6277]$		 $0.3803$
$[0.2761, 0.5251]$		 $0.3722$
$[0.2758, 0.5053]$
CP $89.33\%$
$[72.42\%, 96.96\%]$		 $92.67\%$
$[77.71\%, 98.88\%]$		 $95.67\%$
$[83.38\%, 99.33\%]$		 $96.83\%$
$[88.71\%, 99.67\%]$
CPFAS RMSE $0.4354$
$[0.3199, 2.0301]$		 $0.4172$
$[0.3022, 1.1063]$		 $0.3912$
$[0.2611, 0.5682]$		 $0.3813$
$[0.2448, 0.5665]$
CP $71.67\%$
$[54.17\%, 84.5\%]$		 $85.17\%$
$[71.17\%, 94.63\%]$		 $92.5\%$
$[82.08\%, 98.29\%]$		 $95.0\%$
$[86.42\%, 99.29\%]$
Table Options

Download as excel

$2^{\mathrm{nd}}$ component Number of iterations
$10$ $20$ $50$ $100$
CPFBS RMSE $0.4328$
$[ 0.3011 , 0.7473 ]$		 $0.3928$
$[ 0.2771 , 0.6258 ]$		 $0.3772$
$[ 0.2609 , 0.5752 ]$		 $0.3704$
$[ 0.2438 , 0.5737 ]$
CP $89\%$
$[ 72 \% , 97 \% ]$		 $93\%$
$[78\%, 99\%]$		 $96\%$
$[83\%, 100\%]$		 $97\%$
$[87\%, 100\%]$
CPFAS RMSE $0.4351$
$[ 0.2927 , 2.2515 ]$		 $0.4146 $
$[ 0.2532 , 1.216 ]$		 $ 0.3993$
$[ 0.2433 , 0.7047 ]$		 $0.3798$
$[ 0.2315 , 0.7068 ]$
CP $73 \% $
$[ 53 \% , 85 \% ]$		 $85 \%$
$[69\%, 95\%]$		 $92\%$
$[82\%, 99\%]$		 $95\%$
$[86\%, 100\%]$
Three components Number of iterations
$10$ $20$ $50$ $100$
CPFBS RMSE $0.4351$
$[0.2983, 0.7969]$		 $0.3990$
$[0.2771, 0.6277]$		 $0.3803$
$[0.2761, 0.5251]$		 $0.3722$
$[0.2758, 0.5053]$
CP $89.33\%$
$[72.42\%, 96.96\%]$		 $92.67\%$
$[77.71\%, 98.88\%]$		 $95.67\%$
$[83.38\%, 99.33\%]$		 $96.83\%$
$[88.71\%, 99.67\%]$
CPFAS RMSE $0.4354$
$[0.3199, 2.0301]$		 $0.4172$
$[0.3022, 1.1063]$		 $0.3912$
$[0.2611, 0.5682]$		 $0.3813$
$[0.2448, 0.5665]$
CP $71.67\%$
$[54.17\%, 84.5\%]$		 $85.17\%$
$[71.17\%, 94.63\%]$		 $92.5\%$
$[82.08\%, 98.29\%]$		 $95.0\%$
$[86.42\%, 99.29\%]$
[1]

Annalisa Pascarella, Alberto Sorrentino, Cristina Campi, Michele Piana. Particle filtering, beamforming and multiple signal classification for the analysis of magnetoencephalography time series: a comparison of algorithms. Inverse Problems and Imaging, 2010, 4 (1) : 169-190. doi: 10.3934/ipi.2010.4.169
[2]

Xin Li, Feng Bao, Kyle Gallivan. A drift homotopy implicit particle filter method for nonlinear filtering problems. Discrete and Continuous Dynamical Systems - S, 2022, 15 (4) : 727-746. doi: 10.3934/dcdss.2021097
[3]

Yaxian Xu, Ajay Jasra. Particle filters for inference of high-dimensional multivariate stochastic volatility models with cross-leverage effects. Foundations of Data Science, 2019, 1 (1) : 61-85. doi: 10.3934/fods.2019003
[4]

Jin-Won Kim, Amirhossein Taghvaei, Yongxin Chen, Prashant G. Mehta. Feedback particle filter for collective inference. Foundations of Data Science, 2021, 3 (3) : 543-561. doi: 10.3934/fods.2021018
[5]

Ming Yan, Alex A. T. Bui, Jason Cong, Luminita A. Vese. General convergent expectation maximization (EM)-type algorithms for image reconstruction. Inverse Problems and Imaging, 2013, 7 (3) : 1007-1029. doi: 10.3934/ipi.2013.7.1007
[6]

Jiani Wang, Liwei Zhang. Statistical inference of semidefinite programming with multiple parameters. Journal of Industrial and Management Optimization, 2020, 16 (3) : 1527-1538. doi: 10.3934/jimo.2019015
[7]

Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems and Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002
[8]

Hassan Abdallah, Adam Regalski, Mohammad Behzad Kang, Maria Berishaj, Nkechi Nnadi, Asadur Chowdury, Vaibhav A. Diwadkar, Andrew Salch. Statistical inference for persistent homology applied to simulated fMRI time series data. Foundations of Data Science, 2022  doi: 10.3934/fods.2022014
[9]

Evangelos Evangelou. Approximate Bayesian inference for geostatistical generalised linear models. Foundations of Data Science, 2019, 1 (1) : 39-60. doi: 10.3934/fods.2019002
[10]

Oliver J. Maclaren, Helen M. Byrne, Alexander G. Fletcher, Philip K. Maini. Models, measurement and inference in epithelial tissue dynamics. Mathematical Biosciences & Engineering, 2015, 12 (6) : 1321-1340. doi: 10.3934/mbe.2015.12.1321
[11]

Araz Hashemi, George Yin, Le Yi Wang. Sign-error adaptive filtering algorithms involving Markovian parameters. Mathematical Control and Related Fields, 2015, 5 (4) : 781-806. doi: 10.3934/mcrf.2015.5.781
[12]

Jean-Paul Chehab. Damping, stabilization, and numerical filtering for the modeling and the simulation of time dependent PDEs. Discrete and Continuous Dynamical Systems - S, 2021, 14 (8) : 2693-2728. doi: 10.3934/dcdss.2021002
[13]

Samuel N. Cohen. Uncertainty and filtering of hidden Markov models in discrete time. Probability, Uncertainty and Quantitative Risk, 2020, 5 (0) : 4-. doi: 10.1186/s41546-020-00046-x
[14]

Seung-Yeal Ha, Doron Levy. Particle, kinetic and fluid models for phototaxis. Discrete and Continuous Dynamical Systems - B, 2009, 12 (1) : 77-108. doi: 10.3934/dcdsb.2009.12.77
[15]

David Cowan. Rigid particle systems and their billiard models. Discrete and Continuous Dynamical Systems, 2008, 22 (1&2) : 111-130. doi: 10.3934/dcds.2008.22.111
[16]

Xiaoming Wang. Numerical algorithms for stationary statistical properties of dissipative dynamical systems. Discrete and Continuous Dynamical Systems, 2016, 36 (8) : 4599-4618. doi: 10.3934/dcds.2016.36.4599
[17]

H. Thomas Banks, Shuhua Hu, Zackary R. Kenz, Hien T. Tran. A comparison of nonlinear filtering approaches in the context of an HIV model. Mathematical Biosciences & Engineering, 2010, 7 (2) : 213-236. doi: 10.3934/mbe.2010.7.213
[18]

Vadim Kaushansky, Christoph Reisinger. Simulation of a simple particle system interacting through hitting times. Discrete and Continuous Dynamical Systems - B, 2019, 24 (10) : 5481-5502. doi: 10.3934/dcdsb.2019067
[19]

Mathieu Lutz. Application of Lie transform techniques for simulation of a charged particle beam. Discrete and Continuous Dynamical Systems - S, 2015, 8 (1) : 185-221. doi: 10.3934/dcdss.2015.8.185
[20]

Linda J. S. Allen, P. van den Driessche. Stochastic epidemic models with a backward bifurcation. Mathematical Biosciences & Engineering, 2006, 3 (3) : 445-458. doi: 10.3934/mbe.2006.3.445

2021 Impact Factor: 1.865

Tools

Metrics

  • PDF downloads (240)
  • HTML views (271)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy