Article Contents
Article Contents

# Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix

• * Corresponding author: Marie Turčičová

This research was partially supported by TA CR grant TL01000238, NSF grant ICER-1664175 and by the project TO01000219 funded by Norway Grants and TA CR

• We present an ensemble filtering method based on a linear model for the precision matrix (the inverse of the covariance) with the parameters determined by Score Matching Estimation. The method provides a rigorous covariance regularization when the underlying random field is Gaussian Markov. The parameters are found by solving a system of linear equations. The analysis step uses the inverse formulation of the Kalman update. Several filter versions, differing in the construction of the analysis ensemble, are proposed, as well as a Score matching version of the Extended Kalman Filter.

Mathematics Subject Classification: 62F12, 62H12, 62P12.

 Citation:

• Figure 1.  Block band-diagonal structure of inverse covariance matrix for two-dimensional Markov fields of type $10\times10$ points. There are assumed 4, 8, 12 neighbours of any gridpoint

Figure 2.  The sparsity structure of the precision matrix (a) and of the matrix $\tilde{D}$ of the linear system for the estimation of $\boldsymbol{\beta}$ (b)

Figure 3.  Performance illustration of EnKF, ETKF, SMEF (Algorithm 2), SMETKF (Section 4.3) and SMEF-GR (Algorithm 1) for the Lorenz 96 model, for selected combinations of ensemble size and observation error. $N = 10, \rho = 3$ (a), $N = 20, \rho = 2$ (b), $N = 30, \rho = 1$ (c). Observation step 2

Figure 4.  RMSE of non-ensemble filters, averaged from 50 replications. $\rho = 2$ (a), $\rho = 3$ (b). Observation step 2

Figure 5.  Trajectories of RMSE of all 50 replications, $\rho = 2$. ExKF(a), SMExKF (Algorithm 3)(b)

Figure 6.  RMSE, averaged over 1000 replications, $N = 50$, observation step 4, $\rho = 3$ (a), $\rho = 4$ (b)

Table 1.  Lorenz 96: RMSE of different filtering algorithms averaged from 1000 time steps and 50 replications of the run, initialized from different randomly selected states. The "div" columns show the number of divergent replications

 EnKF ETKF SMEF SMETKF SMEF-GR N $\rho$ RMSE div RMSE div RMSE div RMSE div RMSE div 10 1 4.31 50 3.54 50 1.00 1 0.55 0 0.80 0 10 2 4.30 50 3.63 50 0.89 0 0.84 0 1.18 13 10 3 4.27 50 3.85 50 1.52 2 1.42 0 1.59 35 20 1 2.79 48 0.45 0 0.43 0 0.40 0 0.61 0 20 2 3.55 48 0.74 0 0.62 1 0.55 0 0.87 0 20 3 3.37 48 0.97 0 1.15 8 0.82 2 1.16 3 30 1 0.44 0 0.45 0 0.41 0 0.40 0 0.56 0 30 2 0.69 0 0.68 0 0.55 0 0.55 0 0.75 0 30 3 0.95 0 0.88 0 0.74 0 0.72 2 0.94 0

Table 2.  Lorenz 96: RMSE of the two non-ensemble filtering algorithms averaged from 1000 time steps and 50 replications of the run, initialized from different randomly selected states. The "div" columns show the number of divergent replications

 ExKF SMExKF Obs. step $\rho$ RMSE div RMSE div 2 1 0.34 0 0.54 0 2 2 0.53 1 0.71 0 2 3 1.71 16 0.86 0

Table 3.  Lorenz 96: RMSE of different filtering algorithms averaged from 1000 time steps and 50 replications of the run, initialized from different randomly selected states. The "div" columns show the number of divergent replications. Observation step 4

 EnKF ETKF SMEF SMETKF SMEF-GR N r RMSE div RMSE div RMSE div RMSE div RMSE div 30 3.0 3.46 46 3.35 40 2.49 30 3.51 34 2.53 1 30 4.0 3.56 40 3.50 41 2.63 45 3.66 43 2.67 0 40 3.0 3.55 36 3.32 24 2.05 23 3.35 39 2.34 0 40 4.0 3.75 30 3.58 23 2.42 47 3.56 42 2.49 0 50 3.0 3.85 33 3.53 15 2.02 19 3.25 35 2.18 0 50 4.0 3.98 39 3.75 28 2.22 41 3.39 44 2.36 1
•  [1] T. W. Anderson, Asymptotically efficient estimation of covariance matrices with linear structure, Ann. Statist., 1 (1973), 135-141.  doi: 10.1214/aos/1193342389. [2] O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Ltd., Chichester, 1978. [3] M. Bukal, I. Marković and I. Petrović, Score matching based assumed density filtering with the von Mises-Fisher distribution, 20th International Conference on Information Fusion (Fusion), Xi'an, China, 2017. [4] G. Burgers, P. J. van Leeuwen and G. Evensen, Analysis scheme in the ensemble Kalman filter, Monthly Weather Review, 126 (1998), 1719-1724.  doi: 10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2. [5] A. P. Dempster, Covariance selection, Biometrics, 28 (1972), 157-175.  doi: 10.2307/2528966. [6] P. G. M. Forbes and S. Lauritzen, Linear estimating equations for exponential families with application to Gaussian linear concentration models, Linear Algebra Appl., 473 (2015), 261-283.  doi: 10.1016/j.laa.2014.08.015. [7] R. Furrer and T. Bengtsson, Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants, J. Multivariate Anal., 98 (2007), 227-255.  doi: 10.1016/j.jmva.2006.08.003. [8] J. E. Gentle, Matrix Algebra. Theory, Computations and Applications in Statistics, 2$^nd$ edition, Springer Texts in Statistics, Springer, Cham, 2017. doi: 10.1007/978-3-319-64867-5. [9] A. Hyvärinen, Estimation of non-normalized statistical models by score matching, J. Mach. Learn. Res., 6 (2005), 695-709. [10] A. Hyvärinen, Some extensions of score matching, Comput. Statist. Data Anal., 51 (2007), 2499-2512.  doi: 10.1016/j.csda.2006.09.003. [11] I. Kasanický, J. Mandel and M. Vejmelka, Spectral diagonal ensemble Kalman filters, Nonlin. Processes Geophys., 22 (2015), 485-497.  doi: 10.5194/npg-22-485-2015. [12] M. Katzfuss, J. R. Stroud and C. K. Wikle, Understanding the ensemble Kalman filter, Amer. Statist., 70 (2016), 350-357.  doi: 10.1080/00031305.2016.1141709. [13] S. L. Lauritzen,  Graphical Models, Oxford Statistical Science Series, 17, Oxford Science Publications, The Clarendon Press, Oxford University Press, New York, 1996. [14] K. Law, A. Stuart and K. Zygalakis, Data assimilation. A Mathematical Introduction, Texts in Applied Mathematics, 62, Springer, Cham, 2015. doi: 10.1007/978-3-319-20325-6. [15] K. Law, H. Tembine and R. Tempone, Deterministic mean-field ensemble Kalman filtering, SIAM J. Sci. Comput., 38 (2016), A1251–A1279. doi: 10.1137/140984415. [16] F. Le Gland, V. Monbet and V.-D. Tran, Large sample asymptotics for the ensemble Kalman filter, in The Oxford Handbook of Nonlinear Filtering, Oxford Univ. Press, Oxford, 2011, 598–631. [17] E. L. Lehmann and J. P. Romano, Testing Statistical Hypotheses, 3$^{rd}$ edition, Springer Texts in Statistics, Springer, New York, 2005. doi: 10.1007/0-387-27605-X. [18] F. Lindgren, H. Rue and J. Lindström, An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach, J. R. Stat. Soc. Ser. B Stat. Methodol., 73 (2011), 423-498.  doi: 10.1111/j.1467-9868.2011.00777.x. [19] D. M. Livings, S. L. Dance and N. K. Nichols, Unbiased ensemble square root filters, Phys. D, 237 (2008), 1021-1028.  doi: 10.1016/j.physd.2008.01.005. [20] E. N. Lorenz, Predictability - A problem partly solved, in Predictability of Weather and Climate, Cambridge University Press, 2006, 40–58. doi: 10.1017/CBO9780511617652.004. [21] J. Mandel, L. Cobb and J. D. Beezley, On the convergence of the ensemble Kalman filter, Appl. Math., 56 (2011), 533-541.  doi: 10.1007/s10492-011-0031-2. [22] E. D. Nino-Ruiz and A. Sandu, Ensemble Kalman filter implementations based on shrinkage covariance matrix estimation, Ocean Dynamics, 65 (2015), 1423-1439.  doi: 10.1007/s10236-015-0888-9. [23] O. Pannekoucke, L. Berre and G. Desroziers, Filtering properties of wavelets for local background-error correlations, Quart. J. Roy. Meterol. Soc., 133 (2007), 363-379.  doi: 10.1002/qj.33. [24] J. A. Rozanov, Markov random fields, and stochastic partial differential equations, Mat. Sb. (N.S.), 103 (1977), 590-613. [25] H. Rue and L. Held, Gaussian Markov Random Fields. Theory and Applications, Monographs on Statistics and Applied Probability, 104, Chapman & Hall/CRC, Boca Raton, FL, 2005. doi: 10.1201/9780203492024. [26] P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus A, 60 (2008), 361-371.  doi: 10.1111/j.1600-0870.2007.00299.x. [27] D. Simpson, F. Lindgren and H. Rue, Think continuous: Markovian Gaussian models in spatial statistics, Spatial Statistics, 1 (2012), 16-29.  doi: 10.1016/j.spasta.2012.02.003. [28] A. Spantini, R. Baptista and Y. Marzouk, Coupling techniques for nonlinear ensemble filtering, preprint, arXiv: 1907.00389. [29] M. K. Tippett, J. L. Anderson, C. H. Bishop, T. M. Hamill and J. S. Whitaker, Ensemble square root filters, Monthly Weather Review, 131 (2003), 1485-1490.  doi: 10.1175/1520-0493(2003)131<1485:ESRF>2.0.CO;2. [30] F. Tronarp, R. Hostettler and S. Särkkä, Continuous-discrete von Mises-Fisher filtering on $S^2$ for reference vector tracking, 21st International Conference on Information Fusion (FUSION), Cambridge, UK, 2018. doi: 10.23919/ICIF.2018.8455299. [31] M. Turčičová, Covariance Estimation for Filtering in High Dimension, Ph.D thesis, Charles University, 2021. Available from: http://hdl.handle.net/20.500.11956/136400. [32] G. Ueno and T. Tsuchiya, Covariance regularization in inverse space, Quart. J. Roy. Meterol. Soc., 135 (2009), 1133-1156.  doi: 10.1002/qj.445. [33] A. W. van der Vaart,  Asymptotic Statistics, Cambridge Series in Statistical and Probabilistic Mathematics, 3, Cambridge University Press, Cambridge, 1998.  doi: 10.1017/CBO9780511802256. [34] X. Wang, C. H. Bishop and S. J. Julier, Which is better, an ensemble of positive–negative pairs or a centered spherical simplex ensemble?, Monthly Weather Review, 132 (2004), 1590-1605.  doi: 10.1175/1520-0493(2004)132<1590:WIBAEO>2.0.CO;2. [35] A. Weaver and P. Courtier, Correlation modelling on the sphere using a generalized diffusion equation, Quart. J. Roy. Meterol. Soc., 127 (2001), 1815-1846.  doi: 10.1002/qj.49712757518. [36] A. T. Weaver and I. Mirouze, On the diffusion equation and its application to isotropic and anisotropic correlation modelling in variational assimilation, Quart. J. Roy. Meterol. Soc., 139 (2013), 242-260.  doi: 10.1002/qj.1955.

Figures(6)

Tables(3)