Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix

Marie Turčičová; Jan Mandel; Kryštof Eben

doi:10.3934/fods.2021030

Article Contents

2021, Volume 3, Issue 4: 793-824. Doi: 10.3934/fods.2021030

This issue Previous Article An adaptation for iterative structured matrix completion Next Article

Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix

1.
Department of Statistical Modelling, Institute of Computer Science, Pod Vodárenskou věží 271/2,182 07, Prague, Czech Republic
2.
Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics, Charles University Sokolovská 83 186 75, Prague, Czech Republic
3.
Department of Mathematical and Statistical Sciences, University of Colorado Denver, PO Box 173364, Denver, CO 80217-3364, USA
4.
Department of Complex Systems, Institute of Computer Science, Pod Vodárenskou věží 271/2,182 07, Prague, Czech Republic

^* Corresponding author: Marie Turčičová
^* Corresponding author: Marie Turčičová

Received: May 2021

Early access: November 2021

Published: December 2021

This research was partially supported by TA CR grant TL01000238, NSF grant ICER-1664175 and by the project TO01000219 funded by Norway Grants and TA CR.

Abstract / Introduction Full Text(HTML) Figure(6) / Table(3) Related Papers Cited by

Abstract

We present an ensemble filtering method based on a linear model for the precision matrix (the inverse of the covariance) with the parameters determined by Score Matching Estimation. The method provides a rigorous covariance regularization when the underlying random field is Gaussian Markov. The parameters are found by solving a system of linear equations. The analysis step uses the inverse formulation of the Kalman update. Several filter versions, differing in the construction of the analysis ensemble, are proposed, as well as a Score matching version of the Extended Kalman Filter.

Keywords:

Mathematics Subject Classification: 62F12, 62H12, 62P12.

Citation:

Full Text(HTML)

Figure 1. Block band-diagonal structure of inverse covariance matrix for two-dimensional Markov fields of type $ 10\times10 $ points. There are assumed 4, 8, 12 neighbours of any gridpoint

Download: Full-size image PowerPoint slide

Figure 2. The sparsity structure of the precision matrix (a) and of the matrix $ \tilde{D} $ of the linear system for the estimation of $ \boldsymbol{\beta} $ (b)

Download: Full-size image PowerPoint slide

Figure 3. Performance illustration of EnKF, ETKF, SMEF (Algorithm 2), SMETKF (Section 4.3) and SMEF-GR (Algorithm 1) for the Lorenz 96 model, for selected combinations of ensemble size and observation error. $ N = 10, \rho = 3 $ (a), $ N = 20, \rho = 2 $ (b), $ N = 30, \rho = 1 $ (c). Observation step 2

Download: Full-size image PowerPoint slide

Figure 4. RMSE of non-ensemble filters, averaged from 50 replications. $ \rho = 2 $ (a), $ \rho = 3 $ (b). Observation step 2

Download: Full-size image PowerPoint slide

Figure 5. Trajectories of RMSE of all 50 replications, $ \rho = 2 $. ExKF(a), SMExKF (Algorithm 3)(b)

Download: Full-size image PowerPoint slide

Figure 6. RMSE, averaged over 1000 replications, $ N = 50 $, observation step 4, $ \rho = 3 $ (a), $ \rho = 4 $ (b)

Download: Full-size image PowerPoint slide

Table 1. Lorenz 96: RMSE of different filtering algorithms averaged from 1000 time steps and 50 replications of the run, initialized from different randomly selected states. The "div" columns show the number of divergent replications

		EnKF		ETKF		SMEF		SMETKF		SMEF-GR
N	$ \rho $	RMSE	div	RMSE	div	RMSE	div	RMSE	div	RMSE	div

10	1	4.31	50	3.54	50	1.00	1	0.55	0	0.80	0
10	2	4.30	50	3.63	50	0.89	0	0.84	0	1.18	13
10	3	4.27	50	3.85	50	1.52	2	1.42	0	1.59	35
20	1	2.79	48	0.45	0	0.43	0	0.40	0	0.61	0
20	2	3.55	48	0.74	0	0.62	1	0.55	0	0.87	0
20	3	3.37	48	0.97	0	1.15	8	0.82	2	1.16	3
30	1	0.44	0	0.45	0	0.41	0	0.40	0	0.56	0
30	2	0.69	0	0.68	0	0.55	0	0.55	0	0.75	0
30	3	0.95	0	0.88	0	0.74	0	0.72	2	0.94	0

| Show Table

DownLoad: CSV

Table 2. Lorenz 96: RMSE of the two non-ensemble filtering algorithms averaged from 1000 time steps and 50 replications of the run, initialized from different randomly selected states. The "div" columns show the number of divergent replications

		ExKF		SMExKF
Obs. step	$ \rho $	RMSE	div	RMSE	div

2	1	0.34	0	0.54	0
2	2	0.53	1	0.71	0
2	3	1.71	16	0.86	0

| Show Table

DownLoad: CSV

Table 3. Lorenz 96: RMSE of different filtering algorithms averaged from 1000 time steps and 50 replications of the run, initialized from different randomly selected states. The "div" columns show the number of divergent replications. Observation step 4

		EnKF		ETKF		SMEF		SMETKF		SMEF-GR
N	r	RMSE	div	RMSE	div	RMSE	div	RMSE	div	RMSE	div

30	3.0	3.46	46	3.35	40	2.49	30	3.51	34	2.53	1
30	4.0	3.56	40	3.50	41	2.63	45	3.66	43	2.67	0
40	3.0	3.55	36	3.32	24	2.05	23	3.35	39	2.34	0
40	4.0	3.75	30	3.58	23	2.42	47	3.56	42	2.49	0
50	3.0	3.85	33	3.53	15	2.02	19	3.25	35	2.18	0
50	4.0	3.98	39	3.75	28	2.22	41	3.39	44	2.36	1

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	T. W. Anderson, Asymptotically efficient estimation of covariance matrices with linear structure, Ann. Statist., 1 (1973), 135-141. doi: 10.1214/aos/1193342389.
[2]	O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Ltd., Chichester, 1978.
[3]	M. Bukal, I. Marković and I. Petrović, Score matching based assumed density filtering with the von Mises-Fisher distribution, 20th International Conference on Information Fusion (Fusion), Xi'an, China, 2017.
[4]	G. Burgers, P. J. van Leeuwen and G. Evensen, Analysis scheme in the ensemble Kalman filter, Monthly Weather Review, 126 (1998), 1719-1724. doi: 10.1175/1520-0493(1998)126<1719:ASITEK>2.0.CO;2.
[5]	A. P. Dempster, Covariance selection, Biometrics, 28 (1972), 157-175. doi: 10.2307/2528966.
[6]	P. G. M. Forbes and S. Lauritzen, Linear estimating equations for exponential families with application to Gaussian linear concentration models, Linear Algebra Appl., 473 (2015), 261-283. doi: 10.1016/j.laa.2014.08.015.
[7]	R. Furrer and T. Bengtsson, Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants, J. Multivariate Anal., 98 (2007), 227-255. doi: 10.1016/j.jmva.2006.08.003.
[8]	J. E. Gentle, Matrix Algebra. Theory, Computations and Applications in Statistics, 2$^nd$ edition, Springer Texts in Statistics, Springer, Cham, 2017. doi: 10.1007/978-3-319-64867-5.
[9]	A. Hyvärinen, Estimation of non-normalized statistical models by score matching, J. Mach. Learn. Res., 6 (2005), 695-709.
[10]	A. Hyvärinen, Some extensions of score matching, Comput. Statist. Data Anal., 51 (2007), 2499-2512. doi: 10.1016/j.csda.2006.09.003.
[11]	I. Kasanický, J. Mandel and M. Vejmelka, Spectral diagonal ensemble Kalman filters, Nonlin. Processes Geophys., 22 (2015), 485-497. doi: 10.5194/npg-22-485-2015.
[12]	M. Katzfuss, J. R. Stroud and C. K. Wikle, Understanding the ensemble Kalman filter, Amer. Statist., 70 (2016), 350-357. doi: 10.1080/00031305.2016.1141709.
[13]	S. L. Lauritzen, Graphical Models, Oxford Statistical Science Series, 17, Oxford Science Publications, The Clarendon Press, Oxford University Press, New York, 1996.
[14]	K. Law, A. Stuart and K. Zygalakis, Data assimilation. A Mathematical Introduction, Texts in Applied Mathematics, 62, Springer, Cham, 2015. doi: 10.1007/978-3-319-20325-6.
[15]	K. Law, H. Tembine and R. Tempone, Deterministic mean-field ensemble Kalman filtering, SIAM J. Sci. Comput., 38 (2016), A1251–A1279. doi: 10.1137/140984415.
[16]	F. Le Gland, V. Monbet and V.-D. Tran, Large sample asymptotics for the ensemble Kalman filter, in The Oxford Handbook of Nonlinear Filtering, Oxford Univ. Press, Oxford, 2011, 598–631.
[17]	E. L. Lehmann and J. P. Romano, Testing Statistical Hypotheses, 3$^{rd}$ edition, Springer Texts in Statistics, Springer, New York, 2005. doi: 10.1007/0-387-27605-X.
[18]	F. Lindgren, H. Rue and J. Lindström, An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach, J. R. Stat. Soc. Ser. B Stat. Methodol., 73 (2011), 423-498. doi: 10.1111/j.1467-9868.2011.00777.x.
[19]	D. M. Livings, S. L. Dance and N. K. Nichols, Unbiased ensemble square root filters, Phys. D, 237 (2008), 1021-1028. doi: 10.1016/j.physd.2008.01.005.
[20]	E. N. Lorenz, Predictability - A problem partly solved, in Predictability of Weather and Climate, Cambridge University Press, 2006, 40–58. doi: 10.1017/CBO9780511617652.004.
[21]	J. Mandel, L. Cobb and J. D. Beezley, On the convergence of the ensemble Kalman filter, Appl. Math., 56 (2011), 533-541. doi: 10.1007/s10492-011-0031-2.
[22]	E. D. Nino-Ruiz and A. Sandu, Ensemble Kalman filter implementations based on shrinkage covariance matrix estimation, Ocean Dynamics, 65 (2015), 1423-1439. doi: 10.1007/s10236-015-0888-9.
[23]	O. Pannekoucke, L. Berre and G. Desroziers, Filtering properties of wavelets for local background-error correlations, Quart. J. Roy. Meterol. Soc., 133 (2007), 363-379. doi: 10.1002/qj.33.
[24]	J. A. Rozanov, Markov random fields, and stochastic partial differential equations, Mat. Sb. (N.S.), 103 (1977), 590-613.
[25]	H. Rue and L. Held, Gaussian Markov Random Fields. Theory and Applications, Monographs on Statistics and Applied Probability, 104, Chapman & Hall/CRC, Boca Raton, FL, 2005. doi: 10.1201/9780203492024.
[26]	P. Sakov and P. Oke, A deterministic formulation of the ensemble Kalman filter: An alternative to ensemble square root filters, Tellus A, 60 (2008), 361-371. doi: 10.1111/j.1600-0870.2007.00299.x.
[27]	D. Simpson, F. Lindgren and H. Rue, Think continuous: Markovian Gaussian models in spatial statistics, Spatial Statistics, 1 (2012), 16-29. doi: 10.1016/j.spasta.2012.02.003.
[28]	A. Spantini, R. Baptista and Y. Marzouk, Coupling techniques for nonlinear ensemble filtering, preprint, arXiv: 1907.00389.
[29]	M. K. Tippett, J. L. Anderson, C. H. Bishop, T. M. Hamill and J. S. Whitaker, Ensemble square root filters, Monthly Weather Review, 131 (2003), 1485-1490. doi: 10.1175/1520-0493(2003)131<1485:ESRF>2.0.CO;2.
[30]	F. Tronarp, R. Hostettler and S. Särkkä, Continuous-discrete von Mises-Fisher filtering on $S^2$ for reference vector tracking, 21st International Conference on Information Fusion (FUSION), Cambridge, UK, 2018. doi: 10.23919/ICIF.2018.8455299.
[31]	M. Turčičová, Covariance Estimation for Filtering in High Dimension, Ph.D thesis, Charles University, 2021. Available from: http://hdl.handle.net/20.500.11956/136400.
[32]	G. Ueno and T. Tsuchiya, Covariance regularization in inverse space, Quart. J. Roy. Meterol. Soc., 135 (2009), 1133-1156. doi: 10.1002/qj.445.
[33]	A. W. van der Vaart, Asymptotic Statistics, Cambridge Series in Statistical and Probabilistic Mathematics, 3, Cambridge University Press, Cambridge, 1998. doi: 10.1017/CBO9780511802256.
[34]	X. Wang, C. H. Bishop and S. J. Julier, Which is better, an ensemble of positive–negative pairs or a centered spherical simplex ensemble?, Monthly Weather Review, 132 (2004), 1590-1605. doi: 10.1175/1520-0493(2004)132<1590:WIBAEO>2.0.CO;2.
[35]	A. Weaver and P. Courtier, Correlation modelling on the sphere using a generalized diffusion equation, Quart. J. Roy. Meterol. Soc., 127 (2001), 1815-1846. doi: 10.1002/qj.49712757518.
[36]	A. T. Weaver and I. Mirouze, On the diffusion equation and its application to isotropic and anisotropic correlation modelling in variational assimilation, Quart. J. Roy. Meterol. Soc., 139 (2013), 242-260. doi: 10.1002/qj.1955.