# American Institute of Mathematical Sciences

doi: 10.3934/fods.2020018

## Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds

 1 Department of Mathematics, University of Wisconsin-Madison, Madison, WI 53706 USA 2 Department of Mathematics, Duke University, Durham, NC 27708 USA

* Corresponding author: Zhiyan Ding

Zhiyan Ding and Qin Li are supported in part by NSF CAREER DMS-1750488, NSF TRIPODS 1740707 and Wisconsin Data Science Initiative. The work of Jianfeng Lu is supported in part by National Science Foundation via grants DMS-1454939 and DMS-2012286. All three authors thank the two anonymous referees for the very helpful suggestions

Received  May 2020 Revised  July 2020 Published  November 2020

Ensemble Kalman Inversion (EnKI) [23] and Ensemble Square Root Filter (EnSRF) [36] are popular sampling methods for obtaining a target posterior distribution. They can be seem as one step (the analysis step) in the data assimilation method Ensemble Kalman Filter [17,3]. Despite their popularity, they are, however, not unbiased when the forward map is nonlinear [12,16,25]. Important Sampling (IS), on the other hand, obtains the unbiased sampling at the expense of large variance of weights, leading to slow convergence of high moments.

We propose WEnKI and WEnSRF, the weighted versions of EnKI and EnSRF in this paper. It follows the same gradient flow as that of EnKI/EnSRF with weight corrections. Compared to the classical methods, the new methods are unbiased, and compared with IS, the method has bounded weight variance. Both properties will be proved rigorously in this paper. We further discuss the stability of the underlying Fokker-Planck equation. This partially explains why EnKI, despite being inconsistent, performs well occasionally in nonlinear settings. Numerical evidence will be demonstrated at the end.

Citation: Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, doi: 10.3934/fods.2020018
##### References:

show all references

##### References:
Example $1$: from left top to bottom right: WEnKI; WEnSRF; WEnKF, as shown in Remark 1 and equation (44); IS; EnKI and EnSRF. (All evolutional equation take $\Delta t = 10^{-3}$.)
Example $2$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example $3$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example 3: $\log( {\rm{Var}}(Nw(t))+1)$ for WEnKI, WEnSRF and IS
Example $4$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example $5$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example 5: $\log( {\rm{Var}}(Nw(t))+1)$ for WEnKI, WEnSRF and IS
Error of moments estimation in Example 3
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.82 0.0056 3.88 0.0098 $\mathbb{E}|u|^2=14.90$ 14.73 0.0114 15.19 0.0192 $\mathbb{E}|u|^3=58.22$ 57.19 0.0177 59.86 0.0281 $\mathbb{E}|u|^4=229.36$ 223.79 0.0243 237.75 0.0366 $\mathbb{E}|u|^5=911.22$ 882.83 0.0312 951.95 0.0447 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.69 0.0413 3.70 0.0391 $\mathbb{E}|u|^2=14.90$ 13.66 0.0833 13.73 0.0785 $\mathbb{E}|u|^3=58.22$ 50.90 0.1258 51.35 0.1181 $\mathbb{E}|u|^4=229.36$ 190.68 0.1687 193.24 0.1575 $\mathbb{E}|u|^5=911.22$ 718.31 0.2117 732.17 0.1965 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.40 0.1156 3.52 0.0858 $\mathbb{E}|u|^2=14.90$ 11.65 0.2181 12.37 0.1699 $\mathbb{E}|u|^3=58.22$ 40.22 0.3093 43.57 0.2517 $\mathbb{E}|u|^4=229.36$ 139.72 0.3908 153.56 0.3305 $\mathbb{E}|u|^5=911.22$ 488.51 0.4639 541.71 0.4055
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.82 0.0056 3.88 0.0098 $\mathbb{E}|u|^2=14.90$ 14.73 0.0114 15.19 0.0192 $\mathbb{E}|u|^3=58.22$ 57.19 0.0177 59.86 0.0281 $\mathbb{E}|u|^4=229.36$ 223.79 0.0243 237.75 0.0366 $\mathbb{E}|u|^5=911.22$ 882.83 0.0312 951.95 0.0447 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.69 0.0413 3.70 0.0391 $\mathbb{E}|u|^2=14.90$ 13.66 0.0833 13.73 0.0785 $\mathbb{E}|u|^3=58.22$ 50.90 0.1258 51.35 0.1181 $\mathbb{E}|u|^4=229.36$ 190.68 0.1687 193.24 0.1575 $\mathbb{E}|u|^5=911.22$ 718.31 0.2117 732.17 0.1965 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.40 0.1156 3.52 0.0858 $\mathbb{E}|u|^2=14.90$ 11.65 0.2181 12.37 0.1699 $\mathbb{E}|u|^3=58.22$ 40.22 0.3093 43.57 0.2517 $\mathbb{E}|u|^4=229.36$ 139.72 0.3908 153.56 0.3305 $\mathbb{E}|u|^5=911.22$ 488.51 0.4639 541.71 0.4055
Simulation time in Example 1-3
 Case WEnKI WEnSRF EnKI EnSRF Example 1 0.362s 0.197s 0.138s 0.178s Example 2 50.041s 41.739s 26.564s 18.518s Example 3 0.198s 0.115s 0.120s 0.072s
 Case WEnKI WEnSRF EnKI EnSRF Example 1 0.362s 0.197s 0.138s 0.178s Example 2 50.041s 41.739s 26.564s 18.518s Example 3 0.198s 0.115s 0.120s 0.072s
Error of moments estimation in Example 5
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.30 0.0055 3.32 0.0017 $\mathbb{E}|u|^2=11.16$ 10.99 0.0147 11.19 0.0023 $\mathbb{E}|u|^3=38.05$ 36.99 0.0279 38.12 0.0019 $\mathbb{E}|u|^4=131.45$ 125.53 0.0451 131.47 0.0001 $\mathbb{E}|u|^5=460.56$ 429.99 0.0664 459.16 0.0030 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 2.96 0.1084 3.28 0.0112 $\mathbb{E}|u|^2=11.16$ 9.07 0.1872 11.04 0.0111 $\mathbb{E}|u|^3=38.05$ 29.17 0.2332 38.25 0.0053 $\mathbb{E}|u|^4=131.45$ 100.32 0.2369 137.43 0.0455 $\mathbb{E}|u|^5=460.56$ 379.73 0.1755 516.22 0.1208 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.40 0.1658 3.24 0.0245 $\mathbb{E}|u|^2=11.16$ 7.72 0.3077 10.50 0.0592 $\mathbb{E}|u|^3=38.05$ 21.74 0.4287 34.10 0.1037 $\mathbb{E}|u|^4=131.45$ 61.62 0.5313 110.81 0.1571 $\mathbb{E}|u|^5=460.56$ 175.99 0.6179 360.27 0.2178
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.30 0.0055 3.32 0.0017 $\mathbb{E}|u|^2=11.16$ 10.99 0.0147 11.19 0.0023 $\mathbb{E}|u|^3=38.05$ 36.99 0.0279 38.12 0.0019 $\mathbb{E}|u|^4=131.45$ 125.53 0.0451 131.47 0.0001 $\mathbb{E}|u|^5=460.56$ 429.99 0.0664 459.16 0.0030 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 2.96 0.1084 3.28 0.0112 $\mathbb{E}|u|^2=11.16$ 9.07 0.1872 11.04 0.0111 $\mathbb{E}|u|^3=38.05$ 29.17 0.2332 38.25 0.0053 $\mathbb{E}|u|^4=131.45$ 100.32 0.2369 137.43 0.0455 $\mathbb{E}|u|^5=460.56$ 379.73 0.1755 516.22 0.1208 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.40 0.1658 3.24 0.0245 $\mathbb{E}|u|^2=11.16$ 7.72 0.3077 10.50 0.0592 $\mathbb{E}|u|^3=38.05$ 21.74 0.4287 34.10 0.1037 $\mathbb{E}|u|^4=131.45$ 61.62 0.5313 110.81 0.1571 $\mathbb{E}|u|^5=460.56$ 175.99 0.6179 360.27 0.2178
 [1] Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020  doi: 10.3934/fods.2020017 [2] Peter Poláčik, Pavol Quittner. Entire and ancient solutions of a supercritical semilinear heat equation. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 413-438. doi: 10.3934/dcds.2020136 [3] Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345 [4] Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020384 [5] Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317 [6] Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079 [7] Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081 [8] Hirokazu Ninomiya. Entire solutions of the Allen–Cahn–Nagumo equation in a multi-dimensional space. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 395-412. doi: 10.3934/dcds.2020364 [9] Jiaquan Liu, Xiangqing Liu, Zhi-Qiang Wang. Sign-changing solutions for a parameter-dependent quasilinear equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020454 [10] Thierry Cazenave, Ivan Naumkin. Local smooth solutions of the nonlinear Klein-gordon equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020448 [11] Reza Chaharpashlou, Abdon Atangana, Reza Saadati. On the fuzzy stability results for fractional stochastic Volterra integral equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020432 [12] Leilei Wei, Yinnian He. A fully discrete local discontinuous Galerkin method with the generalized numerical flux to solve the tempered fractional reaction-diffusion equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020319 [13] Xin-Guang Yang, Lu Li, Xingjie Yan, Ling Ding. The structure and stability of pullback attractors for 3D Brinkman-Forchheimer equation with delay. Electronic Research Archive, 2020, 28 (4) : 1395-1418. doi: 10.3934/era.2020074 [14] Kihoon Seong. Low regularity a priori estimates for the fourth order cubic nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5437-5473. doi: 10.3934/cpaa.2020247 [15] Ahmad Z. Fino, Wenhui Chen. A global existence result for two-dimensional semilinear strongly damped wave equation with mixed nonlinearity in an exterior domain. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5387-5411. doi: 10.3934/cpaa.2020243 [16] José Luis López. A quantum approach to Keller-Segel dynamics via a dissipative nonlinear Schrödinger equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020376 [17] S. Sadeghi, H. Jafari, S. Nemati. Solving fractional Advection-diffusion equation using Genocchi operational matrix based on Atangana-Baleanu derivative. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020435 [18] Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020276 [19] Lihong Zhang, Wenwen Hou, Bashir Ahmad, Guotao Wang. Radial symmetry for logarithmic Choquard equation involving a generalized tempered fractional $p$-Laplacian. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020445 [20] Yongxiu Shi, Haitao Wan. Refined asymptotic behavior and uniqueness of large solutions to a quasilinear elliptic equation in a borderline case. Electronic Research Archive, , () : -. doi: 10.3934/era.2020119

Impact Factor: