# American Institute of Mathematical Sciences

doi: 10.3934/fods.2020018

## Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds

 1 Department of Mathematics, University of Wisconsin-Madison, Madison, WI 53706 USA 2 Department of Mathematics, Duke University, Durham, NC 27708 USA

* Corresponding author: Zhiyan Ding

Zhiyan Ding and Qin Li are supported in part by NSF CAREER DMS-1750488, NSF TRIPODS 1740707 and Wisconsin Data Science Initiative. The work of Jianfeng Lu is supported in part by National Science Foundation via grants DMS-1454939 and DMS-2012286. All three authors thank the two anonymous referees for the very helpful suggestions

Received  May 2020 Revised  July 2020 Published  November 2020

Ensemble Kalman Inversion (EnKI) [23] and Ensemble Square Root Filter (EnSRF) [36] are popular sampling methods for obtaining a target posterior distribution. They can be seem as one step (the analysis step) in the data assimilation method Ensemble Kalman Filter [17,3]. Despite their popularity, they are, however, not unbiased when the forward map is nonlinear [12,16,25]. Important Sampling (IS), on the other hand, obtains the unbiased sampling at the expense of large variance of weights, leading to slow convergence of high moments.

We propose WEnKI and WEnSRF, the weighted versions of EnKI and EnSRF in this paper. It follows the same gradient flow as that of EnKI/EnSRF with weight corrections. Compared to the classical methods, the new methods are unbiased, and compared with IS, the method has bounded weight variance. Both properties will be proved rigorously in this paper. We further discuss the stability of the underlying Fokker-Planck equation. This partially explains why EnKI, despite being inconsistent, performs well occasionally in nonlinear settings. Numerical evidence will be demonstrated at the end.

Citation: Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, doi: 10.3934/fods.2020018
##### References:

show all references

##### References:
Example $1$: from left top to bottom right: WEnKI; WEnSRF; WEnKF, as shown in Remark 1 and equation (44); IS; EnKI and EnSRF. (All evolutional equation take $\Delta t = 10^{-3}$.)
Example $2$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example $3$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example 3: $\log( {\rm{Var}}(Nw(t))+1)$ for WEnKI, WEnSRF and IS
Example $4$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example $5$: from left top to bottom right: WEnKI; WEnSRF; WEnKF; IS; EnKI and EnSRF
Example 5: $\log( {\rm{Var}}(Nw(t))+1)$ for WEnKI, WEnSRF and IS
Error of moments estimation in Example 3
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.82 0.0056 3.88 0.0098 $\mathbb{E}|u|^2=14.90$ 14.73 0.0114 15.19 0.0192 $\mathbb{E}|u|^3=58.22$ 57.19 0.0177 59.86 0.0281 $\mathbb{E}|u|^4=229.36$ 223.79 0.0243 237.75 0.0366 $\mathbb{E}|u|^5=911.22$ 882.83 0.0312 951.95 0.0447 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.69 0.0413 3.70 0.0391 $\mathbb{E}|u|^2=14.90$ 13.66 0.0833 13.73 0.0785 $\mathbb{E}|u|^3=58.22$ 50.90 0.1258 51.35 0.1181 $\mathbb{E}|u|^4=229.36$ 190.68 0.1687 193.24 0.1575 $\mathbb{E}|u|^5=911.22$ 718.31 0.2117 732.17 0.1965 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.40 0.1156 3.52 0.0858 $\mathbb{E}|u|^2=14.90$ 11.65 0.2181 12.37 0.1699 $\mathbb{E}|u|^3=58.22$ 40.22 0.3093 43.57 0.2517 $\mathbb{E}|u|^4=229.36$ 139.72 0.3908 153.56 0.3305 $\mathbb{E}|u|^5=911.22$ 488.51 0.4639 541.71 0.4055
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.82 0.0056 3.88 0.0098 $\mathbb{E}|u|^2=14.90$ 14.73 0.0114 15.19 0.0192 $\mathbb{E}|u|^3=58.22$ 57.19 0.0177 59.86 0.0281 $\mathbb{E}|u|^4=229.36$ 223.79 0.0243 237.75 0.0366 $\mathbb{E}|u|^5=911.22$ 882.83 0.0312 951.95 0.0447 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.69 0.0413 3.70 0.0391 $\mathbb{E}|u|^2=14.90$ 13.66 0.0833 13.73 0.0785 $\mathbb{E}|u|^3=58.22$ 50.90 0.1258 51.35 0.1181 $\mathbb{E}|u|^4=229.36$ 190.68 0.1687 193.24 0.1575 $\mathbb{E}|u|^5=911.22$ 718.31 0.2117 732.17 0.1965 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.84$ 3.40 0.1156 3.52 0.0858 $\mathbb{E}|u|^2=14.90$ 11.65 0.2181 12.37 0.1699 $\mathbb{E}|u|^3=58.22$ 40.22 0.3093 43.57 0.2517 $\mathbb{E}|u|^4=229.36$ 139.72 0.3908 153.56 0.3305 $\mathbb{E}|u|^5=911.22$ 488.51 0.4639 541.71 0.4055
Simulation time in Example 1-3
 Case WEnKI WEnSRF EnKI EnSRF Example 1 0.362s 0.197s 0.138s 0.178s Example 2 50.041s 41.739s 26.564s 18.518s Example 3 0.198s 0.115s 0.120s 0.072s
 Case WEnKI WEnSRF EnKI EnSRF Example 1 0.362s 0.197s 0.138s 0.178s Example 2 50.041s 41.739s 26.564s 18.518s Example 3 0.198s 0.115s 0.120s 0.072s
Error of moments estimation in Example 5
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.30 0.0055 3.32 0.0017 $\mathbb{E}|u|^2=11.16$ 10.99 0.0147 11.19 0.0023 $\mathbb{E}|u|^3=38.05$ 36.99 0.0279 38.12 0.0019 $\mathbb{E}|u|^4=131.45$ 125.53 0.0451 131.47 0.0001 $\mathbb{E}|u|^5=460.56$ 429.99 0.0664 459.16 0.0030 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 2.96 0.1084 3.28 0.0112 $\mathbb{E}|u|^2=11.16$ 9.07 0.1872 11.04 0.0111 $\mathbb{E}|u|^3=38.05$ 29.17 0.2332 38.25 0.0053 $\mathbb{E}|u|^4=131.45$ 100.32 0.2369 137.43 0.0455 $\mathbb{E}|u|^5=460.56$ 379.73 0.1755 516.22 0.1208 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.40 0.1658 3.24 0.0245 $\mathbb{E}|u|^2=11.16$ 7.72 0.3077 10.50 0.0592 $\mathbb{E}|u|^3=38.05$ 21.74 0.4287 34.10 0.1037 $\mathbb{E}|u|^4=131.45$ 61.62 0.5313 110.81 0.1571 $\mathbb{E}|u|^5=460.56$ 175.99 0.6179 360.27 0.2178
 WEnKI WEnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.30 0.0055 3.32 0.0017 $\mathbb{E}|u|^2=11.16$ 10.99 0.0147 11.19 0.0023 $\mathbb{E}|u|^3=38.05$ 36.99 0.0279 38.12 0.0019 $\mathbb{E}|u|^4=131.45$ 125.53 0.0451 131.47 0.0001 $\mathbb{E}|u|^5=460.56$ 429.99 0.0664 459.16 0.0030 EnKI EnSRF Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 2.96 0.1084 3.28 0.0112 $\mathbb{E}|u|^2=11.16$ 9.07 0.1872 11.04 0.0111 $\mathbb{E}|u|^3=38.05$ 29.17 0.2332 38.25 0.0053 $\mathbb{E}|u|^4=131.45$ 100.32 0.2369 137.43 0.0455 $\mathbb{E}|u|^5=460.56$ 379.73 0.1755 516.22 0.1208 WEnKF IS Moments Est. Re. Error Est. Re. Error $\mathbb{E}|u|^1=3.32$ 3.40 0.1658 3.24 0.0245 $\mathbb{E}|u|^2=11.16$ 7.72 0.3077 10.50 0.0592 $\mathbb{E}|u|^3=38.05$ 21.74 0.4287 34.10 0.1037 $\mathbb{E}|u|^4=131.45$ 61.62 0.5313 110.81 0.1571 $\mathbb{E}|u|^5=460.56$ 175.99 0.6179 360.27 0.2178
 [1] Seung-Yeal Ha, Myeongju Kang, Bora Moon. Collective behaviors of a Winfree ensemble on an infinite cylinder. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2749-2779. doi: 10.3934/dcdsb.2020204 [2] Chih-Chiang Fang. Bayesian decision making in determining optimal leased term and preventive maintenance scheme for leased facilities. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020127 [3] Xiaomao Deng, Xiao-Chuan Cai, Jun Zou. A parallel space-time domain decomposition method for unsteady source inversion problems. Inverse Problems & Imaging, 2015, 9 (4) : 1069-1091. doi: 10.3934/ipi.2015.9.1069 [4] Vladimir Georgiev, Sandra Lucente. Focusing nlkg equation with singular potential. Communications on Pure & Applied Analysis, 2018, 17 (4) : 1387-1406. doi: 10.3934/cpaa.2018068 [5] Daoyin He, Ingo Witt, Huicheng Yin. On the strauss index of semilinear tricomi equation. Communications on Pure & Applied Analysis, 2020, 19 (10) : 4817-4838. doi: 10.3934/cpaa.2020213 [6] Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437 [7] Simone Cacace, Maurizio Falcone. A dynamic domain decomposition for the eikonal-diffusion equation. Discrete & Continuous Dynamical Systems - S, 2016, 9 (1) : 109-123. doi: 10.3934/dcdss.2016.9.109 [8] Naeem M. H. Alkoumi, Pedro J. Torres. Estimates on the number of limit cycles of a generalized Abel equation. Discrete & Continuous Dynamical Systems - A, 2011, 31 (1) : 25-34. doi: 10.3934/dcds.2011.31.25 [9] Jumpei Inoue, Kousuke Kuto. On the unboundedness of the ratio of species and resources for the diffusive logistic equation. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2441-2450. doi: 10.3934/dcdsb.2020186 [10] Wentao Huang, Jianlin Xiang. Soliton solutions for a quasilinear Schrödinger equation with critical exponent. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1309-1333. doi: 10.3934/cpaa.2016.15.1309 [11] Kin Ming Hui, Soojung Kim. Asymptotic large time behavior of singular solutions of the fast diffusion equation. Discrete & Continuous Dynamical Systems - A, 2017, 37 (11) : 5943-5977. doi: 10.3934/dcds.2017258 [12] Thierry Cazenave, Ivan Naumkin. Local smooth solutions of the nonlinear Klein-gordon equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1649-1672. doi: 10.3934/dcdss.2020448 [13] Jiaquan Liu, Xiangqing Liu, Zhi-Qiang Wang. Sign-changing solutions for a parameter-dependent quasilinear equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1779-1799. doi: 10.3934/dcdss.2020454 [14] Murat Uzunca, Ayşe Sarıaydın-Filibelioǧlu. Adaptive discontinuous galerkin finite elements for advective Allen-Cahn equation. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 269-281. doi: 10.3934/naco.2020025 [15] Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 117-139. doi: 10.3934/dcdsb.2019175 [16] Ademir Fernando Pazoto, Lionel Rosier. Uniform stabilization in weighted Sobolev spaces for the KdV equation posed on the half-line. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1511-1535. doi: 10.3934/dcdsb.2010.14.1511 [17] Abdulrazzaq T. Abed, Azzam S. Y. Aladool. Applying particle swarm optimization based on Padé approximant to solve ordinary differential equation. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021008 [18] Vo Anh Khoa, Thi Kim Thoa Thieu, Ekeoma Rowland Ijioma. On a pore-scale stationary diffusion equation: Scaling effects and correctors for the homogenization limit. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2451-2477. doi: 10.3934/dcdsb.2020190 [19] Zhiming Guo, Zhi-Chun Yang, Xingfu Zou. Existence and uniqueness of positive solution to a non-local differential equation with homogeneous Dirichlet boundary condition---A non-monotone case. Communications on Pure & Applied Analysis, 2012, 11 (5) : 1825-1838. doi: 10.3934/cpaa.2012.11.1825 [20] Ka Luen Cheung, Man Chun Leung. Asymptotic behavior of positive solutions of the equation $\Delta u + K u^{\frac{n+2}{n-2}} = 0$ in $IR^n$ and positive scalar curvature. Conference Publications, 2001, 2001 (Special) : 109-120. doi: 10.3934/proc.2001.2001.109

Impact Factor: