American Institute of Mathematical Sciences

Advanced
November  2020, 3(4): 301-308. doi: 10.3934/mfc.2020012

Aims: Average information matrix splitting

Shengxin Zhu 1,, , Tongxiang Gu 2, and  Xingping Liu 2,

1. 

Laboratory for Intelligent Computing and Financial Technology, Department of Mathematics, Xi'an Jiaotong-Liverpool University, Suzhou, 215123, China
2. 

Laboratory of Computational Physics, Institute of Applied Physics and Computational Mathematics, Beijing 100088, China

* Corresponding author: Shengxin Zhu

Received  November 2019 Revised  February 2020 Published  June 2020

Fund Project: This research is supported by Foundation of LCP(6142A05180501), Jiangsu Science and Technology Basic Research Program (BK20171237), Key Program Special Fund of XJTLU (KSF-E-21, KSF-P-02), Research Development Fund of XJTLU (RDF-2017-02-23) and partially supported by NSFC (No.11771002, 11571047, 11671049, 11671051, 6162003, and 11871339)

For linear mixed models with co-variance matrices which are not linearly dependent on variance component parameters, we prove that the average of the observed information and the Fisher information can be split into two parts. The essential part enjoys a simple and computational friendly formula, while the other part which involves a lot of computations is a random zero matrix and thus is negligible.

Keywords: Observed information matrix, Fisher information matrix, Newton method, linear mixed model, variance parameter estimation, average information.
Mathematics Subject Classification: Primary: 65J12, 62P10; Secondary: 65C60, 65F99.
Citation: Shengxin Zhu, Tongxiang Gu, Xingping Liu. Aims: Average information matrix splitting. Mathematical Foundations of Computing, 2020, 3 (4) : 301-308. doi: 10.3934/mfc.2020012
References:
[1]

Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36. doi: 10.1109/ICBDA.2019.8713193.  Google Scholar

[2]

Z. Chen, S. Zhu, Q. Niu and T. Zuo, Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.  doi: 10.1109/ACCESS.2020.2973170.  Google Scholar

[3]

B. Efron and D. V. Hinkley, Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.  doi: 10.1093/biomet/65.3.457.  Google Scholar

[4]

R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford, 1999.  Google Scholar

[5]

B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85. doi: 10.1145/3318299.3318342.  Google Scholar

[6]

A. R. Gilmour, R. Thompson and B. R. Cullis, Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.  doi: 10.2307/2533274.  Google Scholar

[7]

G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005.  Google Scholar

[8]

F. N. Gumedze and T. T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.  doi: 10.1016/j.laa.2011.04.015.  Google Scholar

[9]

A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp. doi: 10.3390/e18060236.  Google Scholar

[10]

W. Janke, D. Johnston and R. Kenna, Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.  doi: 10.1016/j.physa.2004.01.023.  Google Scholar

[11]

R. I. Jennrich and P. F. Sampson, Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.  doi: 10.2307/1267911.  Google Scholar

[12]

D. Johnson and R. Thompson, Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.  doi: 10.3168/jds.S0022-0302(95)76654-1.  Google Scholar

[13]

N. T. Longford, A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.  doi: 10.1093/biomet/74.4.817.  Google Scholar

[14]

K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97. doi: 10.1186/1297-9686-29-2-97.  Google Scholar

[15]

J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005. doi: 10.1002/0470013192.bsa302.  Google Scholar

[16]

J. W. Pratt, F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.  doi: 10.1214/aos/1176343457.  Google Scholar

[17]

M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116. doi: 10.1103/PhysRevE.84.041116.  Google Scholar

[18]

S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006.  Google Scholar

[19]

M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar

[20]

R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000. doi: 10.1007/978-3-642-05156-2.  Google Scholar

[21]

Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar

[22]

S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar

[23]

R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997.  Google Scholar

[24]

R. Zamir, A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.  doi: 10.1109/18.669301.  Google Scholar

[25]

S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar

[26]

S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar

[27]

S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar

[28]

T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar

show all references

References:
[1]

Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36. doi: 10.1109/ICBDA.2019.8713193.  Google Scholar

[2]

Z. Chen, S. Zhu, Q. Niu and T. Zuo, Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.  doi: 10.1109/ACCESS.2020.2973170.  Google Scholar

[3]

B. Efron and D. V. Hinkley, Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.  doi: 10.1093/biomet/65.3.457.  Google Scholar

[4]

R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford, 1999.  Google Scholar

[5]

B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85. doi: 10.1145/3318299.3318342.  Google Scholar

[6]

A. R. Gilmour, R. Thompson and B. R. Cullis, Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.  doi: 10.2307/2533274.  Google Scholar

[7]

G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005.  Google Scholar

[8]

F. N. Gumedze and T. T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.  doi: 10.1016/j.laa.2011.04.015.  Google Scholar

[9]

A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp. doi: 10.3390/e18060236.  Google Scholar

[10]

W. Janke, D. Johnston and R. Kenna, Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.  doi: 10.1016/j.physa.2004.01.023.  Google Scholar

[11]

R. I. Jennrich and P. F. Sampson, Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.  doi: 10.2307/1267911.  Google Scholar

[12]

D. Johnson and R. Thompson, Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.  doi: 10.3168/jds.S0022-0302(95)76654-1.  Google Scholar

[13]

N. T. Longford, A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.  doi: 10.1093/biomet/74.4.817.  Google Scholar

[14]

K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97. doi: 10.1186/1297-9686-29-2-97.  Google Scholar

[15]

J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005. doi: 10.1002/0470013192.bsa302.  Google Scholar

[16]

J. W. Pratt, F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.  doi: 10.1214/aos/1176343457.  Google Scholar

[17]

M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116. doi: 10.1103/PhysRevE.84.041116.  Google Scholar

[18]

S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006.  Google Scholar

[19]

M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar

[20]

R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000. doi: 10.1007/978-3-642-05156-2.  Google Scholar

[21]

Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar

[22]

S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar

[23]

R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997.  Google Scholar

[24]

R. Zamir, A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.  doi: 10.1109/18.669301.  Google Scholar

[25]

S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar

[26]

S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar

[27]

S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar

[28]

T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar

[1]

Nicolas Rougerie. On two properties of the Fisher information. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2020049
[2]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047
[3]

Peizhao Yu, Guoshan Zhang, Yi Zhang. Decoupling of cubic polynomial matrix systems. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 13-26. doi: 10.3934/naco.2020012
[4]

Parikshit Upadhyaya, Elias Jarlebring, Emanuel H. Rubensson. A density matrix approach to the convergence of the self-consistent field iteration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 99-115. doi: 10.3934/naco.2020018
[5]

S. Sadeghi, H. Jafari, S. Nemati. Solving fractional Advection-diffusion equation using Genocchi operational matrix based on Atangana-Baleanu derivative. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020435
[6]

Yuri Fedorov, Božidar Jovanović. Continuous and discrete Neumann systems on Stiefel varieties as matrix generalizations of the Jacobi–Mumford systems. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020375
[7]

Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020076
[8]

Sihem Guerarra. Maximum and minimum ranks and inertias of the Hermitian parts of the least rank solution of the matrix equation AXB = C. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 75-86. doi: 10.3934/naco.2020016
[9]

Jiaquan Liu, Xiangqing Liu, Zhi-Qiang Wang. Sign-changing solutions for a parameter-dependent quasilinear equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020454
[10]

Zhiyan Ding, Qin Li, Jianfeng Lu. Ensemble Kalman Inversion for nonlinear problems: Weights, consistency, and variance bounds. Foundations of Data Science, 2020  doi: 10.3934/fods.2020018
[11]

Zhenzhen Wang, Tianshou Zhou. Asymptotic behaviors and stochastic traveling waves in stochastic Fisher-KPP equations. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020323
[12]

Håkon Hoel, Gaukhar Shaimerdenova, Raúl Tempone. Multilevel Ensemble Kalman Filtering based on a sample average of independent EnKF estimators. Foundations of Data Science, 2020  doi: 10.3934/fods.2020017
[13]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444
[14]

Aihua Fan, Jörg Schmeling, Weixiao Shen. $ L^\infty $-estimation of generalized Thue-Morse trigonometric polynomials and ergodic maximization. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 297-327. doi: 10.3934/dcds.2020363
[15]

Yongge Tian, Pengyang Xie. Simultaneous optimal predictions under two seemingly unrelated linear random-effects models. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020168
[16]

Huiying Fan, Tao Ma. Parabolic equations involving Laguerre operators and weighted mixed-norm estimates. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5487-5508. doi: 10.3934/cpaa.2020249
[17]

Juan Pablo Pinasco, Mauro Rodriguez Cartabia, Nicolas Saintier. Evolutionary game theory in mixed strategies: From microscopic interactions to kinetic equations. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2020051
[18]

Jie Zhang, Yuping Duan, Yue Lu, Michael K. Ng, Huibin Chang. Bilinear constraint based ADMM for mixed Poisson-Gaussian noise removal. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020071
[19]

Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013
[20]

Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019

 Impact Factor: 

Tools

Metrics

  • PDF downloads (40)
  • HTML views (246)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences