American Institute of Mathematical Sciences

Advanced
November  2020, 3(4): 301-308. doi: 10.3934/mfc.2020012

AIMS: Average information matrix splitting

Shengxin Zhu 1,, , Tongxiang Gu 2, and  Xingping Liu 2,

1. 

Laboratory for Intelligent Computing and Financial Technology, Department of Mathematics, Xi'an Jiaotong-Liverpool University, Suzhou, 215123, China
2. 

Laboratory of Computational Physics, Institute of Applied Physics and Computational Mathematics, Beijing 100088, China

* Corresponding author: Shengxin Zhu

Received  November 2019 Revised  February 2020 Published  June 2020

Fund Project: This research is supported by Foundation of LCP(6142A05180501), Jiangsu Science and Technology Basic Research Program (BK20171237), Key Program Special Fund of XJTLU (KSF-E-21, KSF-P-02), Research Development Fund of XJTLU (RDF-2017-02-23) and partially supported by NSFC (No.11771002, 11571047, 11671049, 11671051, 6162003, and 11871339)

For linear mixed models with co-variance matrices which are not linearly dependent on variance component parameters, we prove that the average of the observed information and the Fisher information can be split into two parts. The essential part enjoys a simple and computational friendly formula, while the other part which involves a lot of computations is a random zero matrix and thus is negligible.

Keywords: Observed information matrix, Fisher information matrix, Newton method, linear mixed model, variance parameter estimation, average information.
Mathematics Subject Classification: Primary: 65J12, 62P10; Secondary: 65C60, 65F99.
Citation: Shengxin Zhu, Tongxiang Gu, Xingping Liu. AIMS: Average information matrix splitting. Mathematical Foundations of Computing, 2020, 3 (4) : 301-308. doi: 10.3934/mfc.2020012
References:
[1]

Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36. doi: 10.1109/ICBDA.2019.8713193.  Google Scholar

[2]

Z. Chen, S. Zhu, Q. Niu and T. Zuo, Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.  doi: 10.1109/ACCESS.2020.2973170.  Google Scholar

[3]

B. Efron and D. V. Hinkley, Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.  doi: 10.1093/biomet/65.3.457.  Google Scholar

[4]

R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford, 1999.  Google Scholar

[5]

B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85. doi: 10.1145/3318299.3318342.  Google Scholar

[6]

A. R. Gilmour, R. Thompson and B. R. Cullis, Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.  doi: 10.2307/2533274.  Google Scholar

[7]

G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005.  Google Scholar

[8]

F. N. Gumedze and T. T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.  doi: 10.1016/j.laa.2011.04.015.  Google Scholar

[9]

A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp. doi: 10.3390/e18060236.  Google Scholar

[10]

W. Janke, D. Johnston and R. Kenna, Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.  doi: 10.1016/j.physa.2004.01.023.  Google Scholar

[11]

R. I. Jennrich and P. F. Sampson, Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.  doi: 10.2307/1267911.  Google Scholar

[12]

D. Johnson and R. Thompson, Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.  doi: 10.3168/jds.S0022-0302(95)76654-1.  Google Scholar

[13]

N. T. Longford, A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.  doi: 10.1093/biomet/74.4.817.  Google Scholar

[14]

K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97. doi: 10.1186/1297-9686-29-2-97.  Google Scholar

[15]

J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005. doi: 10.1002/0470013192.bsa302.  Google Scholar

[16]

J. W. Pratt, F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.  doi: 10.1214/aos/1176343457.  Google Scholar

[17]

M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116. doi: 10.1103/PhysRevE.84.041116.  Google Scholar

[18]

S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006.  Google Scholar

[19]

M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar

[20]

R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000. doi: 10.1007/978-3-642-05156-2.  Google Scholar

[21]

Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar

[22]

S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar

[23]

R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997.  Google Scholar

[24]

R. Zamir, A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.  doi: 10.1109/18.669301.  Google Scholar

[25]

S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar

[26]

S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar

[27]

S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar

[28]

T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar

show all references

References:
[1]

Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36. doi: 10.1109/ICBDA.2019.8713193.  Google Scholar

[2]

Z. Chen, S. Zhu, Q. Niu and T. Zuo, Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.  doi: 10.1109/ACCESS.2020.2973170.  Google Scholar

[3]

B. Efron and D. V. Hinkley, Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.  doi: 10.1093/biomet/65.3.457.  Google Scholar

[4]

R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford, 1999.  Google Scholar

[5]

B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85. doi: 10.1145/3318299.3318342.  Google Scholar

[6]

A. R. Gilmour, R. Thompson and B. R. Cullis, Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.  doi: 10.2307/2533274.  Google Scholar

[7]

G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005.  Google Scholar

[8]

F. N. Gumedze and T. T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.  doi: 10.1016/j.laa.2011.04.015.  Google Scholar

[9]

A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp. doi: 10.3390/e18060236.  Google Scholar

[10]

W. Janke, D. Johnston and R. Kenna, Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.  doi: 10.1016/j.physa.2004.01.023.  Google Scholar

[11]

R. I. Jennrich and P. F. Sampson, Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.  doi: 10.2307/1267911.  Google Scholar

[12]

D. Johnson and R. Thompson, Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.  doi: 10.3168/jds.S0022-0302(95)76654-1.  Google Scholar

[13]

N. T. Longford, A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.  doi: 10.1093/biomet/74.4.817.  Google Scholar

[14]

K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97. doi: 10.1186/1297-9686-29-2-97.  Google Scholar

[15]

J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005. doi: 10.1002/0470013192.bsa302.  Google Scholar

[16]

J. W. Pratt, F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.  doi: 10.1214/aos/1176343457.  Google Scholar

[17]

M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116. doi: 10.1103/PhysRevE.84.041116.  Google Scholar

[18]

S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006.  Google Scholar

[19]

M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar

[20]

R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000. doi: 10.1007/978-3-642-05156-2.  Google Scholar

[21]

Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar

[22]

S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar

[23]

R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997.  Google Scholar

[24]

R. Zamir, A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.  doi: 10.1109/18.669301.  Google Scholar

[25]

S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar

[26]

S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar

[27]

S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar

[28]

T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar

[1]

Juliang Zhang, Jian Chen. Information sharing in a make-to-stock supply chain. Journal of Industrial & Management Optimization, 2014, 10 (4) : 1169-1189. doi: 10.3934/jimo.2014.10.1169
[2]

Tao Wu, Yu Lei, Jiao Shi, Maoguo Gong. An evolutionary multiobjective method for low-rank and sparse matrix decomposition. Big Data & Information Analytics, 2017, 2 (1) : 23-37. doi: 10.3934/bdia.2017006
[3]

Charles Fulton, David Pearson, Steven Pruess. Characterization of the spectral density function for a one-sided tridiagonal Jacobi matrix operator. Conference Publications, 2013, 2013 (special) : 247-257. doi: 10.3934/proc.2013.2013.247
[4]

Min Li. A three term Polak-Ribière-Polyak conjugate gradient method close to the memoryless BFGS quasi-Newton method. Journal of Industrial & Management Optimization, 2020, 16 (1) : 245-260. doi: 10.3934/jimo.2018149
[5]

Yueqiang Shang, Qihui Zhang. A subgrid stabilizing postprocessed mixed finite element method for the time-dependent Navier-Stokes equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3119-3142. doi: 10.3934/dcdsb.2020222
[6]

Xingchun Wang, Yongjin Wang. Variance-optimal hedging for target volatility options. Journal of Industrial & Management Optimization, 2014, 10 (1) : 207-218. doi: 10.3934/jimo.2014.10.207
[7]

Jiaquan Liu, Xiangqing Liu, Zhi-Qiang Wang. Sign-changing solutions for a parameter-dependent quasilinear equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1779-1799. doi: 10.3934/dcdss.2020454
[8]

Hsin-Lun Li. Mixed Hegselmann-Krause dynamics. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021084
[9]

Jianping Gao, Shangjiang Guo, Wenxian Shen. Persistence and time periodic positive solutions of doubly nonlocal Fisher-KPP equations in time periodic and space heterogeneous media. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2645-2676. doi: 10.3934/dcdsb.2020199
[10]

Demetres D. Kouvatsos, Jumma S. Alanazi, Kevin Smith. A unified ME algorithm for arbitrary open QNMs with mixed blocking mechanisms. Numerical Algebra, Control & Optimization, 2011, 1 (4) : 781-816. doi: 10.3934/naco.2011.1.781
[11]

Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems & Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271
[12]

Carlos Fresneda-Portillo, Sergey E. Mikhailov. Analysis of Boundary-Domain Integral Equations to the mixed BVP for a compressible stokes system with variable viscosity. Communications on Pure & Applied Analysis, 2019, 18 (6) : 3059-3088. doi: 10.3934/cpaa.2019137
[13]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[14]

Guillaume Bal, Wenjia Jing. Homogenization and corrector theory for linear transport in random media. Discrete & Continuous Dynamical Systems, 2010, 28 (4) : 1311-1343. doi: 10.3934/dcds.2010.28.1311
[15]

Nizami A. Gasilov. Solving a system of linear differential equations with interval coefficients. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2739-2747. doi: 10.3934/dcdsb.2020203
[16]

Qiang Guo, Dong Liang. An adaptive wavelet method and its analysis for parabolic equations. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 327-345. doi: 10.3934/naco.2013.3.327
[17]

Quan Hai, Shutang Liu. Mean-square delay-distribution-dependent exponential synchronization of chaotic neural networks with mixed random time-varying delays and restricted disturbances. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3097-3118. doi: 10.3934/dcdsb.2020221
[18]

Alexander A. Davydov, Massimo Giulietti, Stefano Marcugini, Fernanda Pambianco. Linear nonbinary covering codes and saturating sets in projective spaces. Advances in Mathematics of Communications, 2011, 5 (1) : 119-147. doi: 10.3934/amc.2011.5.119
[19]

W. Cary Huffman. On the theory of $\mathbb{F}_q$-linear $\mathbb{F}_{q^t}$-codes. Advances in Mathematics of Communications, 2013, 7 (3) : 349-378. doi: 10.3934/amc.2013.7.349
[20]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399

 Impact Factor: 

Tools

Metrics

  • PDF downloads (84)
  • HTML views (350)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences