- Previous Article
- MFC Home
- This Issue
-
Next Article
Support vector machine classifiers by non-Euclidean margins
AIMS: Average information matrix splitting
1. | Laboratory for Intelligent Computing and Financial Technology, Department of Mathematics, Xi'an Jiaotong-Liverpool University, Suzhou, 215123, China |
2. | Laboratory of Computational Physics, Institute of Applied Physics and Computational Mathematics, Beijing 100088, China |
For linear mixed models with co-variance matrices which are not linearly dependent on variance component parameters, we prove that the average of the observed information and the Fisher information can be split into two parts. The essential part enjoys a simple and computational friendly formula, while the other part which involves a lot of computations is a random zero matrix and thus is negligible.
References:
[1] |
Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36.
doi: 10.1109/ICBDA.2019.8713193. |
[2] |
Z. Chen, S. Zhu, Q. Niu and T. Zuo,
Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.
doi: 10.1109/ACCESS.2020.2973170. |
[3] |
B. Efron and D. V. Hinkley,
Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.
doi: 10.1093/biomet/65.3.457. |
[4] |
R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford,
1999. |
[5] |
B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85.
doi: 10.1145/3318299.3318342. |
[6] |
A. R. Gilmour, R. Thompson and B. R. Cullis,
Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.
doi: 10.2307/2533274. |
[7] |
G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005. |
[8] |
F. N. Gumedze and T. T. Dunne,
Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.
doi: 10.1016/j.laa.2011.04.015. |
[9] |
A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp.
doi: 10.3390/e18060236. |
[10] |
W. Janke, D. Johnston and R. Kenna,
Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.
doi: 10.1016/j.physa.2004.01.023. |
[11] |
R. I. Jennrich and P. F. Sampson,
Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.
doi: 10.2307/1267911. |
[12] |
D. Johnson and R. Thompson,
Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.
doi: 10.3168/jds.S0022-0302(95)76654-1. |
[13] |
N. T. Longford,
A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.
doi: 10.1093/biomet/74.4.817. |
[14] |
K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97.
doi: 10.1186/1297-9686-29-2-97. |
[15] |
J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005.
doi: 10.1002/0470013192.bsa302. |
[16] |
J. W. Pratt,
F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.
doi: 10.1214/aos/1176343457. |
[17] |
M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116.
doi: 10.1103/PhysRevE.84.041116. |
[18] |
S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006. |
[19] |
M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar |
[20] |
R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000.
doi: 10.1007/978-3-642-05156-2. |
[21] |
Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar |
[22] |
S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar |
[23] |
R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997. |
[24] |
R. Zamir,
A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.
doi: 10.1109/18.669301. |
[25] |
S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar |
[26] |
S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar |
[27] |
S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar |
[28] |
T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar |
show all references
References:
[1] |
Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36.
doi: 10.1109/ICBDA.2019.8713193. |
[2] |
Z. Chen, S. Zhu, Q. Niu and T. Zuo,
Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.
doi: 10.1109/ACCESS.2020.2973170. |
[3] |
B. Efron and D. V. Hinkley,
Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.
doi: 10.1093/biomet/65.3.457. |
[4] |
R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford,
1999. |
[5] |
B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85.
doi: 10.1145/3318299.3318342. |
[6] |
A. R. Gilmour, R. Thompson and B. R. Cullis,
Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.
doi: 10.2307/2533274. |
[7] |
G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005. |
[8] |
F. N. Gumedze and T. T. Dunne,
Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.
doi: 10.1016/j.laa.2011.04.015. |
[9] |
A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp.
doi: 10.3390/e18060236. |
[10] |
W. Janke, D. Johnston and R. Kenna,
Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.
doi: 10.1016/j.physa.2004.01.023. |
[11] |
R. I. Jennrich and P. F. Sampson,
Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.
doi: 10.2307/1267911. |
[12] |
D. Johnson and R. Thompson,
Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.
doi: 10.3168/jds.S0022-0302(95)76654-1. |
[13] |
N. T. Longford,
A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.
doi: 10.1093/biomet/74.4.817. |
[14] |
K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97.
doi: 10.1186/1297-9686-29-2-97. |
[15] |
J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005.
doi: 10.1002/0470013192.bsa302. |
[16] |
J. W. Pratt,
F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.
doi: 10.1214/aos/1176343457. |
[17] |
M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116.
doi: 10.1103/PhysRevE.84.041116. |
[18] |
S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006. |
[19] |
M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar |
[20] |
R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000.
doi: 10.1007/978-3-642-05156-2. |
[21] |
Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar |
[22] |
S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar |
[23] |
R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997. |
[24] |
R. Zamir,
A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.
doi: 10.1109/18.669301. |
[25] |
S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar |
[26] |
S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar |
[27] |
S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar |
[28] |
T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar |
[1] |
Juliang Zhang, Jian Chen. Information sharing in a make-to-stock supply chain. Journal of Industrial & Management Optimization, 2014, 10 (4) : 1169-1189. doi: 10.3934/jimo.2014.10.1169 |
[2] |
Tao Wu, Yu Lei, Jiao Shi, Maoguo Gong. An evolutionary multiobjective method for low-rank and sparse matrix decomposition. Big Data & Information Analytics, 2017, 2 (1) : 23-37. doi: 10.3934/bdia.2017006 |
[3] |
Charles Fulton, David Pearson, Steven Pruess. Characterization of the spectral density function for a one-sided tridiagonal Jacobi matrix operator. Conference Publications, 2013, 2013 (special) : 247-257. doi: 10.3934/proc.2013.2013.247 |
[4] |
Min Li. A three term Polak-Ribière-Polyak conjugate gradient method close to the memoryless BFGS quasi-Newton method. Journal of Industrial & Management Optimization, 2020, 16 (1) : 245-260. doi: 10.3934/jimo.2018149 |
[5] |
Yueqiang Shang, Qihui Zhang. A subgrid stabilizing postprocessed mixed finite element method for the time-dependent Navier-Stokes equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3119-3142. doi: 10.3934/dcdsb.2020222 |
[6] |
Xingchun Wang, Yongjin Wang. Variance-optimal hedging for target volatility options. Journal of Industrial & Management Optimization, 2014, 10 (1) : 207-218. doi: 10.3934/jimo.2014.10.207 |
[7] |
Jiaquan Liu, Xiangqing Liu, Zhi-Qiang Wang. Sign-changing solutions for a parameter-dependent quasilinear equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1779-1799. doi: 10.3934/dcdss.2020454 |
[8] |
Hsin-Lun Li. Mixed Hegselmann-Krause dynamics. Discrete & Continuous Dynamical Systems - B, 2021 doi: 10.3934/dcdsb.2021084 |
[9] |
Jianping Gao, Shangjiang Guo, Wenxian Shen. Persistence and time periodic positive solutions of doubly nonlocal Fisher-KPP equations in time periodic and space heterogeneous media. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2645-2676. doi: 10.3934/dcdsb.2020199 |
[10] |
Demetres D. Kouvatsos, Jumma S. Alanazi, Kevin Smith. A unified ME algorithm for arbitrary open QNMs with mixed blocking mechanisms. Numerical Algebra, Control & Optimization, 2011, 1 (4) : 781-816. doi: 10.3934/naco.2011.1.781 |
[11] |
Armin Lechleiter, Tobias Rienmüller. Factorization method for the inverse Stokes problem. Inverse Problems & Imaging, 2013, 7 (4) : 1271-1293. doi: 10.3934/ipi.2013.7.1271 |
[12] |
Carlos Fresneda-Portillo, Sergey E. Mikhailov. Analysis of Boundary-Domain Integral Equations to the mixed BVP for a compressible stokes system with variable viscosity. Communications on Pure & Applied Analysis, 2019, 18 (6) : 3059-3088. doi: 10.3934/cpaa.2019137 |
[13] |
Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437 |
[14] |
Guillaume Bal, Wenjia Jing. Homogenization and corrector theory for linear transport in random media. Discrete & Continuous Dynamical Systems, 2010, 28 (4) : 1311-1343. doi: 10.3934/dcds.2010.28.1311 |
[15] |
Nizami A. Gasilov. Solving a system of linear differential equations with interval coefficients. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2739-2747. doi: 10.3934/dcdsb.2020203 |
[16] |
Qiang Guo, Dong Liang. An adaptive wavelet method and its analysis for parabolic equations. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 327-345. doi: 10.3934/naco.2013.3.327 |
[17] |
Quan Hai, Shutang Liu. Mean-square delay-distribution-dependent exponential synchronization of chaotic neural networks with mixed random time-varying delays and restricted disturbances. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3097-3118. doi: 10.3934/dcdsb.2020221 |
[18] |
Alexander A. Davydov, Massimo Giulietti, Stefano Marcugini, Fernanda Pambianco. Linear nonbinary covering codes and saturating sets in projective spaces. Advances in Mathematics of Communications, 2011, 5 (1) : 119-147. doi: 10.3934/amc.2011.5.119 |
[19] |
W. Cary Huffman. On the theory of $\mathbb{F}_q$-linear $\mathbb{F}_{q^t}$-codes. Advances in Mathematics of Communications, 2013, 7 (3) : 349-378. doi: 10.3934/amc.2013.7.349 |
[20] |
Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]