Figure 1.
Time complexity m for achieving relative errors
-
Previous Article
Stochastic AUC optimization with general loss
- CPAA Home
- This Issue
-
Next Article
Kernel-based maximum correntropy criterion with gradient descent method
August
2020, 19(8): 4179-4189.
doi: 10.3934/cpaa.2020187
A numerical method to compute Fisher information for a special case of heterogeneous negative binomial regression
| 1. | Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China |
| 2. | Department of Sociology, The University of British Columbia, V6T 1Z1, Vancouver, BC, Canada |
| 3. | Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China |
| 4. | Department of Sociology and Social Science Research Institute, Duke University, 27708, Durham, NC, USA |
Negative binomial regression has been widely applied in various research settings to account for counts with overdispersion. Yet, when the gamma scale parameter, $ \nu $, is parameterized, there is no direct algorithmic solution to the Fisher Information matrix of the associated heterogeneous negative binomial regression, which seriously limits its applications to a wide range of complex problems. In this research, we propose a numerical method to calculate the Fisher information of heterogeneous negative binomial regression and accordingly develop a preliminary framework for analyzing incomplete counts with overdispersion. This method is implemented in R and illustrated using an empirical example of teenage drug use in America.
Keywords:
Regression analysis,
incomplete counts,
overdispersion,
heterogeneous negative binomial regression,
Fisher information,
gamma scale parameter.
Mathematics Subject Classification:
Primary: 62J12; Secondary: 49M15.
Citation:
Xin Guo, Qiang Fu, Yue Wang, Kenneth C. Land. A numerical method to compute Fisher information for a special case of heterogeneous negative binomial regression. Communications on Pure and Applied Analysis,
2020, 19
(8)
: 4179-4189.
doi: 10.3934/cpaa.2020187
![]()
References:
| [1] |
P. D. Allison and R. P. Waterman,
Fixed–effects negative binomial regression models, Sociol. Methodol., 32 (2002), 247-265.
doi: 10.1111/1467-9531.00117. |
| [2] |
B. M. Bolker, M. E. Brooks, C. J. Clark, S. W. Geange, J. R. Poulsen, M. H. H. Stevens and J. S. S. White,
Generalized linear mixed models: a practical guide for ecology and evolution, Trends Ecol. Evol., 24 (2009), 127-135.
doi: 10.1016/j.tree.2008.10.008. |
| [3] |
A. C. Cameron and P. K. Trivedi, Regression analysis of count data, vol. 53, Cambridge University Press, 2013.
doi: 10.1017/CBO9781139013567.
|
| [4] |
A. C. Cameron and F. A. Windmeijer,
R-squared measures for count data regression models with applications to health-care utilization, J. Busin. Econ. Statist., 14 (1996), 209-220.
|
| [5] |
B. Efron and D. V. Hinkley,
Assessing the accuracy of the maximum likelihood estimator: observed versus expected Fisher information, Biometrika, 65 (1978), 457-487.
doi: 10.1093/biomet/65.3.457. |
| [6] |
S. Ehsan Saffari, R. Adnan and W. Greene,
Hurdle negative binomial regression model with right censored count data, SORT Statist. Oper. Res. Trans., 36 (2012), 0181-194.
|
| [7] |
K. V. Finn,
Patterns of alcohol and marijuana use at school, J. Res. Adol., 16 (2006), 69-77.
|
| [8] |
R. A. Fisher,
The negative binomial distribution, Ann. Eugen., 11 (1941), 182-187.
|
| [9] |
Q. Fu, X. Guo and K. C. Land,
A Poisson-multinomial mixture approach to grouped and right-censored counts, Commun. Statist. Theory Meth., 47 (2018), 427-447.
doi: 10.1080/03610926.2017.1303736. |
| [10] |
Q. Fu, X. Guo and K. C. Land, Optimizing count responses in surveys: A machine-learning approach, Sociol. Meth. Res., (2018).
doi: 10.1177/0049124117747302. |
| [11] |
Q. Fu, K. C. Land and V. L. Lamb,
Bullying victimization, socioeconomic status and behavioral characteristics of 12th graders in the united states, 1989 to 2009: Repetitive trends and persistent risk differentials, Child Indi. Res., 6 (2013), 1-21.
doi: 10.1007/s12187-012-9152-8. |
| [12] |
Q. Fu, K. C. Land and V. L. Lamb,
Violent physical bullying victimization at school: has there been a recent increase in exposure or intensity? an age-period-cohort analysis in the united states, 1991 to 2012, Child Indi. Res., 9 (2016), 485-513.
|
| [13] |
Q. Fu, C. Wu, H. Liu, Z. Shi and J. Gu,
Live like mosquitoes: Hukou, rural–urban disparity, and depression, Chin. J. Sociol., 4 (2018), 56-78.
|
| [14] |
W. H. Greene, Accounting for excess zeros and sample selection in Poisson and negative binomial regression models, in NYU working paper no. EC-94-10. |
| [15] |
R. M. Groves, F. J. Fowler Jr, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau, Survey Methodology, vol. 561, John Wiley & Sons, 2011. |
| [16] |
J. M. Hilbe, Negative Binomial Regression, 2$^nd$ edition, Cambridge University Press, Cambridge, 2011.
doi: 10.1017/CBO9780511973420.
|
| [17] |
R. A. Horn and C. R. Johnson, Matrix analysis, 2$^nd$ edition, Cambridge University Press, Cambridge, 2013.
|
| [18] |
L. D. Johnston, P. M. O'Malley and J. G. Bachman,
Bachman, Monitoring the Future: National results on adolescent drug use: Overview of key findings, Focus, 1 (2003), 213-234.
|
| [19] |
L. D. Johnston, P. M. O'Malley, R. A. Miech, J. G. Bachman and J. E. Schulenberg, Monitoring the future national survey results on drug use, 1975–2016: Overview, key findings on adolescent drug use, 2017. Available from: https://files.eric.ed.gov/fulltext/ED578534.pdf. |
| [20] |
L. D. Johnston, P. M. O'Malley, R. A. Miech, J. G. Bachman and J. E. Schulenberg, Monitoring the Future national survey results on drug use, 1975-2016: Overview, key findings on adolescent drug use, Inst. Social Res.. |
| [21] |
F. Kunstner, L. Balles and P. Hennig, Limitations of the empirical Fisher approximation, preprint, arXiv: 1905.12558. |
| [22] |
K. C. Land, P. L. McCall and D. S. Nagin,
A comparison of Poisson, negative binomial, and semiparametric mixed Poisson regression models: With empirical applications to criminal careers data, Sociol. Meth. Res., 24 (1996), 387-442.
|
| [23] |
E. L. Lehmann and G. Casella, Theory of Point Estimation, 2$^{nd}$ edition, Springer Texts in Statistics, Springer-Verlag, New York, 1998. |
| [24] |
L. R. Pacek, R. J. Malcolm and S. S. Martins,
Race/ethnicity differences between alcohol, marijuana, and co-occurring alcohol and marijuana use disorders and their association with public health and social problems using a national sample, Amer. Addi., 21 (2012), 435-444.
|
| [25] |
W. W. Piegorsch,
Maximum likelihood estimation for the negative binomial dispersion parameter, Biometrics, 46 (1990), 863-867.
doi: 10.2307/2532104. |
show all references
References:
| [1] |
P. D. Allison and R. P. Waterman,
Fixed–effects negative binomial regression models, Sociol. Methodol., 32 (2002), 247-265.
doi: 10.1111/1467-9531.00117. |
| [2] |
B. M. Bolker, M. E. Brooks, C. J. Clark, S. W. Geange, J. R. Poulsen, M. H. H. Stevens and J. S. S. White,
Generalized linear mixed models: a practical guide for ecology and evolution, Trends Ecol. Evol., 24 (2009), 127-135.
doi: 10.1016/j.tree.2008.10.008. |
| [3] |
A. C. Cameron and P. K. Trivedi, Regression analysis of count data, vol. 53, Cambridge University Press, 2013.
doi: 10.1017/CBO9781139013567.
|
| [4] |
A. C. Cameron and F. A. Windmeijer,
R-squared measures for count data regression models with applications to health-care utilization, J. Busin. Econ. Statist., 14 (1996), 209-220.
|
| [5] |
B. Efron and D. V. Hinkley,
Assessing the accuracy of the maximum likelihood estimator: observed versus expected Fisher information, Biometrika, 65 (1978), 457-487.
doi: 10.1093/biomet/65.3.457. |
| [6] |
S. Ehsan Saffari, R. Adnan and W. Greene,
Hurdle negative binomial regression model with right censored count data, SORT Statist. Oper. Res. Trans., 36 (2012), 0181-194.
|
| [7] |
K. V. Finn,
Patterns of alcohol and marijuana use at school, J. Res. Adol., 16 (2006), 69-77.
|
| [8] |
R. A. Fisher,
The negative binomial distribution, Ann. Eugen., 11 (1941), 182-187.
|
| [9] |
Q. Fu, X. Guo and K. C. Land,
A Poisson-multinomial mixture approach to grouped and right-censored counts, Commun. Statist. Theory Meth., 47 (2018), 427-447.
doi: 10.1080/03610926.2017.1303736. |
| [10] |
Q. Fu, X. Guo and K. C. Land, Optimizing count responses in surveys: A machine-learning approach, Sociol. Meth. Res., (2018).
doi: 10.1177/0049124117747302. |
| [11] |
Q. Fu, K. C. Land and V. L. Lamb,
Bullying victimization, socioeconomic status and behavioral characteristics of 12th graders in the united states, 1989 to 2009: Repetitive trends and persistent risk differentials, Child Indi. Res., 6 (2013), 1-21.
doi: 10.1007/s12187-012-9152-8. |
| [12] |
Q. Fu, K. C. Land and V. L. Lamb,
Violent physical bullying victimization at school: has there been a recent increase in exposure or intensity? an age-period-cohort analysis in the united states, 1991 to 2012, Child Indi. Res., 9 (2016), 485-513.
|
| [13] |
Q. Fu, C. Wu, H. Liu, Z. Shi and J. Gu,
Live like mosquitoes: Hukou, rural–urban disparity, and depression, Chin. J. Sociol., 4 (2018), 56-78.
|
| [14] |
W. H. Greene, Accounting for excess zeros and sample selection in Poisson and negative binomial regression models, in NYU working paper no. EC-94-10. |
| [15] |
R. M. Groves, F. J. Fowler Jr, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau, Survey Methodology, vol. 561, John Wiley & Sons, 2011. |
| [16] |
J. M. Hilbe, Negative Binomial Regression, 2$^nd$ edition, Cambridge University Press, Cambridge, 2011.
doi: 10.1017/CBO9780511973420.
|
| [17] |
R. A. Horn and C. R. Johnson, Matrix analysis, 2$^nd$ edition, Cambridge University Press, Cambridge, 2013.
|
| [18] |
L. D. Johnston, P. M. O'Malley and J. G. Bachman,
Bachman, Monitoring the Future: National results on adolescent drug use: Overview of key findings, Focus, 1 (2003), 213-234.
|
| [19] |
L. D. Johnston, P. M. O'Malley, R. A. Miech, J. G. Bachman and J. E. Schulenberg, Monitoring the future national survey results on drug use, 1975–2016: Overview, key findings on adolescent drug use, 2017. Available from: https://files.eric.ed.gov/fulltext/ED578534.pdf. |
| [20] |
L. D. Johnston, P. M. O'Malley, R. A. Miech, J. G. Bachman and J. E. Schulenberg, Monitoring the Future national survey results on drug use, 1975-2016: Overview, key findings on adolescent drug use, Inst. Social Res.. |
| [21] |
F. Kunstner, L. Balles and P. Hennig, Limitations of the empirical Fisher approximation, preprint, arXiv: 1905.12558. |
| [22] |
K. C. Land, P. L. McCall and D. S. Nagin,
A comparison of Poisson, negative binomial, and semiparametric mixed Poisson regression models: With empirical applications to criminal careers data, Sociol. Meth. Res., 24 (1996), 387-442.
|
| [23] |
E. L. Lehmann and G. Casella, Theory of Point Estimation, 2$^{nd}$ edition, Springer Texts in Statistics, Springer-Verlag, New York, 1998. |
| [24] |
L. R. Pacek, R. J. Malcolm and S. S. Martins,
Race/ethnicity differences between alcohol, marijuana, and co-occurring alcohol and marijuana use disorders and their association with public health and social problems using a national sample, Amer. Addi., 21 (2012), 435-444.
|
| [25] |
W. W. Piegorsch,
Maximum likelihood estimation for the negative binomial dispersion parameter, Biometrics, 46 (1990), 863-867.
doi: 10.2307/2532104. |
Table 1.
Heterogeneous negative-binomial regression analysis of lifetime marijuana use among American youth (Number of observations = 8,874). Data source: the 2012 wave of the Monitoring the Future study
| Coefficient | Coefficient | Z value | 95% confidence interval | |
| Covariates for estimating µ | ||||
| Intercept | 0.677*** | 0.183 | 3.696 | [0.318, 1.036] |
| 10th graders | 1.551*** | 0.153 | 10.145 | [1.251, 1.850] |
| 12th graders | 2.002*** | 0.168 | 11.927 | [1.673, 2.331] |
| Male | 1.268*** | 0.125 | 10.143 | [1.023, 1.513] |
| African American | -0.796*** | 0.149 | -5.361 | [-1.087, -0.505] |
| Metropolitan areas | 0.148 | 0.150 | 0.983 | [-0.147, 0.442] |
| Covariates for estimating ν | ||||
| Intercept | -3.627*** | 0.082 | -44.331 | [-3.787, -3.466] |
| 10th graders | 0.972*** | 0.068 | 14.374 | [0.839, 1.104] |
| 12th graders | 1.332*** | 0.074 | 18.018 | [1.188, 1.477] |
| Male | -0.006 | 0.051 | -0.107 | [-0.106, 0.095] |
| African American | 0.268*** | 0.077 | 3.480 | [0.117, 0.418] |
| Metropolitan areas | 0.117 . | 0.063 | 1.844 | [-0.007, 0.240] |
| Goodness of fit | ||||
| AIC | 18400 | BIC | 18480 | |
| McFadden’s R2 | 0.04828 | McFadden’s adjusted R2 | 0.04703 | |
| Note: ***p<0.001 ** p<0.01 * p<0.05 . P<0.1 | ||||
| Coefficient | Coefficient | Z value | 95% confidence interval | |
| Covariates for estimating µ | ||||
| Intercept | 0.677*** | 0.183 | 3.696 | [0.318, 1.036] |
| 10th graders | 1.551*** | 0.153 | 10.145 | [1.251, 1.850] |
| 12th graders | 2.002*** | 0.168 | 11.927 | [1.673, 2.331] |
| Male | 1.268*** | 0.125 | 10.143 | [1.023, 1.513] |
| African American | -0.796*** | 0.149 | -5.361 | [-1.087, -0.505] |
| Metropolitan areas | 0.148 | 0.150 | 0.983 | [-0.147, 0.442] |
| Covariates for estimating ν | ||||
| Intercept | -3.627*** | 0.082 | -44.331 | [-3.787, -3.466] |
| 10th graders | 0.972*** | 0.068 | 14.374 | [0.839, 1.104] |
| 12th graders | 1.332*** | 0.074 | 18.018 | [1.188, 1.477] |
| Male | -0.006 | 0.051 | -0.107 | [-0.106, 0.095] |
| African American | 0.268*** | 0.077 | 3.480 | [0.117, 0.418] |
| Metropolitan areas | 0.117 . | 0.063 | 1.844 | [-0.007, 0.240] |
| Goodness of fit | ||||
| AIC | 18400 | BIC | 18480 | |
| McFadden’s R2 | 0.04828 | McFadden’s adjusted R2 | 0.04703 | |
| Note: ***p<0.001 ** p<0.01 * p<0.05 . P<0.1 | ||||
| [1] |
Wei Li, Yun Teng. Enterprise inefficient investment behavior analysis based on regression analysis. Discrete and Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1015-1025. doi: 10.3934/dcdss.2019069 |
| [2] |
Jiang Xie, Junfu Xu, Celine Nie, Qing Nie. Machine learning of swimming data via wisdom of crowd and regression analysis. Mathematical Biosciences & Engineering, 2017, 14 (2) : 511-527. doi: 10.3934/mbe.2017031 |
| [3] |
Bingzheng Li, Zhengzhan Dai. Error analysis on regularized regression based on the Maximum correntropy criterion. Mathematical Foundations of Computing, 2020, 3 (1) : 25-40. doi: 10.3934/mfc.2020003 |
| [4] |
Wenming Li, Yongge Tian, Ruixia Yuan. Statistical analysis of a linear regression model with restrictions and superfluous variables. Journal of Industrial and Management Optimization, 2022 doi: 10.3934/jimo.2022079 |
| [5] |
Wai-Ki Ching, Jia-Wen Gu, Harry Zheng. On correlated defaults and incomplete information. Journal of Industrial and Management Optimization, 2021, 17 (2) : 889-908. doi: 10.3934/jimo.2020003 |
| [6] |
Shuhua Wang, Zhenlong Chen, Baohuai Sheng. Convergence of online pairwise regression learning with quadratic loss. Communications on Pure and Applied Analysis, 2020, 19 (8) : 4023-4054. doi: 10.3934/cpaa.2020178 |
| [7] |
Adil Bagirov, Sona Taheri, Soodabeh Asadi. A difference of convex optimization algorithm for piecewise linear regression. Journal of Industrial and Management Optimization, 2019, 15 (2) : 909-932. doi: 10.3934/jimo.2018077 |
| [8] |
Shaoyong Lai, Qichang Xie. A selection problem for a constrained linear regression model. Journal of Industrial and Management Optimization, 2008, 4 (4) : 757-766. doi: 10.3934/jimo.2008.4.757 |
| [9] |
Nicolas Rougerie. On two properties of the Fisher information. Kinetic and Related Models, 2021, 14 (1) : 77-88. doi: 10.3934/krm.2020049 |
| [10] |
Song Wang, Quanxi Shao, Xian Zhou. Knot-optimizing spline networks (KOSNETS) for nonparametric regression. Journal of Industrial and Management Optimization, 2008, 4 (1) : 33-52. doi: 10.3934/jimo.2008.4.33 |
| [11] |
Baohuai Sheng, Huanxiang Liu, Huimin Wang. Learning rates for the kernel regularized regression with a differentiable strongly convex loss. Communications on Pure and Applied Analysis, 2020, 19 (8) : 3973-4005. doi: 10.3934/cpaa.2020176 |
| [12] |
Erik Kropat, Gerhard Wilhelm Weber. Fuzzy target-environment networks and fuzzy-regression approaches. Numerical Algebra, Control and Optimization, 2018, 8 (2) : 135-155. doi: 10.3934/naco.2018008 |
| [13] |
Yang Mi, Kang Zheng, Song Wang. Homography estimation along short videos by recurrent convolutional regression network. Mathematical Foundations of Computing, 2020, 3 (2) : 125-140. doi: 10.3934/mfc.2020014 |
| [14] |
Qing Xu, Xiaohua (Michael) Xuan. Nonlinear regression without i.i.d. assumption. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 8-. doi: 10.1186/s41546-019-0042-6 |
| [15] |
Lican Kang, Yanming Lai, Yanyan Liu, Yuan Luo, Jing Zhang. High-dimensional linear regression with hard thresholding regularization: Theory and algorithm. Journal of Industrial and Management Optimization, 2022 doi: 10.3934/jimo.2022034 |
| [16] |
Feimin Zhong, Jinxing Xie, Jing Jiao. Solutions for bargaining games with incomplete information: General type space and action space. Journal of Industrial and Management Optimization, 2018, 14 (3) : 953-966. doi: 10.3934/jimo.2017084 |
| [17] |
Miquel Oliu-Barton. Asymptotically optimal strategies in repeated games with incomplete information and vanishing weights. Journal of Dynamics and Games, 2019, 6 (4) : 259-275. doi: 10.3934/jdg.2019018 |
| [18] |
Yanqing Liu, Jiyuan Tao, Huan Zhang, Xianchao Xiu, Lingchen Kong. Fused LASSO penalized least absolute deviation estimator for high dimensional linear regression. Numerical Algebra, Control and Optimization, 2018, 8 (1) : 97-117. doi: 10.3934/naco.2018006 |
| [19] |
Lianjun Zhang, Lingchen Kong, Yan Li, Shenglong Zhou. A smoothing iterative method for quantile regression with nonconvex $ \ell_p $ penalty. Journal of Industrial and Management Optimization, 2017, 13 (1) : 93-112. doi: 10.3934/jimo.2016006 |
| [20] |
Yuyuan Ouyang, Trevor Squires. Some worst-case datasets of deterministic first-order methods for solving binary logistic regression. Inverse Problems and Imaging, 2021, 15 (1) : 63-77. doi: 10.3934/ipi.2020047 |
2020 Impact Factor: 1.916
Tools
Metrics
Other articles
by authors
[Back to Top]