doi: 10.3934/jimo.2020128

Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models

Faculty of Mathematics, Statistics and Computer Science, Semnan University, P.O. Box 35195–363, Semnan, Iran

* Corresponding author: Mahdi Roozbeh

Received  September 2019 Revised  May 2020 Published  August 2020

In classical regression analysis, the ordinary least–squares estimation is the best strategy when the essential assumptions such as normality and independency to the error terms as well as ignorable multicollinearity in the covariates are met. However, if one of these assumptions is violated, then the results may be misleading. Especially, outliers violate the assumption of normally distributed residuals in the least–squares regression. In this situation, robust estimators are widely used because of their lack of sensitivity to outlying data points. Multicollinearity is another common problem in multiple regression models with inappropriate effects on the least–squares estimators. So, it is of great importance to use the estimation methods provided to tackle the mentioned problems. As known, robust regressions are among the popular methods for analyzing the data that are contaminated with outliers. In this guideline, here we suggest two mixed–integer nonlinear optimization models which their solutions can be considered as appropriate estimators when the outliers and multicollinearity simultaneously appear in the data set. Capable to be effectively solved by metaheuristic algorithms, the models are designed based on penalization schemes with the ability of down–weighting or ignoring unusual data and multicollinearity effects. We establish that our models are computationally advantageous in the perspective of the flop count. We also deal with a robust ridge methodology. Finally, three real data sets are analyzed to examine performance of the proposed methods.

Citation: Mahdi Roozbeh, Saman Babaie–Kafaki, Zohre Aminifard. Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2020128
References:
[1]

E. H. L. Aarts, J. H. M. Korst and P. J. M. van Laarhoren, Simulated annealing, in Local Search in Combinatorial Optimization, Wiley-Intersci. Ser. Discrete Math. Optim., Wiley-Intersci. Publ., Wiley, Chichester, 1997, 91–121.  Google Scholar

[2]

E. Akdenïz DuranW. K. Härdle and M. Osipenko, Difference based ridge and Liu type estimators in semiparametric regression models, J. Multivariate Anal., 105 (2012), 164-175.  doi: 10.1016/j.jmva.2011.08.018.  Google Scholar

[3]

F. Akdenïz and M. Roozbeh, Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models, Statist. Papers, 60 (2019), 1717-1739.  doi: 10.1007/s00362-017-0893-9.  Google Scholar

[4]

M. Amini and M. Roozbeh, Optimal partial ridge estimation in restricted semiparametric regression models, J. Multivariate Anal., 136 (2015), 26-40.  doi: 10.1016/j.jmva.2015.01.005.  Google Scholar

[5]

M. Arashi and T. Valizadeh, Performance of Kibria's methods in partial linear ridge regression model, Statist. Pap., 56 (2015), 231-246.  doi: 10.1007/s00362-014-0578-6.  Google Scholar

[6]

M. Awad and R. Khanna, Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Apress, Berkeley, CA, 2015. doi: 10.1007/978-1-4302-5990-9.  Google Scholar

[7]

S. Babaie–KafakiR. Ghanbari and N. Mahdavi–Amiri, An efficient and practically robust hybrid metaheuristic algorithm for solving fuzzy bus terminal location problems, Asia–Pac. J. Oper. Res., 29 (2012), 1-25.  doi: 10.1142/S0217595912500091.  Google Scholar

[8]

S. Babaie-KafakiR. Ghanbari and N. Mahdavi-Amiri, Hybridizations of genetic algorithms and neighborhood search metaheuristics for fuzzy bus terminal location problems, Appl. Soft Comput., 46 (2016), 220-229.  doi: 10.1016/j.asoc.2016.03.005.  Google Scholar

[9]

S. Roozbeh and M. Babaie-Kafakiand, A revised Cholesky decomposition to combat multicollinearity in multiple regression models, J. Stat. Comput. Simul., 87 (2017), 2298-2308.  doi: 10.1080/00949655.2017.1328599.  Google Scholar

[10]

M. R. Baye and D. F. Parker, Combining ridge and principal component regression: A money demand illustration, Comm. Statist. A—Theory Methods, 13 (1984), 197-205.  doi: 10.1080/03610928408828675.  Google Scholar

[11]

E. R. Berndt, The Practice of Econometrics, New York, Addison-Wesley, 1991. Google Scholar

[12]

D. Bertsimas and J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, Massachusetts, 1997. Google Scholar

[13]

P. BühlmannM. Kalisch and L. Meier, High–dimensional statistics with a view towards applications in biology, Ann. Rev. Stat. Appl., 1 (2014), 255-278.   Google Scholar

[14]

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi–Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.  doi: 10.1137/0726042.  Google Scholar

[15]

M. Hassanzadeh BashtianM. Arashi and S. M. M. Tabatabaey, Using improved estimation strategies to combat multicollinearity, J. Stat. Comput. Simul., 81 (2011), 1773-1797.  doi: 10.1080/00949655.2010.505925.  Google Scholar

[16]

S. Hawkins, H. He, G. Williams and R. Baxter, Outlier detection using replicator neural networks, in International Conference on Data Warehousing and Knowledge Discovery, Springer, Berlin, Heidelberg, (2002), 170–180. doi: 10.1007/3-540-46145-0_17.  Google Scholar

[17]

D. Henderson, S. H. Jacobson and A. W. Johnson, The theory and practice of simulated annealing, in Handbook of Metaheuristics, Kluwer Academic Publishers, Boston, MA, (2003), 287–319. doi: 10.1007/0-306-48056-5_10.  Google Scholar

[18]

A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation for non–orthogonal problems, Technometrics, 12 (1970), 55-67.   Google Scholar

[19]

P. W. Holland and R. E. Welsch, Robust regression using iteratively reweighted least–squares, Comm. Statist. Theo. Meth., 6 (1977), 813-827.   Google Scholar

[20]

G. James, D. Witten, T. Hastie and R. Tibshirani, An Introduction to Statistical Learning, Springer, New York, 2013. doi: 10.1007/978-1-4614-7138-7.  Google Scholar

[21]

S. Kaçiranlar and S. Sakallioǧlu, Combining the Liu estimator and the principal component regression estimator, Comm. Statist. Theory Methods, 30 (2001), 2699-2705.  doi: 10.1081/STA-100108454.  Google Scholar

[22]

A. KaratzoglouD. Meyer and K. Hornik, Support Vector Machines in R, J. Stat. Softw., 15 (2006), 1-28.   Google Scholar

[23]

K. J. Liu, A new class of biased estimate in linear regression, Comm. Statist. Theory Methods, 22 (1993), 393-402.  doi: 10.1080/03610929308831027.  Google Scholar

[24]

A. Mohammad NezhadR. Aliakbari Shandiz and A. H. Eshraghniaye Jahromi, A particle swarm–BFGS algorithm for nonlinear programming problems, Comput. Oper. Res., 40 (2013), 963-972.  doi: 10.1016/j.cor.2012.11.008.  Google Scholar

[25]

G. Piazza and T. Politi, An upper bound for the condition number of a matrix in spectral norm, J. Comput. Appl. Math., 143 (2002), 141-144.  doi: 10.1016/S0377-0427(02)00396-5.  Google Scholar

[26]

W. M. Pride and O. C. Ferrel, Marketing, 15th edition, South-Western, Cengage Learning, International Edition, 2010. Google Scholar

[27]

C. R. Reeves, Modern heuristic techniques, in Modern Heuristic Search Methods, John Wiley and Sons, Chichester, (1996), 1–24. Google Scholar

[28]

M. Roozbeh, Optimal QR-based estimation in partially linear regression models with correlated errors using GCV criterion, Computational Statistics & Data Analysis, 117 (2018), 45-61.  doi: 10.1016/j.csda.2017.08.002.  Google Scholar

[29]

M. RoozbehS. Babaie-Kafaki and M. Arashi, A class of biased estimators based on QR decomposition, Linear Algebra Appl., 508 (2016), 190-205.  doi: 10.1016/j.laa.2016.07.009.  Google Scholar

[30]

M. RoozbehS. Babaie-Kafaki and A. Naeimi Sadigh, A heuristic approach to combat multicollinearity in least trimmed squares regression analysis, Appl. Math. Model, 57 (2018), 105-120.  doi: 10.1016/j.apm.2017.11.011.  Google Scholar

[31]

M. Roozbeh, Robust ridge estimator in restricted semiparametric regression models, J. Multivariate Anal., 147 (2016), 127-144.  doi: 10.1016/j.jmva.2016.01.005.  Google Scholar

[32]

P. J. Rousseeuw, Least median of squares regression, J. Amer. Statist. Assoc., 79 (1984), 871-880.  doi: 10.1080/01621459.1984.10477105.  Google Scholar

[33]

P. J. Rousseeuw, and A. M. Leroy, Robust Regression and Outlier Detection, John Wiley and Sons, New York, 1987. doi: 10.1002/0471725382.  Google Scholar

[34]

S. J. Sheather, A Modern Approach to Regression with R, Springer, New York, 2009. doi: 10.1007/978-0-387-09608-7.  Google Scholar

[35]

W. Sun and Y. X. Yuan, Optimization Theory and Methods: Nonlinear Programming, Springer, New York, 2006.  Google Scholar

[36]

P. Tryfos, Methods for Business Analysis and Forecasting: Text & Cases, John Wiley and Sons, New York, 1998. Google Scholar

[37]

D. S. Watkins, Fundamentals of Matrix Computations, 2nd edition, John Wiley and Sons, New York, 2002. doi: 10.1002/0471249718.  Google Scholar

[38]

X. S. Yang, Nature–Inspired Optimization Algorithms, Elsevier, Amsterdam, 2014.  Google Scholar

show all references

References:
[1]

E. H. L. Aarts, J. H. M. Korst and P. J. M. van Laarhoren, Simulated annealing, in Local Search in Combinatorial Optimization, Wiley-Intersci. Ser. Discrete Math. Optim., Wiley-Intersci. Publ., Wiley, Chichester, 1997, 91–121.  Google Scholar

[2]

E. Akdenïz DuranW. K. Härdle and M. Osipenko, Difference based ridge and Liu type estimators in semiparametric regression models, J. Multivariate Anal., 105 (2012), 164-175.  doi: 10.1016/j.jmva.2011.08.018.  Google Scholar

[3]

F. Akdenïz and M. Roozbeh, Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models, Statist. Papers, 60 (2019), 1717-1739.  doi: 10.1007/s00362-017-0893-9.  Google Scholar

[4]

M. Amini and M. Roozbeh, Optimal partial ridge estimation in restricted semiparametric regression models, J. Multivariate Anal., 136 (2015), 26-40.  doi: 10.1016/j.jmva.2015.01.005.  Google Scholar

[5]

M. Arashi and T. Valizadeh, Performance of Kibria's methods in partial linear ridge regression model, Statist. Pap., 56 (2015), 231-246.  doi: 10.1007/s00362-014-0578-6.  Google Scholar

[6]

M. Awad and R. Khanna, Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Apress, Berkeley, CA, 2015. doi: 10.1007/978-1-4302-5990-9.  Google Scholar

[7]

S. Babaie–KafakiR. Ghanbari and N. Mahdavi–Amiri, An efficient and practically robust hybrid metaheuristic algorithm for solving fuzzy bus terminal location problems, Asia–Pac. J. Oper. Res., 29 (2012), 1-25.  doi: 10.1142/S0217595912500091.  Google Scholar

[8]

S. Babaie-KafakiR. Ghanbari and N. Mahdavi-Amiri, Hybridizations of genetic algorithms and neighborhood search metaheuristics for fuzzy bus terminal location problems, Appl. Soft Comput., 46 (2016), 220-229.  doi: 10.1016/j.asoc.2016.03.005.  Google Scholar

[9]

S. Roozbeh and M. Babaie-Kafakiand, A revised Cholesky decomposition to combat multicollinearity in multiple regression models, J. Stat. Comput. Simul., 87 (2017), 2298-2308.  doi: 10.1080/00949655.2017.1328599.  Google Scholar

[10]

M. R. Baye and D. F. Parker, Combining ridge and principal component regression: A money demand illustration, Comm. Statist. A—Theory Methods, 13 (1984), 197-205.  doi: 10.1080/03610928408828675.  Google Scholar

[11]

E. R. Berndt, The Practice of Econometrics, New York, Addison-Wesley, 1991. Google Scholar

[12]

D. Bertsimas and J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, Massachusetts, 1997. Google Scholar

[13]

P. BühlmannM. Kalisch and L. Meier, High–dimensional statistics with a view towards applications in biology, Ann. Rev. Stat. Appl., 1 (2014), 255-278.   Google Scholar

[14]

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi–Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.  doi: 10.1137/0726042.  Google Scholar

[15]

M. Hassanzadeh BashtianM. Arashi and S. M. M. Tabatabaey, Using improved estimation strategies to combat multicollinearity, J. Stat. Comput. Simul., 81 (2011), 1773-1797.  doi: 10.1080/00949655.2010.505925.  Google Scholar

[16]

S. Hawkins, H. He, G. Williams and R. Baxter, Outlier detection using replicator neural networks, in International Conference on Data Warehousing and Knowledge Discovery, Springer, Berlin, Heidelberg, (2002), 170–180. doi: 10.1007/3-540-46145-0_17.  Google Scholar

[17]

D. Henderson, S. H. Jacobson and A. W. Johnson, The theory and practice of simulated annealing, in Handbook of Metaheuristics, Kluwer Academic Publishers, Boston, MA, (2003), 287–319. doi: 10.1007/0-306-48056-5_10.  Google Scholar

[18]

A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation for non–orthogonal problems, Technometrics, 12 (1970), 55-67.   Google Scholar

[19]

P. W. Holland and R. E. Welsch, Robust regression using iteratively reweighted least–squares, Comm. Statist. Theo. Meth., 6 (1977), 813-827.   Google Scholar

[20]

G. James, D. Witten, T. Hastie and R. Tibshirani, An Introduction to Statistical Learning, Springer, New York, 2013. doi: 10.1007/978-1-4614-7138-7.  Google Scholar

[21]

S. Kaçiranlar and S. Sakallioǧlu, Combining the Liu estimator and the principal component regression estimator, Comm. Statist. Theory Methods, 30 (2001), 2699-2705.  doi: 10.1081/STA-100108454.  Google Scholar

[22]

A. KaratzoglouD. Meyer and K. Hornik, Support Vector Machines in R, J. Stat. Softw., 15 (2006), 1-28.   Google Scholar

[23]

K. J. Liu, A new class of biased estimate in linear regression, Comm. Statist. Theory Methods, 22 (1993), 393-402.  doi: 10.1080/03610929308831027.  Google Scholar

[24]

A. Mohammad NezhadR. Aliakbari Shandiz and A. H. Eshraghniaye Jahromi, A particle swarm–BFGS algorithm for nonlinear programming problems, Comput. Oper. Res., 40 (2013), 963-972.  doi: 10.1016/j.cor.2012.11.008.  Google Scholar

[25]

G. Piazza and T. Politi, An upper bound for the condition number of a matrix in spectral norm, J. Comput. Appl. Math., 143 (2002), 141-144.  doi: 10.1016/S0377-0427(02)00396-5.  Google Scholar

[26]

W. M. Pride and O. C. Ferrel, Marketing, 15th edition, South-Western, Cengage Learning, International Edition, 2010. Google Scholar

[27]

C. R. Reeves, Modern heuristic techniques, in Modern Heuristic Search Methods, John Wiley and Sons, Chichester, (1996), 1–24. Google Scholar

[28]

M. Roozbeh, Optimal QR-based estimation in partially linear regression models with correlated errors using GCV criterion, Computational Statistics & Data Analysis, 117 (2018), 45-61.  doi: 10.1016/j.csda.2017.08.002.  Google Scholar

[29]

M. RoozbehS. Babaie-Kafaki and M. Arashi, A class of biased estimators based on QR decomposition, Linear Algebra Appl., 508 (2016), 190-205.  doi: 10.1016/j.laa.2016.07.009.  Google Scholar

[30]

M. RoozbehS. Babaie-Kafaki and A. Naeimi Sadigh, A heuristic approach to combat multicollinearity in least trimmed squares regression analysis, Appl. Math. Model, 57 (2018), 105-120.  doi: 10.1016/j.apm.2017.11.011.  Google Scholar

[31]

M. Roozbeh, Robust ridge estimator in restricted semiparametric regression models, J. Multivariate Anal., 147 (2016), 127-144.  doi: 10.1016/j.jmva.2016.01.005.  Google Scholar

[32]

P. J. Rousseeuw, Least median of squares regression, J. Amer. Statist. Assoc., 79 (1984), 871-880.  doi: 10.1080/01621459.1984.10477105.  Google Scholar

[33]

P. J. Rousseeuw, and A. M. Leroy, Robust Regression and Outlier Detection, John Wiley and Sons, New York, 1987. doi: 10.1002/0471725382.  Google Scholar

[34]

S. J. Sheather, A Modern Approach to Regression with R, Springer, New York, 2009. doi: 10.1007/978-0-387-09608-7.  Google Scholar

[35]

W. Sun and Y. X. Yuan, Optimization Theory and Methods: Nonlinear Programming, Springer, New York, 2006.  Google Scholar

[36]

P. Tryfos, Methods for Business Analysis and Forecasting: Text & Cases, John Wiley and Sons, New York, 1998. Google Scholar

[37]

D. S. Watkins, Fundamentals of Matrix Computations, 2nd edition, John Wiley and Sons, New York, 2002. doi: 10.1002/0471249718.  Google Scholar

[38]

X. S. Yang, Nature–Inspired Optimization Algorithms, Elsevier, Amsterdam, 2014.  Google Scholar

Figure 1.  The diagnostic plots of the model (18)
Figure 2.  The diagram of $ {\rm GCV}(k,z) $ versus the ridge parameter for the bridge projects data set
Figure 3.  The diagnostic plots for the model (20)
Figure 4.  The diagram of $ {\rm GCV}(k,z) $ versus the ridge parameter for the electricity data
Figure 5.  The diagnostic plots for the model (21)
Figure 6.  The diagram of $ {\rm GCV}(k,z) $ versus the ridge parameter for the CPS data
Table 1.  Evaluation of the proposed estimators for the bridge projects data set
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 2.3317 1.91363 2.0304 1.8278
$ \log(CCost) $ 0.1483 0.33718 0.3056 0.2923
$ \log(Dwgs) $ 0.8356 0.58002 0.6210 0.7829
$ \log(Spans) $ 0.1963 0.06662 0.0657 0.0241
$ {\rm SSE} $ 3.8692 1.9788 1.9778 1.0577
$ {\rm R}^2 $ 0.7747 0.8579 0.8600 0.9147
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 1.9140 -0.0125 - -7.8431
$ \log(CCost) $ 0.2360 0.4152 - 0.4236
$ \log(Dwgs) $ 0.8914 0.3933 - 2.8061
$ \log(Spans) $ 0.0467 0.1176 - 0.5110
$ {\rm SSE} $ 1.1504 4.0131 2.7834 1.7108
$ {\rm R}^2 $ 0.9020 0.7663 0.8379 0.9004
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 2.3317 1.91363 2.0304 1.8278
$ \log(CCost) $ 0.1483 0.33718 0.3056 0.2923
$ \log(Dwgs) $ 0.8356 0.58002 0.6210 0.7829
$ \log(Spans) $ 0.1963 0.06662 0.0657 0.0241
$ {\rm SSE} $ 3.8692 1.9788 1.9778 1.0577
$ {\rm R}^2 $ 0.7747 0.8579 0.8600 0.9147
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 1.9140 -0.0125 - -7.8431
$ \log(CCost) $ 0.2360 0.4152 - 0.4236
$ \log(Dwgs) $ 0.8914 0.3933 - 2.8061
$ \log(Spans) $ 0.0467 0.1176 - 0.5110
$ {\rm SSE} $ 1.1504 4.0131 2.7834 1.7108
$ {\rm R}^2 $ 0.9020 0.7663 0.8379 0.9004
Table 2.  The most effective subgroup of predictor variables based on the $ {\rm R}^2_{adj} $ and AIC criteria for the electricity data set
Subset size Predictor variables $ {\rm R}^2_{adj} $ AIC
1 $ Temp $ 0.5523 -1067.814
2 $ Temp,LREG $ 0.5781 -1077.339
3 $ {\bf Temp,LREG,LI} $ 0.5892 -1081.063
4 $ Temp,LREG,LI,x_{9} $ 0.5891 -1080.057
5 $ Temp,LREG,LI,x_{9},x_{10} $ 0.5882 -1078.709
6 $ Temp,LREG,LI,x_{9},x_{10},x_{11} $ 0.5875 -1077.427
7 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1} $ 0.5858 -1075.734
8 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3} $ 0.5837 -1073.897
9 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5} $ 0.5812 -1071.907
10 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4} $ 0.5789 -1069.987
11 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7} $ 0.5764 -1067.997
12 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2} $ 0.5740 -1064.098
13 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6} $ 0.5718 -1064.281
14 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6},x_{8} $ 0.5709 -1063.014
Subset size Predictor variables $ {\rm R}^2_{adj} $ AIC
1 $ Temp $ 0.5523 -1067.814
2 $ Temp,LREG $ 0.5781 -1077.339
3 $ {\bf Temp,LREG,LI} $ 0.5892 -1081.063
4 $ Temp,LREG,LI,x_{9} $ 0.5891 -1080.057
5 $ Temp,LREG,LI,x_{9},x_{10} $ 0.5882 -1078.709
6 $ Temp,LREG,LI,x_{9},x_{10},x_{11} $ 0.5875 -1077.427
7 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1} $ 0.5858 -1075.734
8 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3} $ 0.5837 -1073.897
9 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5} $ 0.5812 -1071.907
10 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4} $ 0.5789 -1069.987
11 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7} $ 0.5764 -1067.997
12 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2} $ 0.5740 -1064.098
13 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6} $ 0.5718 -1064.281
14 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6},x_{8} $ 0.5709 -1063.014
Table 3.  Evaluation of the proposed estimators for the electricity data set
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 4.4069 5.1693 4.9881 5.2039
$ LI $ 0.1925 0.0989 0.1146 0.0956
$ LREG $ -0.0778 -0.0939 -0.1054 -0.0956
$ Temp $ -0.0002 -0.0002 -0.0003 -0.0003
$ {\rm SSE} $ 0.3765 0.2637 0.1982 0.1296
$ {\rm R}^2 $ 0.5962 0.6742 0.7399 0.7559
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 4.0907 0.0881 - 2.6215
$ LI $ 0.2225 0.1545 - 1.2806
$ LREG $ -0.0940 -0.1322 - -3.7418
$ Temp $ -0.0003 -0.7508 - -0.8067
$ {\rm SSE} $ 0.1413 0.3881 0.2629 0.4240
$ {\rm R}^2 $ 0.7468 0.5838 0.7181 0.5452
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 4.4069 5.1693 4.9881 5.2039
$ LI $ 0.1925 0.0989 0.1146 0.0956
$ LREG $ -0.0778 -0.0939 -0.1054 -0.0956
$ Temp $ -0.0002 -0.0002 -0.0003 -0.0003
$ {\rm SSE} $ 0.3765 0.2637 0.1982 0.1296
$ {\rm R}^2 $ 0.5962 0.6742 0.7399 0.7559
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 4.0907 0.0881 - 2.6215
$ LI $ 0.2225 0.1545 - 1.2806
$ LREG $ -0.0940 -0.1322 - -3.7418
$ Temp $ -0.0003 -0.7508 - -0.8067
$ {\rm SSE} $ 0.1413 0.3881 0.2629 0.4240
$ {\rm R}^2 $ 0.7468 0.5838 0.7181 0.5452
Table 4.  Evaluation of the proposed estimators for the CPS data
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 1.0786 0.7498 1.1963 0.9257
$ education $ 0.1794 0.1482 0.2576 0.2018
$ south $ -0.1024 -0.1208 -0.1109 -0.1174
$ sex $ -0.2220 -0.2851 -0.2776 -0.2665
$ experience $ 0.0958 0.0613 0.1630 0.1090
$ union $ 0.2005 0.1939 0.1987 0.1427
$ age $ -0.0854 -0.0473 -0.1510 -0.0960
$ race $ 0.0504 0.0674 0.0482 0.0749
$ occupation $ -0.0074 -0.0122 0.0072 -0.0126
$ sector $ 0.0915 0.0614 0.0411 0.0965
$ married $ 0.0766 0.0590 0.1937 0.0924
$ {\rm SSE} $ 101.17 76.3827 50.5810 49.8101
$ {\rm R}^2 $ 0.3185 0.4049 0.4146 0.4123
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 0.9038 0.0054 - -5.5913
$ education $ 0.1974 0.4997 - 0.6978
$ south $ -0.0916 -0.1141 - -0.4331
$ sex $ -0.2416 -0.2638 - -0.9731
$ experience $ 0.1011 0.2573 - 0.2991
$ union $ 0.1791 0.1511 - 1.0483
$ age $ -0.0888 0.0420 - -0.2590
$ race $ 0.0515 0.0930 - 0.2437
$ occupation $ -0.0140 -0.0526 - 0.0004
$ sector $ 0.0810 0.0918 - 0.3258
$ married $ 0.1216 0.0524 - 0.4156
$ {\rm SSE} $ 49.2827 102.5847 79.0911 84.2234
$ {\rm R}^2 $ 0.4279 0.3089 0.4672 0.4326
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 1.0786 0.7498 1.1963 0.9257
$ education $ 0.1794 0.1482 0.2576 0.2018
$ south $ -0.1024 -0.1208 -0.1109 -0.1174
$ sex $ -0.2220 -0.2851 -0.2776 -0.2665
$ experience $ 0.0958 0.0613 0.1630 0.1090
$ union $ 0.2005 0.1939 0.1987 0.1427
$ age $ -0.0854 -0.0473 -0.1510 -0.0960
$ race $ 0.0504 0.0674 0.0482 0.0749
$ occupation $ -0.0074 -0.0122 0.0072 -0.0126
$ sector $ 0.0915 0.0614 0.0411 0.0965
$ married $ 0.0766 0.0590 0.1937 0.0924
$ {\rm SSE} $ 101.17 76.3827 50.5810 49.8101
$ {\rm R}^2 $ 0.3185 0.4049 0.4146 0.4123
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 0.9038 0.0054 - -5.5913
$ education $ 0.1974 0.4997 - 0.6978
$ south $ -0.0916 -0.1141 - -0.4331
$ sex $ -0.2416 -0.2638 - -0.9731
$ experience $ 0.1011 0.2573 - 0.2991
$ union $ 0.1791 0.1511 - 1.0483
$ age $ -0.0888 0.0420 - -0.2590
$ race $ 0.0515 0.0930 - 0.2437
$ occupation $ -0.0140 -0.0526 - 0.0004
$ sector $ 0.0810 0.0918 - 0.3258
$ married $ 0.1216 0.0524 - 0.4156
$ {\rm SSE} $ 49.2827 102.5847 79.0911 84.2234
$ {\rm R}^2 $ 0.4279 0.3089 0.4672 0.4326
[1]

Yahia Zare Mehrjerdi. A new methodology for solving bi-criterion fractional stochastic programming. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020054

[2]

Christian Beck, Lukas Gonon, Martin Hutzenthaler, Arnulf Jentzen. On existence and uniqueness properties for solutions of stochastic fixed point equations. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020320

[3]

Suhaib Abduljabbar Altamir, Zakariya Yahya Algamal. Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 87-98. doi: 10.3934/naco.2020017

[4]

Min Chen, Olivier Goubet, Shenghao Li. Mathematical analysis of bump to bucket problem. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5567-5580. doi: 10.3934/cpaa.2020251

[5]

Huiying Fan, Tao Ma. Parabolic equations involving Laguerre operators and weighted mixed-norm estimates. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5487-5508. doi: 10.3934/cpaa.2020249

[6]

Juan Pablo Pinasco, Mauro Rodriguez Cartabia, Nicolas Saintier. Evolutionary game theory in mixed strategies: From microscopic interactions to kinetic equations. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2020051

[7]

Jie Zhang, Yuping Duan, Yue Lu, Michael K. Ng, Huibin Chang. Bilinear constraint based ADMM for mixed Poisson-Gaussian noise removal. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020071

[8]

Qianqian Han, Xiao-Song Yang. Qualitative analysis of a generalized Nosé-Hoover oscillator. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020346

[9]

Laurence Cherfils, Stefania Gatti, Alain Miranville, Rémy Guillevin. Analysis of a model for tumor growth and lactate exchanges in a glioma. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020457

[10]

Vieri Benci, Sunra Mosconi, Marco Squassina. Preface: Applications of mathematical analysis to problems in theoretical physics. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020446

[11]

Justin Holmer, Chang Liu. Blow-up for the 1D nonlinear Schrödinger equation with point nonlinearity II: Supercritical blow-up profiles. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020264

[12]

Yining Cao, Chuck Jia, Roger Temam, Joseph Tribbia. Mathematical analysis of a cloud resolving model including the ice microphysics. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 131-167. doi: 10.3934/dcds.2020219

[13]

Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026

[14]

Ahmad Z. Fino, Wenhui Chen. A global existence result for two-dimensional semilinear strongly damped wave equation with mixed nonlinearity in an exterior domain. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5387-5411. doi: 10.3934/cpaa.2020243

[15]

Shenglan Xie, Maoan Han, Peng Zhu. A posteriori error estimate of weak Galerkin fem for second order elliptic problem with mixed boundary condition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020340

[16]

Jianquan Li, Xin Xie, Dian Zhang, Jia Li, Xiaolin Lin. Qualitative analysis of a simple tumor-immune system with time delay of tumor action. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020341

[17]

A. M. Elaiw, N. H. AlShamrani, A. Abdel-Aty, H. Dutta. Stability analysis of a general HIV dynamics model with multi-stages of infected cells and two routes of infection. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020441

2019 Impact Factor: 1.366

Article outline

Figures and Tables

[Back to Top]