American Institute of Mathematical Sciences

Advanced
  • Previous Article
    A polyhedral conic functions based classification method for noisy data
  • JIMO Home
  • This Issue
  • Next Article
    Bayesian decision making in determining optimal leased term and preventive maintenance scheme for leased facilities
November  2021, 17(6): 3475-3491. doi: 10.3934/jimo.2020128

Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models

Mahdi Roozbeh , , Saman Babaie–Kafaki and  Zohre Aminifard

Faculty of Mathematics, Statistics and Computer Science, Semnan University, P.O. Box 35195–363, Semnan, Iran

* Corresponding author: Mahdi Roozbeh

Received  September 2019 Revised  May 2020 Published  November 2021 Early access  August 2020

In classical regression analysis, the ordinary least–squares estimation is the best strategy when the essential assumptions such as normality and independency to the error terms as well as ignorable multicollinearity in the covariates are met. However, if one of these assumptions is violated, then the results may be misleading. Especially, outliers violate the assumption of normally distributed residuals in the least–squares regression. In this situation, robust estimators are widely used because of their lack of sensitivity to outlying data points. Multicollinearity is another common problem in multiple regression models with inappropriate effects on the least–squares estimators. So, it is of great importance to use the estimation methods provided to tackle the mentioned problems. As known, robust regressions are among the popular methods for analyzing the data that are contaminated with outliers. In this guideline, here we suggest two mixed–integer nonlinear optimization models which their solutions can be considered as appropriate estimators when the outliers and multicollinearity simultaneously appear in the data set. Capable to be effectively solved by metaheuristic algorithms, the models are designed based on penalization schemes with the ability of down–weighting or ignoring unusual data and multicollinearity effects. We establish that our models are computationally advantageous in the perspective of the flop count. We also deal with a robust ridge methodology. Finally, three real data sets are analyzed to examine performance of the proposed methods.

Keywords: Regression analysis, multicollinearity, breakdown point, mixed–integer programming, metaheuristic algorithm.
Mathematics Subject Classification: Primary: 62J05, 90C11; Secondary: 90C59.
Citation: Mahdi Roozbeh, Saman Babaie–Kafaki, Zohre Aminifard. Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models. Journal of Industrial and Management Optimization, 2021, 17 (6) : 3475-3491. doi: 10.3934/jimo.2020128
References:
[1]

E. H. L. Aarts, J. H. M. Korst and P. J. M. van Laarhoren, Simulated annealing, in Local Search in Combinatorial Optimization, Wiley-Intersci. Ser. Discrete Math. Optim., Wiley-Intersci. Publ., Wiley, Chichester, 1997, 91–121.

[2]

E. Akdenïz Duran, W. K. Härdle and M. Osipenko, Difference based ridge and Liu type estimators in semiparametric regression models, J. Multivariate Anal., 105 (2012), 164-175.  doi: 10.1016/j.jmva.2011.08.018.

[3]

F. Akdenïz and M. Roozbeh, Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models, Statist. Papers, 60 (2019), 1717-1739.  doi: 10.1007/s00362-017-0893-9.

[4]

M. Amini and M. Roozbeh, Optimal partial ridge estimation in restricted semiparametric regression models, J. Multivariate Anal., 136 (2015), 26-40.  doi: 10.1016/j.jmva.2015.01.005.

[5]

M. Arashi and T. Valizadeh, Performance of Kibria's methods in partial linear ridge regression model, Statist. Pap., 56 (2015), 231-246.  doi: 10.1007/s00362-014-0578-6.

[6]

M. Awad and R. Khanna, Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Apress, Berkeley, CA, 2015. doi: 10.1007/978-1-4302-5990-9.

[7]

S. Babaie–Kafaki, R. Ghanbari and N. Mahdavi–Amiri, An efficient and practically robust hybrid metaheuristic algorithm for solving fuzzy bus terminal location problems, Asia–Pac. J. Oper. Res., 29 (2012), 1-25.  doi: 10.1142/S0217595912500091.

[8]

S. Babaie-Kafaki, R. Ghanbari and N. Mahdavi-Amiri, Hybridizations of genetic algorithms and neighborhood search metaheuristics for fuzzy bus terminal location problems, Appl. Soft Comput., 46 (2016), 220-229.  doi: 10.1016/j.asoc.2016.03.005.

[9]

S. Roozbeh and M. Babaie-Kafakiand, A revised Cholesky decomposition to combat multicollinearity in multiple regression models, J. Stat. Comput. Simul., 87 (2017), 2298-2308.  doi: 10.1080/00949655.2017.1328599.

[10]

M. R. Baye and D. F. Parker, Combining ridge and principal component regression: A money demand illustration, Comm. Statist. A—Theory Methods, 13 (1984), 197-205.  doi: 10.1080/03610928408828675.

[11]

E. R. Berndt, The Practice of Econometrics, New York, Addison-Wesley, 1991.

[12]

D. Bertsimas and J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, Massachusetts, 1997.

[13]

P. Bühlmann, M. Kalisch and L. Meier, High–dimensional statistics with a view towards applications in biology, Ann. Rev. Stat. Appl., 1 (2014), 255-278. 

[14]

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi–Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.  doi: 10.1137/0726042.

[15]

M. Hassanzadeh Bashtian, M. Arashi and S. M. M. Tabatabaey, Using improved estimation strategies to combat multicollinearity, J. Stat. Comput. Simul., 81 (2011), 1773-1797.  doi: 10.1080/00949655.2010.505925.

[16]

S. Hawkins, H. He, G. Williams and R. Baxter, Outlier detection using replicator neural networks, in International Conference on Data Warehousing and Knowledge Discovery, Springer, Berlin, Heidelberg, (2002), 170–180. doi: 10.1007/3-540-46145-0_17.

[17]

D. Henderson, S. H. Jacobson and A. W. Johnson, The theory and practice of simulated annealing, in Handbook of Metaheuristics, Kluwer Academic Publishers, Boston, MA, (2003), 287–319. doi: 10.1007/0-306-48056-5_10.

[18]

A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation for non–orthogonal problems, Technometrics, 12 (1970), 55-67. 

[19]

P. W. Holland and R. E. Welsch, Robust regression using iteratively reweighted least–squares, Comm. Statist. Theo. Meth., 6 (1977), 813-827. 

[20]

G. James, D. Witten, T. Hastie and R. Tibshirani, An Introduction to Statistical Learning, Springer, New York, 2013. doi: 10.1007/978-1-4614-7138-7.

[21]

S. Kaçiranlar and S. Sakallioǧlu, Combining the Liu estimator and the principal component regression estimator, Comm. Statist. Theory Methods, 30 (2001), 2699-2705.  doi: 10.1081/STA-100108454.

[22]

A. Karatzoglou, D. Meyer and K. Hornik, Support Vector Machines in R, J. Stat. Softw., 15 (2006), 1-28. 

[23]

K. J. Liu, A new class of biased estimate in linear regression, Comm. Statist. Theory Methods, 22 (1993), 393-402.  doi: 10.1080/03610929308831027.

[24]

A. Mohammad Nezhad, R. Aliakbari Shandiz and A. H. Eshraghniaye Jahromi, A particle swarm–BFGS algorithm for nonlinear programming problems, Comput. Oper. Res., 40 (2013), 963-972.  doi: 10.1016/j.cor.2012.11.008.

[25]

G. Piazza and T. Politi, An upper bound for the condition number of a matrix in spectral norm, J. Comput. Appl. Math., 143 (2002), 141-144.  doi: 10.1016/S0377-0427(02)00396-5.

[26]

W. M. Pride and O. C. Ferrel, Marketing, 15th edition, South-Western, Cengage Learning, International Edition, 2010.

[27]

C. R. Reeves, Modern heuristic techniques, in Modern Heuristic Search Methods, John Wiley and Sons, Chichester, (1996), 1–24.

[28]

M. Roozbeh, Optimal QR-based estimation in partially linear regression models with correlated errors using GCV criterion, Computational Statistics & Data Analysis, 117 (2018), 45-61.  doi: 10.1016/j.csda.2017.08.002.

[29]

M. Roozbeh, S. Babaie-Kafaki and M. Arashi, A class of biased estimators based on QR decomposition, Linear Algebra Appl., 508 (2016), 190-205.  doi: 10.1016/j.laa.2016.07.009.

[30]

M. Roozbeh, S. Babaie-Kafaki and A. Naeimi Sadigh, A heuristic approach to combat multicollinearity in least trimmed squares regression analysis, Appl. Math. Model, 57 (2018), 105-120.  doi: 10.1016/j.apm.2017.11.011.

[31]

M. Roozbeh, Robust ridge estimator in restricted semiparametric regression models, J. Multivariate Anal., 147 (2016), 127-144.  doi: 10.1016/j.jmva.2016.01.005.

[32]

P. J. Rousseeuw, Least median of squares regression, J. Amer. Statist. Assoc., 79 (1984), 871-880.  doi: 10.1080/01621459.1984.10477105.

[33]

P. J. Rousseeuw, and A. M. Leroy, Robust Regression and Outlier Detection, John Wiley and Sons, New York, 1987. doi: 10.1002/0471725382.

[34]

S. J. Sheather, A Modern Approach to Regression with R, Springer, New York, 2009. doi: 10.1007/978-0-387-09608-7.

[35]

W. Sun and Y. X. Yuan, Optimization Theory and Methods: Nonlinear Programming, Springer, New York, 2006.

[36]

P. Tryfos, Methods for Business Analysis and Forecasting: Text & Cases, John Wiley and Sons, New York, 1998.

[37]

D. S. Watkins, Fundamentals of Matrix Computations, 2nd edition, John Wiley and Sons, New York, 2002. doi: 10.1002/0471249718.

[38]

X. S. Yang, Nature–Inspired Optimization Algorithms, Elsevier, Amsterdam, 2014.

show all references

References:
[1]

E. H. L. Aarts, J. H. M. Korst and P. J. M. van Laarhoren, Simulated annealing, in Local Search in Combinatorial Optimization, Wiley-Intersci. Ser. Discrete Math. Optim., Wiley-Intersci. Publ., Wiley, Chichester, 1997, 91–121.

[2]

E. Akdenïz Duran, W. K. Härdle and M. Osipenko, Difference based ridge and Liu type estimators in semiparametric regression models, J. Multivariate Anal., 105 (2012), 164-175.  doi: 10.1016/j.jmva.2011.08.018.

[3]

F. Akdenïz and M. Roozbeh, Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models, Statist. Papers, 60 (2019), 1717-1739.  doi: 10.1007/s00362-017-0893-9.

[4]

M. Amini and M. Roozbeh, Optimal partial ridge estimation in restricted semiparametric regression models, J. Multivariate Anal., 136 (2015), 26-40.  doi: 10.1016/j.jmva.2015.01.005.

[5]

M. Arashi and T. Valizadeh, Performance of Kibria's methods in partial linear ridge regression model, Statist. Pap., 56 (2015), 231-246.  doi: 10.1007/s00362-014-0578-6.

[6]

M. Awad and R. Khanna, Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Apress, Berkeley, CA, 2015. doi: 10.1007/978-1-4302-5990-9.

[7]

S. Babaie–Kafaki, R. Ghanbari and N. Mahdavi–Amiri, An efficient and practically robust hybrid metaheuristic algorithm for solving fuzzy bus terminal location problems, Asia–Pac. J. Oper. Res., 29 (2012), 1-25.  doi: 10.1142/S0217595912500091.

[8]

S. Babaie-Kafaki, R. Ghanbari and N. Mahdavi-Amiri, Hybridizations of genetic algorithms and neighborhood search metaheuristics for fuzzy bus terminal location problems, Appl. Soft Comput., 46 (2016), 220-229.  doi: 10.1016/j.asoc.2016.03.005.

[9]

S. Roozbeh and M. Babaie-Kafakiand, A revised Cholesky decomposition to combat multicollinearity in multiple regression models, J. Stat. Comput. Simul., 87 (2017), 2298-2308.  doi: 10.1080/00949655.2017.1328599.

[10]

M. R. Baye and D. F. Parker, Combining ridge and principal component regression: A money demand illustration, Comm. Statist. A—Theory Methods, 13 (1984), 197-205.  doi: 10.1080/03610928408828675.

[11]

E. R. Berndt, The Practice of Econometrics, New York, Addison-Wesley, 1991.

[12]

D. Bertsimas and J. N. Tsitsiklis, Introduction to Linear Optimization, Athena Scientific, Massachusetts, 1997.

[13]

P. Bühlmann, M. Kalisch and L. Meier, High–dimensional statistics with a view towards applications in biology, Ann. Rev. Stat. Appl., 1 (2014), 255-278. 

[14]

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi–Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.  doi: 10.1137/0726042.

[15]

M. Hassanzadeh Bashtian, M. Arashi and S. M. M. Tabatabaey, Using improved estimation strategies to combat multicollinearity, J. Stat. Comput. Simul., 81 (2011), 1773-1797.  doi: 10.1080/00949655.2010.505925.

[16]

S. Hawkins, H. He, G. Williams and R. Baxter, Outlier detection using replicator neural networks, in International Conference on Data Warehousing and Knowledge Discovery, Springer, Berlin, Heidelberg, (2002), 170–180. doi: 10.1007/3-540-46145-0_17.

[17]

D. Henderson, S. H. Jacobson and A. W. Johnson, The theory and practice of simulated annealing, in Handbook of Metaheuristics, Kluwer Academic Publishers, Boston, MA, (2003), 287–319. doi: 10.1007/0-306-48056-5_10.

[18]

A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation for non–orthogonal problems, Technometrics, 12 (1970), 55-67. 

[19]

P. W. Holland and R. E. Welsch, Robust regression using iteratively reweighted least–squares, Comm. Statist. Theo. Meth., 6 (1977), 813-827. 

[20]

G. James, D. Witten, T. Hastie and R. Tibshirani, An Introduction to Statistical Learning, Springer, New York, 2013. doi: 10.1007/978-1-4614-7138-7.

[21]

S. Kaçiranlar and S. Sakallioǧlu, Combining the Liu estimator and the principal component regression estimator, Comm. Statist. Theory Methods, 30 (2001), 2699-2705.  doi: 10.1081/STA-100108454.

[22]

A. Karatzoglou, D. Meyer and K. Hornik, Support Vector Machines in R, J. Stat. Softw., 15 (2006), 1-28. 

[23]

K. J. Liu, A new class of biased estimate in linear regression, Comm. Statist. Theory Methods, 22 (1993), 393-402.  doi: 10.1080/03610929308831027.

[24]

A. Mohammad Nezhad, R. Aliakbari Shandiz and A. H. Eshraghniaye Jahromi, A particle swarm–BFGS algorithm for nonlinear programming problems, Comput. Oper. Res., 40 (2013), 963-972.  doi: 10.1016/j.cor.2012.11.008.

[25]

G. Piazza and T. Politi, An upper bound for the condition number of a matrix in spectral norm, J. Comput. Appl. Math., 143 (2002), 141-144.  doi: 10.1016/S0377-0427(02)00396-5.

[26]

W. M. Pride and O. C. Ferrel, Marketing, 15th edition, South-Western, Cengage Learning, International Edition, 2010.

[27]

C. R. Reeves, Modern heuristic techniques, in Modern Heuristic Search Methods, John Wiley and Sons, Chichester, (1996), 1–24.

[28]

M. Roozbeh, Optimal QR-based estimation in partially linear regression models with correlated errors using GCV criterion, Computational Statistics & Data Analysis, 117 (2018), 45-61.  doi: 10.1016/j.csda.2017.08.002.

[29]

M. Roozbeh, S. Babaie-Kafaki and M. Arashi, A class of biased estimators based on QR decomposition, Linear Algebra Appl., 508 (2016), 190-205.  doi: 10.1016/j.laa.2016.07.009.

[30]

M. Roozbeh, S. Babaie-Kafaki and A. Naeimi Sadigh, A heuristic approach to combat multicollinearity in least trimmed squares regression analysis, Appl. Math. Model, 57 (2018), 105-120.  doi: 10.1016/j.apm.2017.11.011.

[31]

M. Roozbeh, Robust ridge estimator in restricted semiparametric regression models, J. Multivariate Anal., 147 (2016), 127-144.  doi: 10.1016/j.jmva.2016.01.005.

[32]

P. J. Rousseeuw, Least median of squares regression, J. Amer. Statist. Assoc., 79 (1984), 871-880.  doi: 10.1080/01621459.1984.10477105.

[33]

P. J. Rousseeuw, and A. M. Leroy, Robust Regression and Outlier Detection, John Wiley and Sons, New York, 1987. doi: 10.1002/0471725382.

[34]

S. J. Sheather, A Modern Approach to Regression with R, Springer, New York, 2009. doi: 10.1007/978-0-387-09608-7.

[35]

W. Sun and Y. X. Yuan, Optimization Theory and Methods: Nonlinear Programming, Springer, New York, 2006.

[36]

P. Tryfos, Methods for Business Analysis and Forecasting: Text & Cases, John Wiley and Sons, New York, 1998.

[37]

D. S. Watkins, Fundamentals of Matrix Computations, 2nd edition, John Wiley and Sons, New York, 2002. doi: 10.1002/0471249718.

[38]

X. S. Yang, Nature–Inspired Optimization Algorithms, Elsevier, Amsterdam, 2014.

Figure 1.  The diagnostic plots of the model (18)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  The diagram of $ {\rm GCV}(k,z) $ versus the ridge parameter for the bridge projects data set
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The diagnostic plots for the model (20)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  The diagram of $ {\rm GCV}(k,z) $ versus the ridge parameter for the electricity data
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  The diagnostic plots for the model (21)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  The diagram of $ {\rm GCV}(k,z) $ versus the ridge parameter for the CPS data
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Evaluation of the proposed estimators for the bridge projects data set
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 2.3317 1.91363 2.0304 1.8278
$ \log(CCost) $ 0.1483 0.33718 0.3056 0.2923
$ \log(Dwgs) $ 0.8356 0.58002 0.6210 0.7829
$ \log(Spans) $ 0.1963 0.06662 0.0657 0.0241
$ {\rm SSE} $ 3.8692 1.9788 1.9778 1.0577
$ {\rm R}^2 $ 0.7747 0.8579 0.8600 0.9147
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 1.9140 -0.0125 - -7.8431
$ \log(CCost) $ 0.2360 0.4152 - 0.4236
$ \log(Dwgs) $ 0.8914 0.3933 - 2.8061
$ \log(Spans) $ 0.0467 0.1176 - 0.5110
$ {\rm SSE} $ 1.1504 4.0131 2.7834 1.7108
$ {\rm R}^2 $ 0.9020 0.7663 0.8379 0.9004
Table Options

Download as excel

Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 2.3317 1.91363 2.0304 1.8278
$ \log(CCost) $ 0.1483 0.33718 0.3056 0.2923
$ \log(Dwgs) $ 0.8356 0.58002 0.6210 0.7829
$ \log(Spans) $ 0.1963 0.06662 0.0657 0.0241
$ {\rm SSE} $ 3.8692 1.9788 1.9778 1.0577
$ {\rm R}^2 $ 0.7747 0.8579 0.8600 0.9147
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 1.9140 -0.0125 - -7.8431
$ \log(CCost) $ 0.2360 0.4152 - 0.4236
$ \log(Dwgs) $ 0.8914 0.3933 - 2.8061
$ \log(Spans) $ 0.0467 0.1176 - 0.5110
$ {\rm SSE} $ 1.1504 4.0131 2.7834 1.7108
$ {\rm R}^2 $ 0.9020 0.7663 0.8379 0.9004
Table 2.  The most effective subgroup of predictor variables based on the $ {\rm R}^2_{adj} $ and AIC criteria for the electricity data set
Subset size Predictor variables $ {\rm R}^2_{adj} $ AIC
1 $ Temp $ 0.5523 -1067.814
2 $ Temp,LREG $ 0.5781 -1077.339
3 $ {\bf Temp,LREG,LI} $ 0.5892 -1081.063
4 $ Temp,LREG,LI,x_{9} $ 0.5891 -1080.057
5 $ Temp,LREG,LI,x_{9},x_{10} $ 0.5882 -1078.709
6 $ Temp,LREG,LI,x_{9},x_{10},x_{11} $ 0.5875 -1077.427
7 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1} $ 0.5858 -1075.734
8 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3} $ 0.5837 -1073.897
9 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5} $ 0.5812 -1071.907
10 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4} $ 0.5789 -1069.987
11 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7} $ 0.5764 -1067.997
12 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2} $ 0.5740 -1064.098
13 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6} $ 0.5718 -1064.281
14 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6},x_{8} $ 0.5709 -1063.014
Table Options

Download as excel

Subset size Predictor variables $ {\rm R}^2_{adj} $ AIC
1 $ Temp $ 0.5523 -1067.814
2 $ Temp,LREG $ 0.5781 -1077.339
3 $ {\bf Temp,LREG,LI} $ 0.5892 -1081.063
4 $ Temp,LREG,LI,x_{9} $ 0.5891 -1080.057
5 $ Temp,LREG,LI,x_{9},x_{10} $ 0.5882 -1078.709
6 $ Temp,LREG,LI,x_{9},x_{10},x_{11} $ 0.5875 -1077.427
7 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1} $ 0.5858 -1075.734
8 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3} $ 0.5837 -1073.897
9 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5} $ 0.5812 -1071.907
10 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4} $ 0.5789 -1069.987
11 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7} $ 0.5764 -1067.997
12 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2} $ 0.5740 -1064.098
13 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6} $ 0.5718 -1064.281
14 $ Temp,LREG,LI,x_{9},x_{10},x_{11},x_{1},x_{3},x_{5},x_{4},x_{7},x_{2},x_{6},x_{8} $ 0.5709 -1063.014
Table 3.  Evaluation of the proposed estimators for the electricity data set
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 4.4069 5.1693 4.9881 5.2039
$ LI $ 0.1925 0.0989 0.1146 0.0956
$ LREG $ -0.0778 -0.0939 -0.1054 -0.0956
$ Temp $ -0.0002 -0.0002 -0.0003 -0.0003
$ {\rm SSE} $ 0.3765 0.2637 0.1982 0.1296
$ {\rm R}^2 $ 0.5962 0.6742 0.7399 0.7559
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 4.0907 0.0881 - 2.6215
$ LI $ 0.2225 0.1545 - 1.2806
$ LREG $ -0.0940 -0.1322 - -3.7418
$ Temp $ -0.0003 -0.7508 - -0.8067
$ {\rm SSE} $ 0.1413 0.3881 0.2629 0.4240
$ {\rm R}^2 $ 0.7468 0.5838 0.7181 0.5452
Table Options

Download as excel

Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 4.4069 5.1693 4.9881 5.2039
$ LI $ 0.1925 0.0989 0.1146 0.0956
$ LREG $ -0.0778 -0.0939 -0.1054 -0.0956
$ Temp $ -0.0002 -0.0002 -0.0003 -0.0003
$ {\rm SSE} $ 0.3765 0.2637 0.1982 0.1296
$ {\rm R}^2 $ 0.5962 0.6742 0.7399 0.7559
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 4.0907 0.0881 - 2.6215
$ LI $ 0.2225 0.1545 - 1.2806
$ LREG $ -0.0940 -0.1322 - -3.7418
$ Temp $ -0.0003 -0.7508 - -0.8067
$ {\rm SSE} $ 0.1413 0.3881 0.2629 0.4240
$ {\rm R}^2 $ 0.7468 0.5838 0.7181 0.5452
Table 4.  Evaluation of the proposed estimators for the CPS data
Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 1.0786 0.7498 1.1963 0.9257
$ education $ 0.1794 0.1482 0.2576 0.2018
$ south $ -0.1024 -0.1208 -0.1109 -0.1174
$ sex $ -0.2220 -0.2851 -0.2776 -0.2665
$ experience $ 0.0958 0.0613 0.1630 0.1090
$ union $ 0.2005 0.1939 0.1987 0.1427
$ age $ -0.0854 -0.0473 -0.1510 -0.0960
$ race $ 0.0504 0.0674 0.0482 0.0749
$ occupation $ -0.0074 -0.0122 0.0072 -0.0126
$ sector $ 0.0915 0.0614 0.0411 0.0965
$ married $ 0.0766 0.0590 0.1937 0.0924
$ {\rm SSE} $ 101.17 76.3827 50.5810 49.8101
$ {\rm R}^2 $ 0.3185 0.4049 0.4146 0.4123
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 0.9038 0.0054 - -5.5913
$ education $ 0.1974 0.4997 - 0.6978
$ south $ -0.0916 -0.1141 - -0.4331
$ sex $ -0.2416 -0.2638 - -0.9731
$ experience $ 0.1011 0.2573 - 0.2991
$ union $ 0.1791 0.1511 - 1.0483
$ age $ -0.0888 0.0420 - -0.2590
$ race $ 0.0515 0.0930 - 0.2437
$ occupation $ -0.0140 -0.0526 - 0.0004
$ sector $ 0.0810 0.0918 - 0.3258
$ married $ 0.1216 0.0524 - 0.4156
$ {\rm SSE} $ 49.2827 102.5847 79.0911 84.2234
$ {\rm R}^2 $ 0.4279 0.3089 0.4672 0.4326
Table Options

Download as excel

Method Coefficients OLS RLTS MLTSCM UBDMLTSCM1
$ Intercept $ 1.0786 0.7498 1.1963 0.9257
$ education $ 0.1794 0.1482 0.2576 0.2018
$ south $ -0.1024 -0.1208 -0.1109 -0.1174
$ sex $ -0.2220 -0.2851 -0.2776 -0.2665
$ experience $ 0.0958 0.0613 0.1630 0.1090
$ union $ 0.2005 0.1939 0.1987 0.1427
$ age $ -0.0854 -0.0473 -0.1510 -0.0960
$ race $ 0.0504 0.0674 0.0482 0.0749
$ occupation $ -0.0074 -0.0122 0.0072 -0.0126
$ sector $ 0.0915 0.0614 0.0411 0.0965
$ married $ 0.0766 0.0590 0.1937 0.0924
$ {\rm SSE} $ 101.17 76.3827 50.5810 49.8101
$ {\rm R}^2 $ 0.3185 0.4049 0.4146 0.4123
Method Coefficients UBDMLTSCM2 LSVR NSVR NNR
$ Intercept $ 0.9038 0.0054 - -5.5913
$ education $ 0.1974 0.4997 - 0.6978
$ south $ -0.0916 -0.1141 - -0.4331
$ sex $ -0.2416 -0.2638 - -0.9731
$ experience $ 0.1011 0.2573 - 0.2991
$ union $ 0.1791 0.1511 - 1.0483
$ age $ -0.0888 0.0420 - -0.2590
$ race $ 0.0515 0.0930 - 0.2437
$ occupation $ -0.0140 -0.0526 - 0.0004
$ sector $ 0.0810 0.0918 - 0.3258
$ married $ 0.1216 0.0524 - 0.4156
$ {\rm SSE} $ 49.2827 102.5847 79.0911 84.2234
$ {\rm R}^2 $ 0.4279 0.3089 0.4672 0.4326
[1]

Fanwen Meng, Kiok Liang Teow, Kelvin Wee Sheng Teo, Chee Kheong Ooi, Seow Yian Tay. Predicting 72-hour reattendance in emergency departments using discriminant analysis via mixed integer programming with electronic medical records. Journal of Industrial and Management Optimization, 2019, 15 (2) : 947-962. doi: 10.3934/jimo.2018079
[2]

Ye Tian, Cheng Lu. Nonconvex quadratic reformulations and solvable conditions for mixed integer quadratic programming problems. Journal of Industrial and Management Optimization, 2011, 7 (4) : 1027-1039. doi: 10.3934/jimo.2011.7.1027
[3]

René Henrion, Christian Küchler, Werner Römisch. Discrepancy distances and scenario reduction in two-stage stochastic mixed-integer programming. Journal of Industrial and Management Optimization, 2008, 4 (2) : 363-384. doi: 10.3934/jimo.2008.4.363
[4]

Louis Caccetta, Syarifah Z. Nordin. Mixed integer programming model for scheduling in unrelated parallel processor system with priority consideration. Numerical Algebra, Control and Optimization, 2014, 4 (2) : 115-132. doi: 10.3934/naco.2014.4.115
[5]

Elham Mardaneh, Ryan Loxton, Qun Lin, Phil Schmidli. A mixed-integer linear programming model for optimal vessel scheduling in offshore oil and gas operations. Journal of Industrial and Management Optimization, 2017, 13 (4) : 1601-1623. doi: 10.3934/jimo.2017009
[6]

Edward S. Canepa, Alexandre M. Bayen, Christian G. Claudel. Spoofing cyber attack detection in probe-based traffic monitoring systems using mixed integer linear programming. Networks and Heterogeneous Media, 2013, 8 (3) : 783-802. doi: 10.3934/nhm.2013.8.783
[7]

Zhiguo Feng, Ka-Fai Cedric Yiu. Manifold relaxations for integer programming. Journal of Industrial and Management Optimization, 2014, 10 (2) : 557-566. doi: 10.3934/jimo.2014.10.557
[8]

Abdel-Rahman Hedar, Alaa Fahim. Filter-based genetic algorithm for mixed variable programming. Numerical Algebra, Control and Optimization, 2011, 1 (1) : 99-116. doi: 10.3934/naco.2011.1.99
[9]

Wan Nor Ashikin Wan Ahmad Fatthi, Adibah Shuib, Rosma Mohd Dom. A mixed integer programming model for solving real-time truck-to-door assignment and scheduling problem at cross docking warehouse. Journal of Industrial and Management Optimization, 2016, 12 (2) : 431-447. doi: 10.3934/jimo.2016.12.431
[10]

Soheil Dolatabadi. Weighted vertices optimizer (WVO): A novel metaheuristic optimization algorithm. Numerical Algebra, Control and Optimization, 2018, 8 (4) : 461-479. doi: 10.3934/naco.2018029
[11]

Guoyong Gu, Junfeng Yang. A unified and tight linear convergence analysis of the relaxed proximal point algorithm. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022107
[12]

Soodabeh Asadi, Hossein Mansouri. A Mehrotra type predictor-corrector interior-point algorithm for linear programming. Numerical Algebra, Control and Optimization, 2019, 9 (2) : 147-156. doi: 10.3934/naco.2019011
[13]

Yongjian Yang, Zhiyou Wu, Fusheng Bai. A filled function method for constrained nonlinear integer programming. Journal of Industrial and Management Optimization, 2008, 4 (2) : 353-362. doi: 10.3934/jimo.2008.4.353
[14]

Adil Bagirov, Sona Taheri, Soodabeh Asadi. A difference of convex optimization algorithm for piecewise linear regression. Journal of Industrial and Management Optimization, 2019, 15 (2) : 909-932. doi: 10.3934/jimo.2018077
[15]

Ayache Benhadid, Fateh Merahi. Complexity analysis of an interior-point algorithm for linear optimization based on a new parametric kernel function with a double barrier term. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022003
[16]

Zhenbo Wang, Shu-Cherng Fang, David Y. Gao, Wenxun Xing. Global extremal conditions for multi-integer quadratic programming. Journal of Industrial and Management Optimization, 2008, 4 (2) : 213-225. doi: 10.3934/jimo.2008.4.213
[17]

Jing Quan, Zhiyou Wu, Guoquan Li. Global optimality conditions for some classes of polynomial integer programming problems. Journal of Industrial and Management Optimization, 2011, 7 (1) : 67-78. doi: 10.3934/jimo.2011.7.67
[18]

Yasmine Cherfaoui, Mustapha Moulaï. Biobjective optimization over the efficient set of multiobjective integer programming problem. Journal of Industrial and Management Optimization, 2021, 17 (1) : 117-131. doi: 10.3934/jimo.2019102
[19]

Mohamed A. Tawhid, Ahmed F. Ali. A simplex grey wolf optimizer for solving integer programming and minimax problems. Numerical Algebra, Control and Optimization, 2017, 7 (3) : 301-323. doi: 10.3934/naco.2017020
[20]

Lican Kang, Yanming Lai, Yanyan Liu, Yuan Luo, Jing Zhang. High-dimensional linear regression with hard thresholding regularization: Theory and algorithm. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022034

2021 Impact Factor: 1.411

Tools

Metrics

  • PDF downloads (269)
  • HTML views (608)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy