American Institute of Mathematical Sciences

Advanced
May  2018, 1(2): 181-200. doi: 10.3934/mfc.2018009

Hybrid binary dragonfly enhanced particle swarm optimization algorithm for solving feature selection problems

Mohamed A. Tawhid 1,2,, and  Kevin B. Dsouza 3,

1. 

Department of Mathematics and Statistics, Faculty of Science, Thompson Rivers University, Kamloops, BC, V2C 0C8, Canada
2. 

Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Moharam Bey 21511, Alexandria, Egyp
3. 

Electrical and Computer Engineering, The University of British Columbia, Vancouver BC V6T 1Z4, Canada

* Corresponding author: Mohamed A. Tawhid

Received  November 2017 Revised  January 2018 Published  May 2018

Fund Project: We are grateful to the anonymous 4 reviewers for constructive feedback and insightful suggestions which greatly improved this article. This research was supported partially by Mitacs Canada. The research of the 1st author is supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC).

In this paper, we present a new hybrid binary version of dragonfly and enhanced particle swarm optimization algorithm in order to solve feature selection problems. The proposed algorithm is called Hybrid Binary Dragonfly Enhanced Particle Swarm Optimization Algorithm(HBDESPO). In the proposed HBDESPO algorithm, we combine the dragonfly algorithm with its ability to encourage diverse solutions with its formation of static swarms and the enhanced version of the particle swarm optimization exploiting the data with its ability to converge to the best global solution in the search space. In order to investigate the general performance of the proposed HBDESPO algorithm, the proposed algorithm is compared with the original optimizers and other optimizers that have been used for feature selection in the past. Further, we use a set of assessment indicators to evaluate and compare the different optimizers over 20 standard data sets obtained from the UCI repository. Results prove the ability of the proposed HBDESPO algorithm to search the feature space for optimal feature combinations.

Keywords: Particle swarm optimization, dragonfly algorithm, binary algorithms, hybridization, meta-heuristics, feature selection problems.
Mathematics Subject Classification: Primary: 68T20, 68T01, 68U20; Secondary: 62P10, 62P30.
Citation: Mohamed A. Tawhid, Kevin B. Dsouza. Hybrid binary dragonfly enhanced particle swarm optimization algorithm for solving feature selection problems. Mathematical Foundations of Computing, 2018, 1 (2) : 181-200. doi: 10.3934/mfc.2018009
References:
[1]

D. K. Agrafiotis and W. Cedeno, Feature selection for structure-activity correlation using binary particle swarms, Journal of Medicinal Chemistry, 45 (2002), 1098-1107.   Google Scholar

[2]

H. Banati and M. Bajaj, Fire fly based feature selection approach, IJCSI International Journal of Computer Science Issues, 8 (2011). Google Scholar

[3]

D. Bell and H. Wang, A formalism for relevance and its application in feature subset selection, Mach. Learn., 41 (2000), 175-195.   Google Scholar

[4]

B. Xue, M. Zhang, W. Browne and X. Yao, A survey on evolutionary computation approaches to feature selection, IEEE Transaction on Evolutionary Computation, 20 (2016), 606-626.  doi: 10.1109/TEVC.2015.2504420.  Google Scholar

[5]

G. Chandrashekar and F. Sahin, A survey on feature selection methods, Electrical and Microelectronic Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA, 2013. Google Scholar

[6]

B. Chizi, L. Rokach and O. Maimon, A survey of feature selection techniques, Encyclopedia of Data Warehousing and Mining, seconded, IGI Global, (2009), 1888-1895.   Google Scholar

[7]

L. Y. Chuang, H. W. Chang, C. J. Tu and C. H. Yang, Improved binary PSO for feature selection using gene expression data, Comput.Biol.Chem., 32 (2008), 29-38.   Google Scholar

[8]

G. Coath and S. K. Halgamuge, A comparison of constraint-handling methods for the application of particle swarm optimization to constrained nonlinear optimization problems, Proceedings of IEEE Congress on Evolutionary Computation 2003 (CEC 2003), Canbella, Australia, (2003), 2419–2425. Google Scholar

[9]

C. A. Coello Coello, E. H. Luna and A. H. Aguirre, Use of particle swarm optimization to design combinational logic circuits, International Conference on Evolvable Systems, (2003), 398-409.  doi: 10.1007/3-540-36553-2_36.  Google Scholar

[10]

C. Cotta, A study of hybridisation techniques and their application to the design of evolutionary algorithms, AI Communications, 11 (1998), 223-224.   Google Scholar

[11]

R. C. Eberhart and J. Kennedy, A new optimizer using particle swarm theory. Proceedings of the Sixth International Symposium on Micromachine and Human Science, Nagoya, Japan, (1995), 39–43. Google Scholar

[12]

E. Emary, H. M. Zawbaa, C. Grosan and A. E. Hassanien, Binary grey wolf optimization approaches for feature selection, Neurocomputing, Elsevier, 172 (2016), 371-381.   Google Scholar

[13]

A. Frank and A. Asuncion, UCI Machine Learning Repository, 2010. Google Scholar

[14]

J. Huang, Y. Cai and X. Xu, A hybrid genetic algorithm for feature selection wrapper based on mutual information, Pattern Recognition Letters archive, 28 (2007), 1825-1844.  doi: 10.1016/j.patrec.2007.05.011.  Google Scholar

[15]

J. Kennedy, R. C. Eberhart and Y. Shi, Swarm Intelligence, Morgan Kaufmann, SanMateo, CA, 2001. Google Scholar

[16]

S. Khalid, A survey of feature selection and feature extraction techniques in machine learning, Science and Information Conference (SAI), 2014. Google Scholar

[17]

R. A. Krohling, H. Knidel and Y. Shi, Solving numerical equations of hydraulic problems using particle swarm optimization, Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2002), Honolulu, Hawaii USA, 2002. Google Scholar

[18]

S. Mirjalili and A. Lewis, S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization, Swarm and Evolutionary Computation, 9 (2012), 1-14.   Google Scholar

[19]

S. Mirjalili, Dragonfly algorithm: A new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems, Neural Computing and Applications, 27 (2016), 1053-1073.   Google Scholar

[20]

R. Y. M. Nakamura, L. A. M. Pereira, K. A. Costa, D. Rodrigues, J. P. Papa and X.-S. Yang, Binary bat algorithm for feature selection, Conference on Graphics, Patterns and Images, (2012), 291-297.   Google Scholar

[21]

Q. Gu, Z. Li and J. Han, Generalized Fisher Score for Feature Selection, In Proc. of the 27th Conference on Uncertainty in Artificial Intelligence (UAI), Barcelona, Spain, 2011. Google Scholar

[22]

E. G. Talbi, A taxonomy of hybrid metaheuristics, Journal of Heuristics, 8 (2002), 541-565.   Google Scholar

[23]

D. Wolpert and W. Macready, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation, 1 (1997), 67-72.   Google Scholar

show all references

References:
[1]

D. K. Agrafiotis and W. Cedeno, Feature selection for structure-activity correlation using binary particle swarms, Journal of Medicinal Chemistry, 45 (2002), 1098-1107.   Google Scholar

[2]

H. Banati and M. Bajaj, Fire fly based feature selection approach, IJCSI International Journal of Computer Science Issues, 8 (2011). Google Scholar

[3]

D. Bell and H. Wang, A formalism for relevance and its application in feature subset selection, Mach. Learn., 41 (2000), 175-195.   Google Scholar

[4]

B. Xue, M. Zhang, W. Browne and X. Yao, A survey on evolutionary computation approaches to feature selection, IEEE Transaction on Evolutionary Computation, 20 (2016), 606-626.  doi: 10.1109/TEVC.2015.2504420.  Google Scholar

[5]

G. Chandrashekar and F. Sahin, A survey on feature selection methods, Electrical and Microelectronic Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA, 2013. Google Scholar

[6]

B. Chizi, L. Rokach and O. Maimon, A survey of feature selection techniques, Encyclopedia of Data Warehousing and Mining, seconded, IGI Global, (2009), 1888-1895.   Google Scholar

[7]

L. Y. Chuang, H. W. Chang, C. J. Tu and C. H. Yang, Improved binary PSO for feature selection using gene expression data, Comput.Biol.Chem., 32 (2008), 29-38.   Google Scholar

[8]

G. Coath and S. K. Halgamuge, A comparison of constraint-handling methods for the application of particle swarm optimization to constrained nonlinear optimization problems, Proceedings of IEEE Congress on Evolutionary Computation 2003 (CEC 2003), Canbella, Australia, (2003), 2419–2425. Google Scholar

[9]

C. A. Coello Coello, E. H. Luna and A. H. Aguirre, Use of particle swarm optimization to design combinational logic circuits, International Conference on Evolvable Systems, (2003), 398-409.  doi: 10.1007/3-540-36553-2_36.  Google Scholar

[10]

C. Cotta, A study of hybridisation techniques and their application to the design of evolutionary algorithms, AI Communications, 11 (1998), 223-224.   Google Scholar

[11]

R. C. Eberhart and J. Kennedy, A new optimizer using particle swarm theory. Proceedings of the Sixth International Symposium on Micromachine and Human Science, Nagoya, Japan, (1995), 39–43. Google Scholar

[12]

E. Emary, H. M. Zawbaa, C. Grosan and A. E. Hassanien, Binary grey wolf optimization approaches for feature selection, Neurocomputing, Elsevier, 172 (2016), 371-381.   Google Scholar

[13]

A. Frank and A. Asuncion, UCI Machine Learning Repository, 2010. Google Scholar

[14]

J. Huang, Y. Cai and X. Xu, A hybrid genetic algorithm for feature selection wrapper based on mutual information, Pattern Recognition Letters archive, 28 (2007), 1825-1844.  doi: 10.1016/j.patrec.2007.05.011.  Google Scholar

[15]

J. Kennedy, R. C. Eberhart and Y. Shi, Swarm Intelligence, Morgan Kaufmann, SanMateo, CA, 2001. Google Scholar

[16]

S. Khalid, A survey of feature selection and feature extraction techniques in machine learning, Science and Information Conference (SAI), 2014. Google Scholar

[17]

R. A. Krohling, H. Knidel and Y. Shi, Solving numerical equations of hydraulic problems using particle swarm optimization, Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2002), Honolulu, Hawaii USA, 2002. Google Scholar

[18]

S. Mirjalili and A. Lewis, S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization, Swarm and Evolutionary Computation, 9 (2012), 1-14.   Google Scholar

[19]

S. Mirjalili, Dragonfly algorithm: A new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems, Neural Computing and Applications, 27 (2016), 1053-1073.   Google Scholar

[20]

R. Y. M. Nakamura, L. A. M. Pereira, K. A. Costa, D. Rodrigues, J. P. Papa and X.-S. Yang, Binary bat algorithm for feature selection, Conference on Graphics, Patterns and Images, (2012), 291-297.   Google Scholar

[21]

Q. Gu, Z. Li and J. Han, Generalized Fisher Score for Feature Selection, In Proc. of the 27th Conference on Uncertainty in Artificial Intelligence (UAI), Barcelona, Spain, 2011. Google Scholar

[22]

E. G. Talbi, A taxonomy of hybrid metaheuristics, Journal of Heuristics, 8 (2002), 541-565.   Google Scholar

[23]

D. Wolpert and W. Macready, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation, 1 (1997), 67-72.   Google Scholar

Figure 1.  The Comparison of performance the HBDEPSO algorithm with other optimizers through main objectives of feature selection. The values are averaged over all the datasets
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  The Comparison of performance the HBDEPSO algorithm with other optimizers through few assessment indicators. The values are averaged over all the datasets
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  Datasets
Dataset# of Attributes# of Instances
Zoo16101
WineEW13178
IonosphereEW34351
WaveformEW405000
BreastEW30569
Breastcancer9699
Congress16435
Exactly131000
Exactly2131000
HeartEW13270
KrvskpEW363196
M-of-n131000
SonarEW60208
SpectEW60208
Tic-tac-toe9958
Lymphography18148
Dermatology34366
Echocardiogram12132
hepatitis19155
LungCancer5632
Table Options

Download as excel

Dataset# of Attributes# of Instances
Zoo16101
WineEW13178
IonosphereEW34351
WaveformEW405000
BreastEW30569
Breastcancer9699
Congress16435
Exactly131000
Exactly2131000
HeartEW13270
KrvskpEW363196
M-of-n131000
SonarEW60208
SpectEW60208
Tic-tac-toe9958
Lymphography18148
Dermatology34366
Echocardiogram12132
hepatitis19155
LungCancer5632
Table 2.  Parameter setting
Parameter Value
No of iterations($max_{iter}$)70
No of search agents($n$)5
Dimension($D$)No. of features in the data
Search domain[0 1]
No of runs($M$)10
$w_{max}$0.9
$w_{min}$0.4
$Deltax_{max}$6
$c_1$2
$c_2$2
$v_{max}$6
$\beta$ in fitness function0.01
$\alpha$ in fitness function0.99
Table Options

Download as excel

Parameter Value
No of iterations($max_{iter}$)70
No of search agents($n$)5
Dimension($D$)No. of features in the data
Search domain[0 1]
No of runs($M$)10
$w_{max}$0.9
$w_{min}$0.4
$Deltax_{max}$6
$c_1$2
$c_2$2
$v_{max}$6
$\beta$ in fitness function0.01
$\alpha$ in fitness function0.99
Table 3.  Mean fitness function obtained from the different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.0400.067 0.0310.1240.0940.1190.082
Wine EW 0.0360.0500.0420.0650.1280.0920.041
IonosphereEW 0.1100.1300.1370.1430.1460.1720.115
WaveformEW0.1790.183 0.1750.1860.1930.1850.175
BreastEW 0.0400.0570.0500.1060.0700.0800.044
Breastcancer 0.0230.0320.0320.0360.0350.0420.030
Congress 0.0280.0420.0330.0590.0530.0730.036
Exactly 0.1030.1780.1040.2690.3030.3160.139
Exactly2 0.2240.2400.2340.2430.2430.2630.241
HeartEW 0.1250.1530.1530.2500.2400.2680.128
KrvskpEW0.0440.0410.0430.0890.1080.080 0.039
M-of-n0.0250.048 0.0240.1080.1670.1540.084
SonarEW 0.1580.1940.1920.2620.2770.2900.179
SpectEW0.148 0.1330.1600.1680.1670.2050.142
Tic-tac-toe 0.2220.2230.2220.2410.2700.2620.227
Lymphography 0.3810.3920.4120.4660.4870.5310.426
Dermatology 0.0160.0170.0160.0310.0810.0990.017
Echocardiogram 0.0510.0580.0830.0720.1120.2000.074
Hepatitis0.118 0.1010.1230.1520.1750.1920.115
LungCancer 0.2190.2550.2200.3180.4270.4550.291
Average 0.1140.1310.1230.1690.1890.2040.131
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.0400.067 0.0310.1240.0940.1190.082
Wine EW 0.0360.0500.0420.0650.1280.0920.041
IonosphereEW 0.1100.1300.1370.1430.1460.1720.115
WaveformEW0.1790.183 0.1750.1860.1930.1850.175
BreastEW 0.0400.0570.0500.1060.0700.0800.044
Breastcancer 0.0230.0320.0320.0360.0350.0420.030
Congress 0.0280.0420.0330.0590.0530.0730.036
Exactly 0.1030.1780.1040.2690.3030.3160.139
Exactly2 0.2240.2400.2340.2430.2430.2630.241
HeartEW 0.1250.1530.1530.2500.2400.2680.128
KrvskpEW0.0440.0410.0430.0890.1080.080 0.039
M-of-n0.0250.048 0.0240.1080.1670.1540.084
SonarEW 0.1580.1940.1920.2620.2770.2900.179
SpectEW0.148 0.1330.1600.1680.1670.2050.142
Tic-tac-toe 0.2220.2230.2220.2410.2700.2620.227
Lymphography 0.3810.3920.4120.4660.4870.5310.426
Dermatology 0.0160.0170.0160.0310.0810.0990.017
Echocardiogram 0.0510.0580.0830.0720.1120.2000.074
Hepatitis0.118 0.1010.1230.1520.1750.1920.115
LungCancer 0.2190.2550.2200.3180.4270.4550.291
Average 0.1140.1310.1230.1690.1890.2040.131
Table 4.  Best fitness function obtained from the different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo 0.0000.0000.0010.0320.0050.0350.004
Wine EW 0.0020.0030.0190.0350.0210.0030.019
IonosphereEW 0.0710.1080.1130.1140.0790.0890.096
WaveformEW0.1710.1810.1650.1740.1760.167 0.162
BreastEW 0.0250.0550.0270.0600.0450.0560.034
Breastcancer0.0140.0240.0180.0290.0240.0270.014
Congress 0.0160.0190.0220.0380.0290.0450.022
Exactly 0.0040.0400.0250.0580.2700.2980.025
Exactly2 0.2110.2350.2190.2160.2120.2410.220
HeartEW0.091 0.0820.1040.1470.1680.1470.082
KrvskpEW0.0410.0340.0330.0410.0600.059 0.029
M-of-n 0.0040.0040.0040.0670.1130.1280.004
SonarEW 0.1180.1560.1180.2200.2050.2340.134
SpectEW0.115 0.0930.1250.1250.1270.1610.115
Tic-tac-toe0.2130.206 0.1850.2170.2360.2420.196
Lymphography 0.2860.3440.3070.3880.4270.4500.349
Dermatology 0.0030.0030.0040.0120.0290.0460.004
Echocardiogram 0.0030.0250.0470.0450.0490.0930.047
Hepatitis 0.0580.0580.0800.0780.1170.0970.061
LungCancer0.093 0.0030.0580.0930.1840.280.094
Average 0.0770.0840.0840.1100.1290.1450.086
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo 0.0000.0000.0010.0320.0050.0350.004
Wine EW 0.0020.0030.0190.0350.0210.0030.019
IonosphereEW 0.0710.1080.1130.1140.0790.0890.096
WaveformEW0.1710.1810.1650.1740.1760.167 0.162
BreastEW 0.0250.0550.0270.0600.0450.0560.034
Breastcancer0.0140.0240.0180.0290.0240.0270.014
Congress 0.0160.0190.0220.0380.0290.0450.022
Exactly 0.0040.0400.0250.0580.2700.2980.025
Exactly2 0.2110.2350.2190.2160.2120.2410.220
HeartEW0.091 0.0820.1040.1470.1680.1470.082
KrvskpEW0.0410.0340.0330.0410.0600.059 0.029
M-of-n 0.0040.0040.0040.0670.1130.1280.004
SonarEW 0.1180.1560.1180.2200.2050.2340.134
SpectEW0.115 0.0930.1250.1250.1270.1610.115
Tic-tac-toe0.2130.206 0.1850.2170.2360.2420.196
Lymphography 0.2860.3440.3070.3880.4270.4500.349
Dermatology 0.0030.0030.0040.0120.0290.0460.004
Echocardiogram 0.0030.0250.0470.0450.0490.0930.047
Hepatitis 0.0580.0580.0800.0780.1170.0970.061
LungCancer0.093 0.0030.0580.0930.1840.280.094
Average 0.0770.0840.0840.1100.1290.1450.086
Table 5.  Worst fitness function obtained from the different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.1210.208 0.0890.2080.2080.2080.206
Wine EW 0.0690.1190.0700.1220.2730.1570.070
IonosphereEW 0.1460.1550.1710.1890.1910.3090.147
WaveformEW 0.1840.1920.1860.1970.2150.1950.186
BreastEW 0.0490.0650.0810.3150.1030.1150.054
Breastcancer 0.0310.0390.0410.0490.0490.0520.038
Congress 0.0430.0630.0490.0850.0920.0890.049
Exactly 0.2130.3080.2510.3490.3260.3420.294
Exactly2 0.2380.2630.2480.2680.2760.2860.265
HeartEW 0.1680.2010.2890.3220.3340.3570.168
KrvskpEW 0.0470.0520.0540.1770.1910.1010.063
M-of-n 0.0490.1360.0730.1570.2320.1700.461
SonarEW 0.1910.2340.2190.3060.3910.3490.262
SpectEW 0.1700.1700.2040.2050.2160.2380.192
Tic-tac-toe 0.2360.2390.2440.2750.3130.2980.243
Lymphography 0.4680.4910.4690.5880.5690.5810.549
Dermatology 0.0290.0530.0290.0610.2900.2220.030
Echocardiogram 0.0700.0920.160.1140.230.8400.115
Hepatitis0.230 0.1380.1740.2120.2340.2530.175
LungCancer0.542 0.4540.5430.7230.8130.5450.722
Average 0.1650.1840.1820.2460.2770.2850.214
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.1210.208 0.0890.2080.2080.2080.206
Wine EW 0.0690.1190.0700.1220.2730.1570.070
IonosphereEW 0.1460.1550.1710.1890.1910.3090.147
WaveformEW 0.1840.1920.1860.1970.2150.1950.186
BreastEW 0.0490.0650.0810.3150.1030.1150.054
Breastcancer 0.0310.0390.0410.0490.0490.0520.038
Congress 0.0430.0630.0490.0850.0920.0890.049
Exactly 0.2130.3080.2510.3490.3260.3420.294
Exactly2 0.2380.2630.2480.2680.2760.2860.265
HeartEW 0.1680.2010.2890.3220.3340.3570.168
KrvskpEW 0.0470.0520.0540.1770.1910.1010.063
M-of-n 0.0490.1360.0730.1570.2320.1700.461
SonarEW 0.1910.2340.2190.3060.3910.3490.262
SpectEW 0.1700.1700.2040.2050.2160.2380.192
Tic-tac-toe 0.2360.2390.2440.2750.3130.2980.243
Lymphography 0.4680.4910.4690.5880.5690.5810.549
Dermatology 0.0290.0530.0290.0610.2900.2220.030
Echocardiogram 0.0700.0920.160.1140.230.8400.115
Hepatitis0.230 0.1380.1740.2120.2340.2530.175
LungCancer0.542 0.4540.5430.7230.8130.5450.722
Average 0.1650.1840.1820.2460.2770.2850.214
Table 6.  Standard deviation of the fitness function obtained from the different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.0520.075 0.0330.0660.0700.0670.056
Wine EW0.0190.0300.0180.0260.0800.057 0.017
IonosphereEW0.0220.0180.0160.0250.0400.057 0.013
WaveformEW 0.0030.0060.0080.0080.01230.0070.006
BreastEW 0.0060.0070.0190.7550.0170.0180.006
Breastcancer 0.0050.0050.0070.0070.0090.0080.009
Congress 0.0070.0160.0080.0130.0190.0150.008
Exactly0.0710.1190.0820.0780.020 0.0160.117
Exactly2 0.0090.0150.0090.0190.0170.0180.015
HeartEW 0.0250.0360.0550.0620.0640.0690.025
KrvskpEW 0.0020.0070.0070.0510.0440.0120.010
M-of-n 0.0180.0510.0220.0320.0360.0190.136
SonarEW 0.0270.0330.0290.0300.0590.0430.037
SpectEW 0.0160.0220.0270.0290.0280.0240.029
Tic-tac-toe 0.0070.0120.0200.0210.0250.0170.014
Lymphography0.0520.0490.0480.0620.047 0.0440.055
Dermatology 0.0070.0140.0080.0140.0750.0500.008
Echocardiogram 0.0240.0260.0300.0240.0550.2280.025
Hepatitis0.050 0.0250.0280.0380.0430.0520.030
LungCancer 0.0920.1510.1800.2330.1940.0930.183
Average 0.0260.0360.0330.0800.0480.0460.040
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.0520.075 0.0330.0660.0700.0670.056
Wine EW0.0190.0300.0180.0260.0800.057 0.017
IonosphereEW0.0220.0180.0160.0250.0400.057 0.013
WaveformEW 0.0030.0060.0080.0080.01230.0070.006
BreastEW 0.0060.0070.0190.7550.0170.0180.006
Breastcancer 0.0050.0050.0070.0070.0090.0080.009
Congress 0.0070.0160.0080.0130.0190.0150.008
Exactly0.0710.1190.0820.0780.020 0.0160.117
Exactly2 0.0090.0150.0090.0190.0170.0180.015
HeartEW 0.0250.0360.0550.0620.0640.0690.025
KrvskpEW 0.0020.0070.0070.0510.0440.0120.010
M-of-n 0.0180.0510.0220.0320.0360.0190.136
SonarEW 0.0270.0330.0290.0300.0590.0430.037
SpectEW 0.0160.0220.0270.0290.0280.0240.029
Tic-tac-toe 0.0070.0120.0200.0210.0250.0170.014
Lymphography0.0520.0490.0480.0620.047 0.0440.055
Dermatology 0.0070.0140.0080.0140.0750.0500.008
Echocardiogram 0.0240.0260.0300.0240.0550.2280.025
Hepatitis0.050 0.0250.0280.0380.0430.0520.030
LungCancer 0.0920.1510.1800.2330.1940.0930.183
Average 0.0260.0360.0330.0800.0480.0460.040
Table 7.  Average performance of the selected features by different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.8440.7880.791 0.8630.7990.8510.852
Wine EW0.916 0.9230.8810.8860.7260.8960.888
IonosphereEW 0.8350.7990.8290.8280.8170.8240.810
WaveformEW 0.8230.8070.8090.8060.7790.8190.806
BreastEW 0.9490.9440.9310.8920.8420.9080.926
Breastcancer 0.9600.9560.9560.9570.9570.9570.958
Congress 0.9450.9310.9430.9150.8930.9280.935
Exactly 0.8950.7980.8840.6870.6470.6800.846
Exactly2 0.7460.7390.7380.7340.7110.7320.736
HeartEW 0.8150.810.7760.7110.6480.7020.811
KrvskpEW 0.9590.9540.9580.9060.7720.9170.958
M-of-n 0.9780.9490.9750.8920.7190.8430.957
SonarEW 0.7050.6580.6820.6940.6780.6820.682
SpectEW0.7620.7520.7570.7500.755 0.7770.747
Tic-tac-toe 0.7480.7450.7400.7340.6470.7130.737
Lymphography0.4060.4170.3540.416 0.4220.3790.411
Dermatology 0.9580.9400.9520.950.8020.9080.945
Echocardiogram0.8750.893 0.9060.8520.8610.8770.863
Hepatitis 0.8190.7880.8130.7980.7880.7880.803
LungCancer 0.4810.4270.3900.4090.3430.3450.345
Average 0.8210.8010.8030.7840.7300.7760.801
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo0.8440.7880.791 0.8630.7990.8510.852
Wine EW0.916 0.9230.8810.8860.7260.8960.888
IonosphereEW 0.8350.7990.8290.8280.8170.8240.810
WaveformEW 0.8230.8070.8090.8060.7790.8190.806
BreastEW 0.9490.9440.9310.8920.8420.9080.926
Breastcancer 0.9600.9560.9560.9570.9570.9570.958
Congress 0.9450.9310.9430.9150.8930.9280.935
Exactly 0.8950.7980.8840.6870.6470.6800.846
Exactly2 0.7460.7390.7380.7340.7110.7320.736
HeartEW 0.8150.810.7760.7110.6480.7020.811
KrvskpEW 0.9590.9540.9580.9060.7720.9170.958
M-of-n 0.9780.9490.9750.8920.7190.8430.957
SonarEW 0.7050.6580.6820.6940.6780.6820.682
SpectEW0.7620.7520.7570.7500.755 0.7770.747
Tic-tac-toe 0.7480.7450.7400.7340.6470.7130.737
Lymphography0.4060.4170.3540.416 0.4220.3790.411
Dermatology 0.9580.9400.9520.950.8020.9080.945
Echocardiogram0.8750.893 0.9060.8520.8610.8770.863
Hepatitis 0.8190.7880.8130.7980.7880.7880.803
LungCancer 0.4810.4270.3900.4090.3430.3450.345
Average 0.8210.8010.8030.7840.7300.7760.801
Table 8.  Average selected feature ratio by different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo 0.2930.3310.3560.4120.5120.4730.4
Wine EW 0.2840.3380.40.3150.5380.5160.338
IonosphereEW 0.3670.3970.3880.4020.5260.5410.397
WaveformEW 0.6330.6660.7090.6760.63410.752
BreastEW 0.2410.2830.2410.2900.4800.4700.3
Breastcancer 0.4110.4220.5110.5660.5110.6440.544
Congress 0.3060.3370.3250.4120.4930.5750.318
Exactly 0.4690.5070.5070.5610.5380.5760.523
Exactly2 0.3920.3920.4920.40.5460.80.415
HeartEW 0.3910.4070.4070.4150.4920.4300.4
KrvskpEW0.486 0.4750.5020.5300.5130.6330.516
M-of-n0.5150.5300.4760.576 0.4460.9230.515
SonarEW0.44 0.4130.4630.420.5210.5330.475
SpectEW 0.4130.4540.4630.4250.4810.5290.440
Tic-tac-toe0.5550.5550.666 0.5110.5770.8660.533
Lymphography 0.390.4380.4160.40.4610.5350.45
Dermatology0.5 0.4110.5110.4790.4940.5440.5
Echocardiogram 0.2250.2330.2660.2830.5080.4830.25
Hepatitis0.2730.2730.321 0.2310.5150.4310.294
LungCancer 0.350.3530.4230.3800.4980.5260.357
Average 0.3970.4110.4420.4340.5140.6010.436
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo 0.2930.3310.3560.4120.5120.4730.4
Wine EW 0.2840.3380.40.3150.5380.5160.338
IonosphereEW 0.3670.3970.3880.4020.5260.5410.397
WaveformEW 0.6330.6660.7090.6760.63410.752
BreastEW 0.2410.2830.2410.2900.4800.4700.3
Breastcancer 0.4110.4220.5110.5660.5110.6440.544
Congress 0.3060.3370.3250.4120.4930.5750.318
Exactly 0.4690.5070.5070.5610.5380.5760.523
Exactly2 0.3920.3920.4920.40.5460.80.415
HeartEW 0.3910.4070.4070.4150.4920.4300.4
KrvskpEW0.486 0.4750.5020.5300.5130.6330.516
M-of-n0.5150.5300.4760.576 0.4460.9230.515
SonarEW0.44 0.4130.4630.420.5210.5330.475
SpectEW 0.4130.4540.4630.4250.4810.5290.440
Tic-tac-toe0.5550.5550.666 0.5110.5770.8660.533
Lymphography 0.390.4380.4160.40.4610.5350.45
Dermatology0.5 0.4110.5110.4790.4940.5440.5
Echocardiogram 0.2250.2330.2660.2830.5080.4830.25
Hepatitis0.2730.2730.321 0.2310.5150.4310.294
LungCancer 0.350.3530.4230.3800.4980.5260.357
Average 0.3970.4110.4420.4340.5140.6010.436
Table 9.  Average Fischer index of the selected features by different algorithms
Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo 161105143156112130140
Wine EW24.13374.67648.48540.5211784.7 20939.741.169
IonosphereEW3.6023.9863.8704.7694.154 5.0424.056
WaveformEW2.3552.3142.3142.1652.029 3.4562.454
BreastEW 7.2E+132.5E+113.3E+131.4E+135.7E+126.9E+133.1E+11
Breastcancer 1.1900.7481.0700.9230.9421.1050.884
Congress 48.58431.79713.99613.04511.00318.31722.088
Exactly 0.3910.2590.1310.3780.3500.2820.144
Exactly2 0.3950.2400.2670.2870.2000.2370.227
HeartEW3.7883.4243.357140.64161.62 430.072.197
KrvskpEW 1396.5544.21940.241023.2639.911187.5913.89
M-of-n 1.7911.7111.7351.7861.6521.3731.693
SonarEW6.4E+67.3E+68.2E+65.5E+68.2E+6 1.2E+79.5E+6
SpectEW 0.0080.0060.0050.0040.0060.0060.006
Tic-tac-toe 0.1680.0900.1610.1190.1360.1170.134
Lymphography 9.773.139.182.434.412.733.51
Dermatology 400269343148210174207
Echocardiogram158.28579.06137662931 13093953037985.60
Hepatitis5.9633.49151.8014.420132.03 5303784.211
LungCancer 42.97331.14840.20329.40530.81033.61522.220
Average 3.6E+121.3E+101.6E+126.9E+112.8E+113.4E+111.5E+10
Table Options

Download as excel

Dataset HBDESPO BDA EPSO BGA BBA BGWO2 HBEPSOD
Zoo 161105143156112130140
Wine EW24.13374.67648.48540.5211784.7 20939.741.169
IonosphereEW3.6023.9863.8704.7694.154 5.0424.056
WaveformEW2.3552.3142.3142.1652.029 3.4562.454
BreastEW 7.2E+132.5E+113.3E+131.4E+135.7E+126.9E+133.1E+11
Breastcancer 1.1900.7481.0700.9230.9421.1050.884
Congress 48.58431.79713.99613.04511.00318.31722.088
Exactly 0.3910.2590.1310.3780.3500.2820.144
Exactly2 0.3950.2400.2670.2870.2000.2370.227
HeartEW3.7883.4243.357140.64161.62 430.072.197
KrvskpEW 1396.5544.21940.241023.2639.911187.5913.89
M-of-n 1.7911.7111.7351.7861.6521.3731.693
SonarEW6.4E+67.3E+68.2E+65.5E+68.2E+6 1.2E+79.5E+6
SpectEW 0.0080.0060.0050.0040.0060.0060.006
Tic-tac-toe 0.1680.0900.1610.1190.1360.1170.134
Lymphography 9.773.139.182.434.412.733.51
Dermatology 400269343148210174207
Echocardiogram158.28579.06137662931 13093953037985.60
Hepatitis5.9633.49151.8014.420132.03 5303784.211
LungCancer 42.97331.14840.20329.40530.81033.61522.220
Average 3.6E+121.3E+101.6E+126.9E+112.8E+113.4E+111.5E+10
[1]

T. W. Leung, Chi Kin Chan, Marvin D. Troutt. A mixed simulated annealing-genetic algorithm approach to the multi-buyer multi-item joint replenishment problem: advantages of meta-heuristics. Journal of Industrial & Management Optimization, 2008, 4 (1) : 53-66. doi: 10.3934/jimo.2008.4.53
[2]

Miao Yu. A solution of TSP based on the ant colony algorithm improved by particle swarm optimization. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 979-987. doi: 10.3934/dcdss.2019066
[3]

Tao Zhang, Yue-Jie Zhang, Qipeng P. Zheng, P. M. Pardalos. A hybrid particle swarm optimization and tabu search algorithm for order planning problems of steel factories based on the Make-To-Stock and Make-To-Order management architecture. Journal of Industrial & Management Optimization, 2011, 7 (1) : 31-51. doi: 10.3934/jimo.2011.7.31
[4]

Mohammed Abdulrazaq Kahya, Suhaib Abduljabbar Altamir, Zakariya Yahya Algamal. Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numerical Algebra, Control & Optimization, 2019, 0 (0) : 0-0. doi: 10.3934/naco.2020017
[5]

Min Zhang, Gang Li. Multi-objective optimization algorithm based on improved particle swarm in cloud computing environment. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1413-1426. doi: 10.3934/dcdss.2019097
[6]

Junyuan Lin, Timothy A. Lucas. A particle swarm optimization model of emergency airplane evacuations with emotion. Networks & Heterogeneous Media, 2015, 10 (3) : 631-646. doi: 10.3934/nhm.2015.10.631
[7]

Qifeng Cheng, Xue Han, Tingting Zhao, V S Sarma Yadavalli. Improved particle swarm optimization and neighborhood field optimization by introducing the re-sampling step of particle filter. Journal of Industrial & Management Optimization, 2019, 15 (1) : 177-198. doi: 10.3934/jimo.2018038
[8]

Ning Lu, Ying Liu. Application of support vector machine model in wind power prediction based on particle swarm optimization. Discrete & Continuous Dynamical Systems - S, 2015, 8 (6) : 1267-1276. doi: 10.3934/dcdss.2015.8.1267
[9]

Xia Zhao, Jianping Dou. Bi-objective integrated supply chain design with transportation choices: A multi-objective particle swarm optimization. Journal of Industrial & Management Optimization, 2019, 15 (3) : 1263-1288. doi: 10.3934/jimo.2018095
[10]

Junjie Peng, Ning Chen, Jiayang Dai, Weihua Gui. A goethite process modeling method by asynchronous fuzzy cognitive Network based on an improved constrained chicken swarm optimization algorithm. Journal of Industrial & Management Optimization, 2017, 13 (5) : 0-0. doi: 10.3934/jimo.2020021
[11]

Jianguo Dai, Wenxue Huang, Yuanyi Pan. A category-based probabilistic approach to feature selection. Big Data & Information Analytics, 2017, 2 (5) : 1-8. doi: 10.3934/bdia.2017020
[12]

Yunmei Lu, Mingyuan Yan, Meng Han, Qingliang Yang, Yanqing Zhang. Privacy preserving feature selection and Multiclass Classification for horizontally distributed data. Mathematical Foundations of Computing, 2018, 1 (4) : 331-348. doi: 10.3934/mfc.2018016
[13]

Nguyen Van Thoai. Decomposition branch and bound algorithm for optimization problems over efficient sets. Journal of Industrial & Management Optimization, 2008, 4 (4) : 647-660. doi: 10.3934/jimo.2008.4.647
[14]

Tran Ngoc Thang, Nguyen Thi Bach Kim. Outcome space algorithm for generalized multiplicative problems and optimization over the efficient set. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1417-1433. doi: 10.3934/jimo.2016.12.1417
[15]

Paul B. Hermanns, Nguyen Van Thoai. Global optimization algorithm for solving bilevel programming problems with quadratic lower levels. Journal of Industrial & Management Optimization, 2010, 6 (1) : 177-196. doi: 10.3934/jimo.2010.6.177
[16]

Mohamed A. Tawhid, Ahmed F. Ali. An effective hybrid firefly algorithm with the cuckoo search for engineering optimization problems. Mathematical Foundations of Computing, 2018, 1 (4) : 349-368. doi: 10.3934/mfc.2018017
[17]

Feng Bao, Thomas Maier. Stochastic gradient descent algorithm for stochastic optimization in solving analytic continuation problems. Foundations of Data Science, 2020, 2 (1) : 1-17. doi: 10.3934/fods.2020001
[18]

Yuying Shi, Ying Gu, Li-Lian Wang, Xue-Cheng Tai. A fast edge detection algorithm using binary labels. Inverse Problems & Imaging, 2015, 9 (2) : 551-578. doi: 10.3934/ipi.2015.9.551
[19]

Mostafa Abouei Ardakan, A. Kourank Beheshti, S. Hamid Mirmohammadi, Hamed Davari Ardakani. A hybrid meta-heuristic algorithm to minimize the number of tardy jobs in a dynamic two-machine flow shop problem. Numerical Algebra, Control & Optimization, 2017, 7 (4) : 465-480. doi: 10.3934/naco.2017029
[20]

Dmitri E. Kvasov, Yaroslav D. Sergeyev. Univariate geometric Lipschitz global optimization algorithms. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 69-90. doi: 10.3934/naco.2012.2.69

 Impact Factor: 

Tools

Article outline

Figures and Tables

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences