February  2019, 2(1): 73-81. doi: 10.3934/mfc.2019006

A classification algorithm with Linear Discriminant Analysis and Axiomatic Fuzzy Sets

1. 

School of Control Science and Engineering, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, China

2. 

Department of Electrical and Computer Engineering, University of Alberta, Edmonton, T6G 2G7, Canada

* Corresponding author: Xiaodong Liu

Published  March 2019

Fund Project: The work is supported by National Natural Science Foundation of China under grants 61673082 and 61533005.

In exploratory data mining, most classifiers pay more attention on the accuracy and speed of learned models, but they are lacking of the interpretability. In this paper, an interpretable and comprehensible classifier is proposed based on Linear Discriminant Analysis (LDA) and Axiomatic Fuzzy Sets (AFS). The algorithm utilizes LDA to extract features with the largest inter-class variance. Besides, the proposed approach aims to explore a transformation from the selected feature space to a semantic space where the samples in the same class are made as close as possible to one another, whereas the samples in the different class are as far as possible from one another. Moreover, the descriptions of each class can be obtained by the proposed approach. When compared with well-known classifiers such as LogisticR, C4.5Tree, SVM and KNN, the proposed method not only can achieve better performance in terms of accuracy but also has the capability of interpretability and comprehension.

Citation: Wenjuan Jia, Yingjie Deng, Chenyang Xin, Xiaodong Liu, Witold Pedrycz. A classification algorithm with Linear Discriminant Analysis and Axiomatic Fuzzy Sets. Mathematical Foundations of Computing, 2019, 2 (1) : 73-81. doi: 10.3934/mfc.2019006
References:
[1]

W. BiM. CaiM. Liu and et al., A Big Data Clustering Algorithm for Mitigating the Risk of Customer Churn, IEEE Transactions on Industrial Informatics, 12 (2016), 1270-1281.   Google Scholar

[2]

A Das and A. Desarkar, Decision tree-based analytics for reducing air pollution, Journal of Information & Knowledge Management, 17 (2018), 1850015. Google Scholar

[3]

K. J. Dsouza and Z. Ansari, Experimental exploration of support vector machine for cancer cell classification, IEEE International Conference on Cloud Computing in Emerging Markets, 2017, 29–34. Google Scholar

[4]

D. Dua and E. Karra Taniskidou, UCI Machine Learning Repository, http://archive.ics.uci.edu/ml. Irvine, CA: University of California, School of Information and Computer Science, 2017. Google Scholar

[5]

L. I. JinmengY. Lin and T. Zhu, k-Nearest Neighbor Classification Algorithm Based on Hubness and Class Weighting, Computer Engineering, 44 (2018), 248-252.   Google Scholar

[6]

F. KhozeimehR. AlizadehsaniM. RoshanzamirA. khosravi and et al., An expert system for selecting wart treatment method, Computer in Biology and Medicine, 81 (2017), 167-175.   Google Scholar

[7]

F. KhozeimehF. Jabberi Azad and et al., Intralesional immunotherapy compared to cryotherapy in the treatment of warts, International Journal of Dermatology, 56 (2017), 474-478.   Google Scholar

[8]

Q. LiY. RenL. Li and W. Liu, From low-level geometric features to high-level semantics: An axiomatic fuzzy set clustering approach, Journal of Intelligent & Fuzzy Systems, 31 (2016), 775-786.  doi: 10.3233/JIFS-169009.  Google Scholar

[9]

Q. LiY. RenL. Li and et al., Fuzzy based affinity learning for spectral clustering., Pattern Recognition, 60 (2016), 531-542.   Google Scholar

[10]

Z. Li, Q. Zhang and X. Duan, et al., A novel semantic approach for multi-ethnic face recognition, International Journal of Pattern Recognition & Artificial Intelligence, 32 (2017), 15. Google Scholar

[11]

X. Liu, W. Jia and Y. Wang, et al., Knowledge discovery and semantic learning in the framework of axiomatic fuzzy set theory, Wires Data Mining And Knowledge Discovery, 8 (2018), e1268. doi: 10.1002/widm.1268.  Google Scholar

[12]

X. Liu, K. Zhu and H. Z. Huang, The representations of fuzzy concepts based on the fuzzy matrix theory and the AFS theory, IEEE International Symposium on Intelligent Control, 2003. Google Scholar

[13]

X. Liu, The fuzzy theory based on afs algebras and afs structure, Journal of Mathematical Analysis and Applications, 217 (1998), 459-478.  doi: 10.1006/jmaa.1997.5718.  Google Scholar

[14]

X. LiuW. WangT. Chai and et al., Approaches to the representations and logic operations of fuzzy concepts in the framework of axiomatic fuzzy set theory I, Information Sciences, 177 (2007), 1007-1026.  doi: 10.1016/j.ins.2006.07.011.  Google Scholar

[15]

X. Liu, The fuzzy sets and systems based on AFS structure, EI algebra and EII algebra, Fuzzy Sets & Systems, 95 (1998), 179-188.  doi: 10.1016/S0165-0114(96)00298-9.  Google Scholar

[16]

X. LiuX. Feng and W. Pedrycz, Extraction of fuzzy rules from fuzzy decision trees: An axiomatic fuzzy sets (AFS) approach, Data & Knowledge Engineering, 84 (2013), 1-25.  doi: 10.1016/j.datak.2012.12.001.  Google Scholar

[17]

Y. RenM. Song and X. Liu, New approaches to the fuzzy clustering via AFS theory, International Journal of Information and Systems Sciences, 3 (2007), 307-325.   Google Scholar

[18]

Y. RenQ. LiW. Liu and et al., Semantic facial descriptor extraction via Axiomatic Fuzzy Set, Neurocomputing, 171 (2016), 1462-1474.   Google Scholar

[19]

A. Swami and R. Jain, Scikit-learn: Machine learning in python, Journal of Machine Learning Research, 12 (2012), 2825-2830.   Google Scholar

[20]

W. Wang and X. Liu, Fuzzy forecasting based on automatic clustering and axiomatic fuzzy set classification, Information Sciences, 294 (2015), 78-94.  doi: 10.1016/j.ins.2014.09.027.  Google Scholar

[21]

I. H. Witten, E. Frank, M. A. Hall and C. J. Pal, Data Mining: Practical Machine Learning Tools and Techniques (Fourth Edition), Elsevier, Singapore, 2017. Google Scholar

[22]

X. ZhangJ. M. Chiou and Y. Ma, Functional prediction through averaging estimated functional linear regression models, Biometrika, 105 (2018), 945-962.  doi: 10.1093/biomet/asy041.  Google Scholar

show all references

References:
[1]

W. BiM. CaiM. Liu and et al., A Big Data Clustering Algorithm for Mitigating the Risk of Customer Churn, IEEE Transactions on Industrial Informatics, 12 (2016), 1270-1281.   Google Scholar

[2]

A Das and A. Desarkar, Decision tree-based analytics for reducing air pollution, Journal of Information & Knowledge Management, 17 (2018), 1850015. Google Scholar

[3]

K. J. Dsouza and Z. Ansari, Experimental exploration of support vector machine for cancer cell classification, IEEE International Conference on Cloud Computing in Emerging Markets, 2017, 29–34. Google Scholar

[4]

D. Dua and E. Karra Taniskidou, UCI Machine Learning Repository, http://archive.ics.uci.edu/ml. Irvine, CA: University of California, School of Information and Computer Science, 2017. Google Scholar

[5]

L. I. JinmengY. Lin and T. Zhu, k-Nearest Neighbor Classification Algorithm Based on Hubness and Class Weighting, Computer Engineering, 44 (2018), 248-252.   Google Scholar

[6]

F. KhozeimehR. AlizadehsaniM. RoshanzamirA. khosravi and et al., An expert system for selecting wart treatment method, Computer in Biology and Medicine, 81 (2017), 167-175.   Google Scholar

[7]

F. KhozeimehF. Jabberi Azad and et al., Intralesional immunotherapy compared to cryotherapy in the treatment of warts, International Journal of Dermatology, 56 (2017), 474-478.   Google Scholar

[8]

Q. LiY. RenL. Li and W. Liu, From low-level geometric features to high-level semantics: An axiomatic fuzzy set clustering approach, Journal of Intelligent & Fuzzy Systems, 31 (2016), 775-786.  doi: 10.3233/JIFS-169009.  Google Scholar

[9]

Q. LiY. RenL. Li and et al., Fuzzy based affinity learning for spectral clustering., Pattern Recognition, 60 (2016), 531-542.   Google Scholar

[10]

Z. Li, Q. Zhang and X. Duan, et al., A novel semantic approach for multi-ethnic face recognition, International Journal of Pattern Recognition & Artificial Intelligence, 32 (2017), 15. Google Scholar

[11]

X. Liu, W. Jia and Y. Wang, et al., Knowledge discovery and semantic learning in the framework of axiomatic fuzzy set theory, Wires Data Mining And Knowledge Discovery, 8 (2018), e1268. doi: 10.1002/widm.1268.  Google Scholar

[12]

X. Liu, K. Zhu and H. Z. Huang, The representations of fuzzy concepts based on the fuzzy matrix theory and the AFS theory, IEEE International Symposium on Intelligent Control, 2003. Google Scholar

[13]

X. Liu, The fuzzy theory based on afs algebras and afs structure, Journal of Mathematical Analysis and Applications, 217 (1998), 459-478.  doi: 10.1006/jmaa.1997.5718.  Google Scholar

[14]

X. LiuW. WangT. Chai and et al., Approaches to the representations and logic operations of fuzzy concepts in the framework of axiomatic fuzzy set theory I, Information Sciences, 177 (2007), 1007-1026.  doi: 10.1016/j.ins.2006.07.011.  Google Scholar

[15]

X. Liu, The fuzzy sets and systems based on AFS structure, EI algebra and EII algebra, Fuzzy Sets & Systems, 95 (1998), 179-188.  doi: 10.1016/S0165-0114(96)00298-9.  Google Scholar

[16]

X. LiuX. Feng and W. Pedrycz, Extraction of fuzzy rules from fuzzy decision trees: An axiomatic fuzzy sets (AFS) approach, Data & Knowledge Engineering, 84 (2013), 1-25.  doi: 10.1016/j.datak.2012.12.001.  Google Scholar

[17]

Y. RenM. Song and X. Liu, New approaches to the fuzzy clustering via AFS theory, International Journal of Information and Systems Sciences, 3 (2007), 307-325.   Google Scholar

[18]

Y. RenQ. LiW. Liu and et al., Semantic facial descriptor extraction via Axiomatic Fuzzy Set, Neurocomputing, 171 (2016), 1462-1474.   Google Scholar

[19]

A. Swami and R. Jain, Scikit-learn: Machine learning in python, Journal of Machine Learning Research, 12 (2012), 2825-2830.   Google Scholar

[20]

W. Wang and X. Liu, Fuzzy forecasting based on automatic clustering and axiomatic fuzzy set classification, Information Sciences, 294 (2015), 78-94.  doi: 10.1016/j.ins.2014.09.027.  Google Scholar

[21]

I. H. Witten, E. Frank, M. A. Hall and C. J. Pal, Data Mining: Practical Machine Learning Tools and Techniques (Fourth Edition), Elsevier, Singapore, 2017. Google Scholar

[22]

X. ZhangJ. M. Chiou and Y. Ma, Functional prediction through averaging estimated functional linear regression models, Biometrika, 105 (2018), 945-962.  doi: 10.1093/biomet/asy041.  Google Scholar

Figure 1.  The proposed classifier flow chart
Figure 2.  Samples in abstract features space
Figure 3.  The membership degree of $ m_1 $ on all data
Figure 4.  The membership degree of description Class 1
Figure 5.  The membership degree of description Class 2
Figure 6.  The membership degree of description Class 3
Figure 7.  The membership degree of three class descriptions on all samples
Table 1.  The maximum, average and minimum of each abstract feature
feature $ f_1 $ $ f_2 $
minimum -1.73 -2.36
average 0.00 0.00
maximum 1.61 2.44
feature $ f_1 $ $ f_2 $
minimum -1.73 -2.36
average 0.00 0.00
maximum 1.61 2.44
Table 2.  The experiments results-accuracy rates(standard deviation)
dataset LogisticR C4.5Tree SVM KNN Our method
wine 0.9556$ \pm $0.0032 0.9364$ \pm $0.0127 0.7900$ \pm $0.0118 0.7074$ \pm $0.0141 0.9666$ \pm $0.0005
iris 0.9593$ \pm $0.0021 0.9520$ \pm $0.0053 0.9826$ \pm $0.0071 0.9647$ \pm $0.0032 0.9867$ \pm $0.0000
heart 0.8399$ \pm $0.0038 0.7544$ \pm $0.0230 0.6918$ \pm $0.0181 0.6604$ \pm $0.0118 0.8407$ \pm $0.0000
breast_C 0.7220$ \pm $0.0209 0.7142$ \pm $0.0209 0.6034$ \pm $0.0241 0.5405$ \pm $0.0118 0.7753$ \pm $0.0038
seeds 0.9228$ \pm $0.0020 0.9286$ \pm $0.0089 0.9271$ \pm $0.0055 0.8828$ \pm $0.0088 0.8590$ \pm $0.0093
USD 0.7309$ \pm $0.0052 0.9321$ \pm $0.0090 0.9510$ \pm $0.0017 0.8247$ \pm $0.0133 0.7912$ \pm $0.0090
column_2c 0.8258$ \pm $0.0030 0.8067$ \pm $0.0205 0.8625$ \pm $0.0031 0.8280$ \pm $0.0064 0.7697$ \pm $0.0040
caesarian 0.6741$ \pm $0.0189 0.5263$ \pm $0.0341 0.6551$ \pm $0.0161 0.5589$ \pm $0.0326 0.7253$ \pm $0.0125
immunotherapy 0.7973$ \pm $0.0139 0.8073$ \pm $0.0262 0.7897$ \pm $0.0000 0.7235$ \pm $0.0208 0.7617$ \pm $0.0091
SHS2015 0.5572$ \pm $0.0133 0.6126$ \pm $0.0133 0.6443$ \pm $0.0213 0.5445$ \pm $0.0561 0.6498$ \pm $0.0028
dataset LogisticR C4.5Tree SVM KNN Our method
wine 0.9556$ \pm $0.0032 0.9364$ \pm $0.0127 0.7900$ \pm $0.0118 0.7074$ \pm $0.0141 0.9666$ \pm $0.0005
iris 0.9593$ \pm $0.0021 0.9520$ \pm $0.0053 0.9826$ \pm $0.0071 0.9647$ \pm $0.0032 0.9867$ \pm $0.0000
heart 0.8399$ \pm $0.0038 0.7544$ \pm $0.0230 0.6918$ \pm $0.0181 0.6604$ \pm $0.0118 0.8407$ \pm $0.0000
breast_C 0.7220$ \pm $0.0209 0.7142$ \pm $0.0209 0.6034$ \pm $0.0241 0.5405$ \pm $0.0118 0.7753$ \pm $0.0038
seeds 0.9228$ \pm $0.0020 0.9286$ \pm $0.0089 0.9271$ \pm $0.0055 0.8828$ \pm $0.0088 0.8590$ \pm $0.0093
USD 0.7309$ \pm $0.0052 0.9321$ \pm $0.0090 0.9510$ \pm $0.0017 0.8247$ \pm $0.0133 0.7912$ \pm $0.0090
column_2c 0.8258$ \pm $0.0030 0.8067$ \pm $0.0205 0.8625$ \pm $0.0031 0.8280$ \pm $0.0064 0.7697$ \pm $0.0040
caesarian 0.6741$ \pm $0.0189 0.5263$ \pm $0.0341 0.6551$ \pm $0.0161 0.5589$ \pm $0.0326 0.7253$ \pm $0.0125
immunotherapy 0.7973$ \pm $0.0139 0.8073$ \pm $0.0262 0.7897$ \pm $0.0000 0.7235$ \pm $0.0208 0.7617$ \pm $0.0091
SHS2015 0.5572$ \pm $0.0133 0.6126$ \pm $0.0133 0.6443$ \pm $0.0213 0.5445$ \pm $0.0561 0.6498$ \pm $0.0028
[1]

Kengo Nakai, Yoshitaka Saiki. Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1079-1092. doi: 10.3934/dcdss.2020352

[2]

Ali Mahmoodirad, Harish Garg, Sadegh Niroomand. Solving fuzzy linear fractional set covering problem by a goal programming based solution approach. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020162

[3]

Min Ji, Xinna Ye, Fangyao Qian, T.C.E. Cheng, Yiwei Jiang. Parallel-machine scheduling in shared manufacturing. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020174

[4]

Reza Chaharpashlou, Abdon Atangana, Reza Saadati. On the fuzzy stability results for fractional stochastic Volterra integral equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020432

[5]

Rajendra K C Khatri, Brendan J Caseria, Yifei Lou, Guanghua Xiao, Yan Cao. Automatic extraction of cell nuclei using dilated convolutional network. Inverse Problems & Imaging, 2021, 15 (1) : 27-40. doi: 10.3934/ipi.2020049

[6]

Ying Lin, Qi Ye. Support vector machine classifiers by non-Euclidean margins. Mathematical Foundations of Computing, 2020, 3 (4) : 279-300. doi: 10.3934/mfc.2020018

[7]

Nicholas Geneva, Nicholas Zabaras. Multi-fidelity generative deep learning turbulent flows. Foundations of Data Science, 2020, 2 (4) : 391-428. doi: 10.3934/fods.2020019

[8]

Mohammed Abdulrazaq Kahya, Suhaib Abduljabbar Altamir, Zakariya Yahya Algamal. Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 87-98. doi: 10.3934/naco.2020017

[9]

Felix Finster, Jürg Fröhlich, Marco Oppio, Claudio F. Paganini. Causal fermion systems and the ETH approach to quantum theory. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020451

[10]

Kung-Ching Chang, Xuefeng Wang, Xie Wu. On the spectral theory of positive operators and PDE applications. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3171-3200. doi: 10.3934/dcds.2020054

[11]

Huu-Quang Nguyen, Ya-Chi Chu, Ruey-Lin Sheu. On the convexity for the range set of two quadratic functions. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020169

[12]

Sumit Kumar Debnath, Pantelimon Stǎnicǎ, Nibedita Kundu, Tanmay Choudhury. Secure and efficient multiparty private set intersection cardinality. Advances in Mathematics of Communications, 2021, 15 (2) : 365-386. doi: 10.3934/amc.2020071

[13]

Editorial Office. Retraction: Xiaohong Zhu, Lihe Zhou, Zili Yang and Joyati Debnath, A new text information extraction algorithm of video image under multimedia environment. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1265-1265. doi: 10.3934/dcdss.2019087

[14]

Pierre-Etienne Druet. A theory of generalised solutions for ideal gas mixtures with Maxwell-Stefan diffusion. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020458

[15]

Sergey Rashkovskiy. Hamilton-Jacobi theory for Hamiltonian and non-Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 563-583. doi: 10.3934/jgm.2020024

[16]

Tuoc Phan, Grozdena Todorova, Borislav Yordanov. Existence uniqueness and regularity theory for elliptic equations with complex-valued potentials. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1071-1099. doi: 10.3934/dcds.2020310

[17]

Juan Pablo Pinasco, Mauro Rodriguez Cartabia, Nicolas Saintier. Evolutionary game theory in mixed strategies: From microscopic interactions to kinetic equations. Kinetic & Related Models, 2021, 14 (1) : 115-148. doi: 10.3934/krm.2020051

[18]

Yasmine Cherfaoui, Mustapha Moulaï. Biobjective optimization over the efficient set of multiobjective integer programming problem. Journal of Industrial & Management Optimization, 2021, 17 (1) : 117-131. doi: 10.3934/jimo.2019102

[19]

Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, 2021, 20 (1) : 449-465. doi: 10.3934/cpaa.2020276

[20]

Jann-Long Chern, Sze-Guang Yang, Zhi-You Chen, Chih-Her Chen. On the family of non-topological solutions for the elliptic system arising from a product Abelian gauge field theory. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3291-3304. doi: 10.3934/dcds.2020127

 Impact Factor: 

Metrics

  • PDF downloads (189)
  • HTML views (1052)
  • Cited by (4)

[Back to Top]