2011, 1(1): 151-169. doi: 10.3934/naco.2011.1.151

Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins

1. 

Graduate School of Engineering, Osaka University, Yamada-Oka 1-2, Suita, Osaka 565-0871, Japan, Japan, Japan, Japan

Received  September 2010 Revised  December 2010 Published  February 2011

The all-together method is one of the support vector machine (SVM) for multiclass classification by using a piece-wise linear function. Recently, we proposed a new hard-margin all-together model maximizing geometric margins in the sense of multiobjective optimization for the high generalization ability, which is called the multiobjective multiclass SVM (MMSVM). Moreover, we derived its solving techniques which can find a Pareto optimal solution for the MMSVM, and verified that classifiers with larger geometric margins were obtained by the proposed techniques through numerical experiments. However, the experiments are not enough to evaluate the classification performance of the proposed model, and the MMSVM is a hard-margin model which can be applied to only piecewise linearly separable data. Therefore, in this paper, we extend the hard-margin model into soft-margin one by introducing penalty functions for the slack margin variables, and derive a single-objective second-order cone programming (SOCP) problem to solve it. Furthermore, through numerical experiments we verify the classification performance of the hard and soft-margin MMSVMs for benchmark problems.
Citation: Keiji Tatsumi, Masashi Akao, Ryo Kawachi, Tetsuzo Tanino. Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 151-169. doi: 10.3934/naco.2011.1.151
References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification,", Springer-Verlag, (2005).   Google Scholar

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming,, Mathematical Programming, 95 (2003), 3.   Google Scholar

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition,, in, (1994), 77.   Google Scholar

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines,, Computational Optimization and Applications, 12 (1999), 53.  doi: 10.1023/A:1008663629662.  Google Scholar

[5]

M. Ehrgott, "Multicriteria Optimization,'', 2nd edition, (2005).   Google Scholar

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs,, Neuro COLT2 Technical Report Series, (2000).   Google Scholar

[7]

U. Kressel, Pairwise classification and support vector machines,, in, (1999), 255.   Google Scholar

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines,, IEEE Trans. Neural Networks, 13 (2002), 181.   Google Scholar

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers,, Mathematical Programming, 95 (2003), 407.   Google Scholar

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms,, IEEE Trans. Neural Networks, 12 (2001), 181.  doi: 10.1109/72.914517.  Google Scholar

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines,, IEEE Trans. Neural Networks, 14 (2004), 45.  doi: 10.1109/TNN.2003.820841.  Google Scholar

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification,, in, 12 (2000), 547.   Google Scholar

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition,, in, (2007), 1095.  doi: 10.1109/SICE.2007.4421147.  Google Scholar

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins,, Pacific Journal of Optimization, 6 (2000), 115.   Google Scholar

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'', Cambridge University Press, (2004).   Google Scholar

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine,, Available from: , ().   Google Scholar

[17]

J. Weston and C. Watkins, Multi-class support vector machines,, Technical report CSD-TR-98-04, (1998), 98.   Google Scholar

[18]

V. N. Vapnik, "Statistical Learning Theory,'', A Wiley-Interscience Publication, (1998).   Google Scholar

show all references

References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification,", Springer-Verlag, (2005).   Google Scholar

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming,, Mathematical Programming, 95 (2003), 3.   Google Scholar

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition,, in, (1994), 77.   Google Scholar

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines,, Computational Optimization and Applications, 12 (1999), 53.  doi: 10.1023/A:1008663629662.  Google Scholar

[5]

M. Ehrgott, "Multicriteria Optimization,'', 2nd edition, (2005).   Google Scholar

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs,, Neuro COLT2 Technical Report Series, (2000).   Google Scholar

[7]

U. Kressel, Pairwise classification and support vector machines,, in, (1999), 255.   Google Scholar

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines,, IEEE Trans. Neural Networks, 13 (2002), 181.   Google Scholar

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers,, Mathematical Programming, 95 (2003), 407.   Google Scholar

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms,, IEEE Trans. Neural Networks, 12 (2001), 181.  doi: 10.1109/72.914517.  Google Scholar

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines,, IEEE Trans. Neural Networks, 14 (2004), 45.  doi: 10.1109/TNN.2003.820841.  Google Scholar

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification,, in, 12 (2000), 547.   Google Scholar

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition,, in, (2007), 1095.  doi: 10.1109/SICE.2007.4421147.  Google Scholar

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins,, Pacific Journal of Optimization, 6 (2000), 115.   Google Scholar

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'', Cambridge University Press, (2004).   Google Scholar

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine,, Available from: , ().   Google Scholar

[17]

J. Weston and C. Watkins, Multi-class support vector machines,, Technical report CSD-TR-98-04, (1998), 98.   Google Scholar

[18]

V. N. Vapnik, "Statistical Learning Theory,'', A Wiley-Interscience Publication, (1998).   Google Scholar

[1]

Ying Lin, Qi Ye. Support vector machine classifiers by non-Euclidean margins. Mathematical Foundations of Computing, 2020, 3 (4) : 279-300. doi: 10.3934/mfc.2020018

[2]

Shasha Hu, Yihong Xu, Yuhan Zhang. Second-Order characterizations for set-valued equilibrium problems with variable ordering structures. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020164

[3]

Weisong Dong, Chang Li. Second order estimates for complex Hessian equations on Hermitian manifolds. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020377

[4]

Shenglan Xie, Maoan Han, Peng Zhu. A posteriori error estimate of weak Galerkin fem for second order elliptic problem with mixed boundary condition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020340

[5]

Soniya Singh, Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of second order impulsive systems with state-dependent delay in Banach spaces. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020103

[6]

Mathew Gluck. Classification of solutions to a system of $ n^{\rm th} $ order equations on $ \mathbb R^n $. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5413-5436. doi: 10.3934/cpaa.2020246

[7]

Hua Chen, Yawei Wei. Multiple solutions for nonlinear cone degenerate elliptic equations. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020272

[8]

Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345

[9]

Yahia Zare Mehrjerdi. A new methodology for solving bi-criterion fractional stochastic programming. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020054

[10]

Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115

[11]

Alberto Bressan, Sondre Tesdal Galtung. A 2-dimensional shape optimization problem for tree branches. Networks & Heterogeneous Media, 2020  doi: 10.3934/nhm.2020031

[12]

Wenjun Liu, Yukun Xiao, Xiaoqing Yue. Classification of finite irreducible conformal modules over Lie conformal algebra $ \mathcal{W}(a, b, r) $. Electronic Research Archive, , () : -. doi: 10.3934/era.2020123

[13]

M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014

[14]

Mohammed Abdulrazaq Kahya, Suhaib Abduljabbar Altamir, Zakariya Yahya Algamal. Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 87-98. doi: 10.3934/naco.2020017

[15]

Manil T. Mohan. First order necessary conditions of optimality for the two dimensional tidal dynamics system. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020045

[16]

Jiahao Qiu, Jianjie Zhao. Maximal factors of order $ d $ of dynamical cubespaces. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 601-620. doi: 10.3934/dcds.2020278

[17]

Kihoon Seong. Low regularity a priori estimates for the fourth order cubic nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5437-5473. doi: 10.3934/cpaa.2020247

[18]

Youshan Tao, Michael Winkler. Critical mass for infinite-time blow-up in a haptotaxis system with nonlinear zero-order interaction. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 439-454. doi: 10.3934/dcds.2020216

[19]

Jun Zhou. Lifespan of solutions to a fourth order parabolic PDE involving the Hessian modeling epitaxial growth. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5581-5590. doi: 10.3934/cpaa.2020252

[20]

Xuefeng Zhang, Yingbo Zhang. Fault-tolerant control against actuator failures for uncertain singular fractional order systems. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 1-12. doi: 10.3934/naco.2020011

 Impact Factor: 

Metrics

  • PDF downloads (39)
  • HTML views (0)
  • Cited by (2)

[Back to Top]