Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins

Keiji Tatsumi; Masashi Akao; Ryo  Kawachi; Tetsuzo Tanino

doi:10.3934/naco.2011.1.151

Article Contents

2011, Volume 1, Issue 1: 151-169. Doi: 10.3934/naco.2011.1.151

This issue Previous Article A nonconvergent example for the iterative water-filling algorithm Next Article Parametric excitation based bipedal walking: Control method and optimization

Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins

1.
Graduate School of Engineering, Osaka University, Yamada-Oka 1-2, Suita, Osaka 565-0871

Received: September 2010

Revised: December 2010

Published: February 2011

Abstract / Introduction Related Papers Cited by

Abstract

The all-together method is one of the support vector machine (SVM) for multiclass classification by using a piece-wise linear function. Recently, we proposed a new hard-margin all-together model maximizing geometric margins in the sense of multiobjective optimization for the high generalization ability, which is called the multiobjective multiclass SVM (MMSVM). Moreover, we derived its solving techniques which can find a Pareto optimal solution for the MMSVM, and verified that classifiers with larger geometric margins were obtained by the proposed techniques through numerical experiments. However, the experiments are not enough to evaluate the classification performance of the proposed model, and the MMSVM is a hard-margin model which can be applied to only piecewise linearly separable data. Therefore, in this paper, we extend the hard-margin model into soft-margin one by introducing penalty functions for the slack margin variables, and derive a single-objective second-order cone programming (SOCP) problem to solve it. Furthermore, through numerical experiments we verify the classification performance of the hard and soft-margin MMSVMs for benchmark problems.

Keywords:

Mathematics Subject Classification: Primary: 62H30, 68T10; Secondary: 90C29.

Citation:

Related Papers

Cited by

References

[1]	S. Abe, "Support Vector Machines for Pattern Classification," Springer-Verlag, New York, 2005.
[2]	F. Alizadeh and D. Goldfarb, Second-order cone programming, Mathematical Programming, Ser. B, 95 (2003), 3-51.
[3]	L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition, in "Proc. Int. Conf. Pattern Recognition,'' (1994), 77-87.
[4]	E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines, Computational Optimization and Applications, 12 (1999), 53-79.doi: 10.1023/A:1008663629662.
[5]	M. Ehrgott, "Multicriteria Optimization,'' 2^nd edition, Springer-Verlag, Berlin, 2005.
[6]	Y. Guermeur, Combining discriminant models with new multiclass SVMs, Neuro COLT2 Technical Report Series, 2000.
[7]	U. Kressel, Pairwise classification and support vector machines, in "Advances in kernel methods - Support vector learning'' (eds. B. Schölkopf, C. Burges, and A. J. Smola), MIT Press, Cambridge, (1999), 255-268.
[8]	C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Networks, 13 (2) (2002), 181-201.
[9]	H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers, Mathematical Programming, Ser. B, 95 (2003), 407-430.
[10]	K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Networks, 12 (2), (2001), 181-201.doi: 10.1109/72.914517.
[11]	A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines, IEEE Trans. Neural Networks, 14 (1), (2004), 45-54.doi: 10.1109/TNN.2003.820841.
[12]	J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification, in "Advances in Neural Information Processing Systems,'' Cambridge, MA: MIT Press, 12 (2000), 547-553.
[13]	K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition, in "Proc. SICE Annual Conference 2007,'' 1095-1098.doi: 10.1109/SICE.2007.4421147.
[14]	K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins, Pacific Journal of Optimization, 6 (1), (2000), 115-140.
[15]	J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'' Cambridge University Press, 2004.
[16]	UCI benchmark repository of artificial and real data sets, University of California Irvine, Available from: http://www.ics.uci.edu/~mlearn/databases/
[17]	J. Weston and C. Watkins, Multi-class support vector machines, Technical report CSD-TR-98-04, Univ. London, Royal Holloway, (1998).
[18]	V. N. Vapnik, "Statistical Learning Theory,'' A Wiley-Interscience Publication, 1998.