American Institute of Mathematical Sciences

Advanced
January  2016, 12(1): 285-301. doi: 10.3934/jimo.2016.12.285

Subgradient-based neural network for nonconvex optimization problems in support vector machines with indefinite kernels

Fengqiu Liu 1, and  Xiaoping Xue 2,

1. 

Department of Mathematics, Harbin University of Science and Technology, Harbin, 150080, China
2. 

Department of Mathematics, Harbin Institute of Technology, Harbin 150001

Received  June 2012 Revised  January 2015 Published  April 2015

Support vector machines (SVMs) with positive semidefinite kernels yield convex quadratic programming problems. SVMs with indefinite kernels yield nonconvex quadratic programming problems. Most optimization methods for SVMs rely on the convexity of objective functions and are not efficient for solving such nonconvex problems. In this paper, we propose a subgradient-based neural network (SGNN) for the problems cast by SVMs with indefinite kernels. It is shown that the state of the proposed neural network has finite length, and as a consequence it converges toward a singleton. The coincidence between the solution and the slow solution of SGNN is also proved starting from the initial value of SGNN. Moreover, we employ the Łojasiewicz inequality to exploit the convergence rate of trajectory of SGNN. The obtained results show that each trajectory is either exponentially convergent, or convergent in finite time, toward a singleton belonging to the set of constrained critical points through a quantitative evaluation of the Łojasiewicz exponent at the equilibrium points. This method is easy to implement without adding any new parameters. Three benchmark data sets from the University of California, Irvine machine learning repository are used in the numerical tests. Experimental results show the efficiency of the proposed neural network.
Keywords: support vector machine., indefinite kernels, neural networks, Sub-gradient, Łojasiewicz inequality.
Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C3.
Citation: Fengqiu Liu, Xiaoping Xue. Subgradient-based neural network for nonconvex optimization problems in support vector machines with indefinite kernels. Journal of Industrial & Management Optimization, 2016, 12 (1) : 285-301. doi: 10.3934/jimo.2016.12.285
References:
[1]

Springer-Verlag, Berlin, 1984. doi: 10.1007/978-3-642-69512-4.  Google Scholar

[2]

MA: Birkauser, Boston, 1990.  Google Scholar

[3]

Automatic Control, IEEE Transactions on, 44 (1999), 1995-2006. doi: 10.1109/9.802909.  Google Scholar

[4]

In Proceedings of the 25th International Conference on Machine Learning, (2008), 136-143. doi: 10.1145/1390156.1390174.  Google Scholar

[5]

Wiley, New York, 1969. Google Scholar

[6]

Fuzzy Systems, IEEE Transactions on, 20 (2012), 135-152. Google Scholar

[7]

Boston, MA: Kluwer, 1988. Google Scholar

[8]

Neural Networks, IEEE Transactions on, 17 (2006), 1471-1486. Google Scholar

[9]

Circuits System I, IEEE Transation on, 51 (2004), 2460-2469. doi: 10.1109/TCSI.2004.838143.  Google Scholar

[10]

Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27 (2005), 482-492. doi: 10.1109/TPAMI.2005.78.  Google Scholar

[11]

In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, (eds. Z. Ghahramani and R. Cowell), Society for Artificial Intelligence and Statistics, United States, (2005), 136-143. Google Scholar

[12]

Biological cybernetics, 52 (1985), 141-152.  Google Scholar

[13]

Mathematical Programming Computation, 1 (2009), 97-118. doi: 10.1007/s12532-009-0005-5.  Google Scholar

[14]

Technical report, Artificial Intelligence Unit Department of Computer Science University of Dortmund, 2006. Google Scholar

[15]

In Advances in Kernel Methods, (eds. Schölkopf, C. Burges and A. Smola), MIT, (1999), 185-208. Google Scholar

[16]

Springer-Verlag, Germany, 1998. doi: 10.1007/978-3-642-02431-3.  Google Scholar

[17]

Math. Program., 127 (2011), 3-30. doi: 10.1007/s10107-010-0420-4.  Google Scholar

[18]

Bioinformatics, 20 (2004), 1682-1689. doi: 10.1093/bioinformatics/bth141.  Google Scholar

[19]

MIT, Cambridge, MA, 2002. Google Scholar

[20]

MIT, Cambridge, MA, 1998. Google Scholar

[21]

Wiley, New York, 1998.  Google Scholar

[22]

Circuits and Systems I: Regular Papers, IEEE Transactions on, 55 (2008), 2378-2391. doi: 10.1109/TCSI.2008.920131.  Google Scholar

[23]

Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 34 (2004), 1621-1629. doi: 10.1109/TSMCB.2003.822955.  Google Scholar

[24]

Journal of Industrial and Management Optimization, 10 (2014), 871-882. doi: 10.3934/jimo.2014.10.871.  Google Scholar

show all references

References:
[1]

Springer-Verlag, Berlin, 1984. doi: 10.1007/978-3-642-69512-4.  Google Scholar

[2]

MA: Birkauser, Boston, 1990.  Google Scholar

[3]

Automatic Control, IEEE Transactions on, 44 (1999), 1995-2006. doi: 10.1109/9.802909.  Google Scholar

[4]

In Proceedings of the 25th International Conference on Machine Learning, (2008), 136-143. doi: 10.1145/1390156.1390174.  Google Scholar

[5]

Wiley, New York, 1969. Google Scholar

[6]

Fuzzy Systems, IEEE Transactions on, 20 (2012), 135-152. Google Scholar

[7]

Boston, MA: Kluwer, 1988. Google Scholar

[8]

Neural Networks, IEEE Transactions on, 17 (2006), 1471-1486. Google Scholar

[9]

Circuits System I, IEEE Transation on, 51 (2004), 2460-2469. doi: 10.1109/TCSI.2004.838143.  Google Scholar

[10]

Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27 (2005), 482-492. doi: 10.1109/TPAMI.2005.78.  Google Scholar

[11]

In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, (eds. Z. Ghahramani and R. Cowell), Society for Artificial Intelligence and Statistics, United States, (2005), 136-143. Google Scholar

[12]

Biological cybernetics, 52 (1985), 141-152.  Google Scholar

[13]

Mathematical Programming Computation, 1 (2009), 97-118. doi: 10.1007/s12532-009-0005-5.  Google Scholar

[14]

Technical report, Artificial Intelligence Unit Department of Computer Science University of Dortmund, 2006. Google Scholar

[15]

In Advances in Kernel Methods, (eds. Schölkopf, C. Burges and A. Smola), MIT, (1999), 185-208. Google Scholar

[16]

Springer-Verlag, Germany, 1998. doi: 10.1007/978-3-642-02431-3.  Google Scholar

[17]

Math. Program., 127 (2011), 3-30. doi: 10.1007/s10107-010-0420-4.  Google Scholar

[18]

Bioinformatics, 20 (2004), 1682-1689. doi: 10.1093/bioinformatics/bth141.  Google Scholar

[19]

MIT, Cambridge, MA, 2002. Google Scholar

[20]

MIT, Cambridge, MA, 1998. Google Scholar

[21]

Wiley, New York, 1998.  Google Scholar

[22]

Circuits and Systems I: Regular Papers, IEEE Transactions on, 55 (2008), 2378-2391. doi: 10.1109/TCSI.2008.920131.  Google Scholar

[23]

Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 34 (2004), 1621-1629. doi: 10.1109/TSMCB.2003.822955.  Google Scholar

[24]

Journal of Industrial and Management Optimization, 10 (2014), 871-882. doi: 10.3934/jimo.2014.10.871.  Google Scholar

[1]

Ahmad Mousavi, Zheming Gao, Lanshan Han, Alvin Lim. Quadratic surface support vector machine with L1 norm regularization. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021046
[2]

Yuta Tanoue. Improved Hoeffding inequality for dependent bounded or sub-Gaussian random variables. Probability, Uncertainty and Quantitative Risk, 2021, 6 (1) : 53-60. doi: 10.3934/puqr.2021003
[3]

Xiaochen Mao, Weijie Ding, Xiangyu Zhou, Song Wang, Xingyong Li. Complexity in time-delay networks of multiple interacting neural groups. Electronic Research Archive, , () : -. doi: 10.3934/era.2021022
[4]

Zengyun Wang, Jinde Cao, Zuowei Cai, Lihong Huang. Finite-time stability of impulsive differential inclusion: Applications to discontinuous impulsive neural networks. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2677-2692. doi: 10.3934/dcdsb.2020200
[5]

Xiaochun Gu, Fang Han, Zhijie Wang, Kaleem Kashif, Wenlian Lu. Enhancement of gamma oscillations in E/I neural networks by increase of difference between external inputs. Electronic Research Archive, , () : -. doi: 10.3934/era.2021035
[6]

Quan Hai, Shutang Liu. Mean-square delay-distribution-dependent exponential synchronization of chaotic neural networks with mixed random time-varying delays and restricted disturbances. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 3097-3118. doi: 10.3934/dcdsb.2020221
[7]

Monia Capanna, Jean C. Nakasato, Marcone C. Pereira, Julio D. Rossi. Homogenization for nonlocal problems with smooth kernels. Discrete & Continuous Dynamical Systems, 2021, 41 (6) : 2777-2808. doi: 10.3934/dcds.2020385
[8]

Valeria Chiado Piat, Sergey S. Nazarov, Andrey Piatnitski. Steklov problems in perforated domains with a coefficient of indefinite sign. Networks & Heterogeneous Media, 2012, 7 (1) : 151-178. doi: 10.3934/nhm.2012.7.151
[9]

Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313
[10]

Zhenbing Gong, Yanping Chen, Wenyu Tao. Jump and variational inequalities for averaging operators with variable kernels. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021045
[11]

Juan Manuel Pastor, Javier García-Algarra, José M. Iriondo, José J. Ramasco, Javier Galeano. Dragging in mutualistic networks. Networks & Heterogeneous Media, 2015, 10 (1) : 37-52. doi: 10.3934/nhm.2015.10.37
[12]

Gheorghe Craciun, Jiaxin Jin, Polly Y. Yu. Single-target networks. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021065
[13]

Ana Rita Nogueira, João Gama, Carlos Abreu Ferreira. Causal discovery in machine learning: Theories and applications. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021008
[14]

Davi Obata. Symmetries of vector fields: The diffeomorphism centralizer. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021063
[15]

Horst R. Thieme. Discrete-time dynamics of structured populations via Feller kernels. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021082
[16]

Fatemeh Abtahi, Zeinab Kamali, Maryam Toutounchi. The BSE concepts for vector-valued Lipschitz algebras. Communications on Pure & Applied Analysis, 2021, 20 (3) : 1171-1186. doi: 10.3934/cpaa.2021011
[17]

Alessandro Gondolo, Fernando Guevara Vasquez. Characterization and synthesis of Rayleigh damped elastodynamic networks. Networks & Heterogeneous Media, 2014, 9 (2) : 299-314. doi: 10.3934/nhm.2014.9.299
[18]

Juan Manuel Pastor, Javier García-Algarra, Javier Galeano, José María Iriondo, José J. Ramasco. A simple and bounded model of population dynamics for mutualistic networks. Networks & Heterogeneous Media, 2015, 10 (1) : 53-70. doi: 10.3934/nhm.2015.10.53
[19]

Rui Hu, Yuan Yuan. Stability, bifurcation analysis in a neural network model with delay and diffusion. Conference Publications, 2009, 2009 (Special) : 367-376. doi: 10.3934/proc.2009.2009.367
[20]

Alfonso Castro, Jorge Cossio, Sigifredo Herrón, Carlos Vélez. Infinitely many radial solutions for a $ p $-Laplacian problem with indefinite weight. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021058

2019 Impact Factor: 1.366

Tools

Metrics

  • PDF downloads (39)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences