
-
Previous Article
Adaptive discontinuous galerkin finite elements for advective Allen-Cahn equation
- NACO Home
- This Issue
-
Next Article
A robust optimization model for sustainable and resilient closed-loop supply chain network design considering conditional value at risk
Discriminant analysis of regularized multidimensional scaling
1. | Department of Mathematics, University of Dhaka, Bangladesh |
2. | School of Mathematics, University of Southampton, UK |
Regularized Multidimensional Scaling with Radial basis function (RMDS) is a nonlinear variant of classical Multi-Dimensional Scaling (cMDS). A key issue that has been addressed in RMDS is the effective selection of centers of the radial basis functions that plays a very important role in reducing the dimension preserving the structure of the data in higher dimensional space. RMDS uses data in unsupervised settings that means RMDS does not use any prior information of the dataset. This article is concerned on the supervised setting. Here we have incorporated the class information of some members of data to the RMDS model. The class separability term improved the method RMDS significantly and also outperforms other discriminant analysis methods such as Linear discriminant analysis (LDA) which is documented through numerical experiments.
References:
[1] |
A. Argyriou, T. Evgeniou and M. Pontil, Multi-task Feature Learning, in Advances in Neural Information Processing Systems (eds. B. Schoelkopf, J. Platt, and T. Hoffman), MIT Press, 2007. Google Scholar |
[2] |
A. Argyriou, T. Evgeniou and M. Pontil, Convex Multi-task Feature Learning, Machine Learning, Special Issue on Inductive Transfer Learning, 73 (2008), 243-272. Google Scholar |
[3] |
J. Bénasséni, Partial additive constant, J. Statist. Comput. Simul., 49 (1994), 179-193. Google Scholar |
[4] |
I. Borg and P. J. F. Groenen, Modern Multidimensional Scaling. Theory and Applications, 2$^{nd}$ edition, Springer Series in Statistics, Springer, 2005. |
[5] |
F. Cailliez,
The analytical solution of the additive constant problem, Psychometrika, 48 (1983), 305-308.
doi: 10.1007/BF02294026. |
[6] |
H. G. Chew and C. C. Lim,
On regularisation parameter transformation of support vector machines, Journal of Industrial and Management Optimization, 5 (2009), 403-415.
doi: 10.3934/jimo.2009.5.403. |
[7] |
L. G. Cooper, A new solution to the additive constant problem in metric and multidimensional scaling, Psychometrika, 37 (1972), 311-321. Google Scholar |
[8] |
T. F. Cox and M. A. Cox, Multidimensional Scaling, 2$^{nd}$ edition, Chapman and Hall/CRC, 2002. |
[9] |
T. F. Cox and G. Ferry,
Discriminant analysis using nonmetric multidimensional scaling, Pattern Recognition, 26 (1993), 145-153.
|
[10] |
J. de Leeuw, Applications of convex analysis to multidimensional scaling, in Recent Developments in Statistics (eds. J. Barra, F. Brodeau, G. Romier, and B. van Cutsen), North Holland Publishing Company, Amsterdem, The Netherlands, 133–145. Google Scholar |
[11] |
J. de Leeuw, Block relaxation algorithms in statistics, in Information Systems and Data Analysis (eds. Bock, H.H. et al.), Springer, Berlin, (1994), 308–325. Google Scholar |
[12] |
W. Glunt, T. L. Hayden, S. Hong and J. Wells,
An alternating projection algorithm for computing the nearest Euclidean distance matrix, SIAM J. Matrix Anal. Appl., 11 (1990), 589-600.
doi: 10.1137/0611042. |
[13] |
W. Glunt, T. L. Hayden and R. Raydan, Molecular conformations from distance matrices, J. Computational Chemistry, 14 (1993), 114-120. Google Scholar |
[14] |
J. C. Gower,
Some distance properties of latent rootand vector methods in multivariate analysis, Biometrika, 53 (1966), 315-328.
doi: 10.1093/biomet/53.3-4.325. |
[15] |
Y. Hao and F. Meng,
A new method on gene selection for tissue classification, Journal of Industrial and Management Optimization, 3 (2007), 739-748.
doi: 10.3934/jimo.2007.3.739. |
[16] |
W. L. G. Koontz and K. Fukunaga, A nonlinear feature extraction algorithm using distance information, IEEE Transactions on Computers, 21 (1972), 56-63. Google Scholar |
[17] |
J. Kruskal,
Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 29 (1964), 1-27.
doi: 10.1007/BF02289565. |
[18] |
T. Li, S. Zhu and M. Ogihara,
Using discriminant analysis for multi-class classification: an experimental investigation, Knowl Inf Syst., 10 (2006), 453-472.
doi: 10.1007/s10115-006-0013-y. |
[19] |
D. Lowe, Novel topographic nonlinear feature extraction using radial basis functions for concentration coding in the artificial nose, IEEE International Conference on Artificial Neural Networks, (1993), 95–99. Google Scholar |
[20] |
K. V. Mardia, J. T. Kent and J. M. Bibby, Multivariate Analysis, $10{^th}$ printing, Academic Press, 1995.
![]() |
[21] |
A. M. Martinez and A. C. Kak,
PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (2001), 228-233.
doi: 10.1109/34.908974. |
[22] |
S. J. Messick and R. P Abelson, The additive constant problem in multidimensional scaling, Psychometrika, 21 (1956), 1-15. Google Scholar |
[23] |
E. Pȩkalaska and R. P. W. Duin, The Dissimilarity Representation for Pattern Recognition: Foundations and Application, Series in Machine Perception Artificial Intelligence 64, World Scientific, 2005. Google Scholar |
[24] |
H.-D. Qi,
A semismooth Newton method for the nearest Euclidean distance matrix problem, SIAM Journal Matrix Analysis and Applications, 34 (2013), 67-93.
doi: 10.1137/110849523. |
[25] |
H.-D. Qi and N. Xiu,
A convex quadratic semidefinite programming approach to the partial additive constant problem in multidimensional scaling, Journal of Statistical Computation and Simulation, 82 (2012), 1317-1336.
doi: 10.1080/00949655.2011.579970. |
[26] |
H.-D. Qi, N. H. Xiu and X. M. Yuan,
A Lagrangian dual approach to the single source localization problem, IEEE Transactions on Signal Processing, 61 (2013), 3815-3826.
doi: 10.1109/TSP.2013.2264814. |
[27] |
H.-D. Qi and X. M. Yuan,
Computing the nearest Euclidean distance matrix with low embedding dimensions, Mathematical Programming, Ser. A, 147 (2014), 351-389.
doi: 10.1007/s10107-013-0726-0. |
[28] |
K. Schittkowski,
Optimal parameter selection in support vector machines, Journal of Industrial and Management Optimization, 1 (2005), 465-476.
doi: 10.3934/jimo.2005.1.465. |
[29] |
I. J. Schoenberg,
Remarks to Maurice Fréchet's article "Sur la définition axiomatque d'une classe d'espaces vectoriels distanciés applicbles vectoriellement sur l'espace de Hilbet, Ann. Math., 36 (1935), 724-732.
doi: 10.2307/1968654. |
[30] |
S Jahanand and H. D. Qi,
Regularized multidimensional scaling with radial basis functions, Journal of Industrial and Management Optimization, 12 (2016), 543-563.
doi: 10.3934/jimo.2016.12.543. |
[31] |
S. Theodoridis and K. Koutroumbas, Pattern Recognition, Elsevier Inc., 2009. Google Scholar |
[32] |
S. Theodoridis and K. Koutroumbas, An Introduction to Pattern Recognition, A MATLAB Approach, Elsevier Inc., 2010. Google Scholar |
[33] |
W. S. Torgerson, Theory and Methods for Scaling, Wiley, New York, 1958. |
[34] |
A. R. Webb, Multidimensional Scaling by iterative majorization using radial basis functions, Pattern Recognition, 28 (1995), 753-759. Google Scholar |
[35] |
A. R. Webb, Nonlinear feature extraction with radial basis functions using a weighted multidimensional scaling stress measure, Pattern Recognition, IEEE Conference Publications, 4 (1996), 635–639. Google Scholar |
[36] |
A. R. Webb, An approach to nonlinear principal component analysis using radially-symmetric kernel functions, Statistics and Computing, 6 (1996), 159-168. Google Scholar |
[37] |
G. Young and A. S. Householder,
Discussion of a set of points in terms of their mutual distances, Psychometrika, 3 (1938), 19-22.
doi: 10.1007/BF02288560. |
[38] |
Y. Yuan, W. Fan and D. Pu,
Spline function smooth support vector machine for classification, Journal of Industrial and Management Optimization, 3 (2007), 529-542.
doi: 10.3934/jimo.2007.3.529. |
show all references
References:
[1] |
A. Argyriou, T. Evgeniou and M. Pontil, Multi-task Feature Learning, in Advances in Neural Information Processing Systems (eds. B. Schoelkopf, J. Platt, and T. Hoffman), MIT Press, 2007. Google Scholar |
[2] |
A. Argyriou, T. Evgeniou and M. Pontil, Convex Multi-task Feature Learning, Machine Learning, Special Issue on Inductive Transfer Learning, 73 (2008), 243-272. Google Scholar |
[3] |
J. Bénasséni, Partial additive constant, J. Statist. Comput. Simul., 49 (1994), 179-193. Google Scholar |
[4] |
I. Borg and P. J. F. Groenen, Modern Multidimensional Scaling. Theory and Applications, 2$^{nd}$ edition, Springer Series in Statistics, Springer, 2005. |
[5] |
F. Cailliez,
The analytical solution of the additive constant problem, Psychometrika, 48 (1983), 305-308.
doi: 10.1007/BF02294026. |
[6] |
H. G. Chew and C. C. Lim,
On regularisation parameter transformation of support vector machines, Journal of Industrial and Management Optimization, 5 (2009), 403-415.
doi: 10.3934/jimo.2009.5.403. |
[7] |
L. G. Cooper, A new solution to the additive constant problem in metric and multidimensional scaling, Psychometrika, 37 (1972), 311-321. Google Scholar |
[8] |
T. F. Cox and M. A. Cox, Multidimensional Scaling, 2$^{nd}$ edition, Chapman and Hall/CRC, 2002. |
[9] |
T. F. Cox and G. Ferry,
Discriminant analysis using nonmetric multidimensional scaling, Pattern Recognition, 26 (1993), 145-153.
|
[10] |
J. de Leeuw, Applications of convex analysis to multidimensional scaling, in Recent Developments in Statistics (eds. J. Barra, F. Brodeau, G. Romier, and B. van Cutsen), North Holland Publishing Company, Amsterdem, The Netherlands, 133–145. Google Scholar |
[11] |
J. de Leeuw, Block relaxation algorithms in statistics, in Information Systems and Data Analysis (eds. Bock, H.H. et al.), Springer, Berlin, (1994), 308–325. Google Scholar |
[12] |
W. Glunt, T. L. Hayden, S. Hong and J. Wells,
An alternating projection algorithm for computing the nearest Euclidean distance matrix, SIAM J. Matrix Anal. Appl., 11 (1990), 589-600.
doi: 10.1137/0611042. |
[13] |
W. Glunt, T. L. Hayden and R. Raydan, Molecular conformations from distance matrices, J. Computational Chemistry, 14 (1993), 114-120. Google Scholar |
[14] |
J. C. Gower,
Some distance properties of latent rootand vector methods in multivariate analysis, Biometrika, 53 (1966), 315-328.
doi: 10.1093/biomet/53.3-4.325. |
[15] |
Y. Hao and F. Meng,
A new method on gene selection for tissue classification, Journal of Industrial and Management Optimization, 3 (2007), 739-748.
doi: 10.3934/jimo.2007.3.739. |
[16] |
W. L. G. Koontz and K. Fukunaga, A nonlinear feature extraction algorithm using distance information, IEEE Transactions on Computers, 21 (1972), 56-63. Google Scholar |
[17] |
J. Kruskal,
Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 29 (1964), 1-27.
doi: 10.1007/BF02289565. |
[18] |
T. Li, S. Zhu and M. Ogihara,
Using discriminant analysis for multi-class classification: an experimental investigation, Knowl Inf Syst., 10 (2006), 453-472.
doi: 10.1007/s10115-006-0013-y. |
[19] |
D. Lowe, Novel topographic nonlinear feature extraction using radial basis functions for concentration coding in the artificial nose, IEEE International Conference on Artificial Neural Networks, (1993), 95–99. Google Scholar |
[20] |
K. V. Mardia, J. T. Kent and J. M. Bibby, Multivariate Analysis, $10{^th}$ printing, Academic Press, 1995.
![]() |
[21] |
A. M. Martinez and A. C. Kak,
PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (2001), 228-233.
doi: 10.1109/34.908974. |
[22] |
S. J. Messick and R. P Abelson, The additive constant problem in multidimensional scaling, Psychometrika, 21 (1956), 1-15. Google Scholar |
[23] |
E. Pȩkalaska and R. P. W. Duin, The Dissimilarity Representation for Pattern Recognition: Foundations and Application, Series in Machine Perception Artificial Intelligence 64, World Scientific, 2005. Google Scholar |
[24] |
H.-D. Qi,
A semismooth Newton method for the nearest Euclidean distance matrix problem, SIAM Journal Matrix Analysis and Applications, 34 (2013), 67-93.
doi: 10.1137/110849523. |
[25] |
H.-D. Qi and N. Xiu,
A convex quadratic semidefinite programming approach to the partial additive constant problem in multidimensional scaling, Journal of Statistical Computation and Simulation, 82 (2012), 1317-1336.
doi: 10.1080/00949655.2011.579970. |
[26] |
H.-D. Qi, N. H. Xiu and X. M. Yuan,
A Lagrangian dual approach to the single source localization problem, IEEE Transactions on Signal Processing, 61 (2013), 3815-3826.
doi: 10.1109/TSP.2013.2264814. |
[27] |
H.-D. Qi and X. M. Yuan,
Computing the nearest Euclidean distance matrix with low embedding dimensions, Mathematical Programming, Ser. A, 147 (2014), 351-389.
doi: 10.1007/s10107-013-0726-0. |
[28] |
K. Schittkowski,
Optimal parameter selection in support vector machines, Journal of Industrial and Management Optimization, 1 (2005), 465-476.
doi: 10.3934/jimo.2005.1.465. |
[29] |
I. J. Schoenberg,
Remarks to Maurice Fréchet's article "Sur la définition axiomatque d'une classe d'espaces vectoriels distanciés applicbles vectoriellement sur l'espace de Hilbet, Ann. Math., 36 (1935), 724-732.
doi: 10.2307/1968654. |
[30] |
S Jahanand and H. D. Qi,
Regularized multidimensional scaling with radial basis functions, Journal of Industrial and Management Optimization, 12 (2016), 543-563.
doi: 10.3934/jimo.2016.12.543. |
[31] |
S. Theodoridis and K. Koutroumbas, Pattern Recognition, Elsevier Inc., 2009. Google Scholar |
[32] |
S. Theodoridis and K. Koutroumbas, An Introduction to Pattern Recognition, A MATLAB Approach, Elsevier Inc., 2010. Google Scholar |
[33] |
W. S. Torgerson, Theory and Methods for Scaling, Wiley, New York, 1958. |
[34] |
A. R. Webb, Multidimensional Scaling by iterative majorization using radial basis functions, Pattern Recognition, 28 (1995), 753-759. Google Scholar |
[35] |
A. R. Webb, Nonlinear feature extraction with radial basis functions using a weighted multidimensional scaling stress measure, Pattern Recognition, IEEE Conference Publications, 4 (1996), 635–639. Google Scholar |
[36] |
A. R. Webb, An approach to nonlinear principal component analysis using radially-symmetric kernel functions, Statistics and Computing, 6 (1996), 159-168. Google Scholar |
[37] |
G. Young and A. S. Householder,
Discussion of a set of points in terms of their mutual distances, Psychometrika, 3 (1938), 19-22.
doi: 10.1007/BF02288560. |
[38] |
Y. Yuan, W. Fan and D. Pu,
Spline function smooth support vector machine for classification, Journal of Industrial and Management Optimization, 3 (2007), 529-542.
doi: 10.3934/jimo.2007.3.529. |





Dataset | Dim | Class | no. of ins. | Source |
Iris | 4 | 3 | 150 | UCI Repository |
Cancer | 9 | 2 | 683 | UCI Repository |
Seeds | 7 | 3 | 210 | UCI Repository |
Dataset | Dim | Class | no. of ins. | Source |
Iris | 4 | 3 | 150 | UCI Repository |
Cancer | 9 | 2 | 683 | UCI Repository |
Seeds | 7 | 3 | 210 | UCI Repository |
Dataset | MDS | RMDS | SRMDS | Improvment over RMDS | |
Iris | Support vector | 18 | 13 | 5 | |
Missclassified points | 6 | 3 | 0 | 66 % | |
Cancer | Support vector | 64 | 54 | 47 | |
Missclassified points | 9 | 5 | 1 | 80 % | |
Seeds C1 | Support vector | 42 | 35 | 23 | |
Missclassified points | 12 | 10 | 5 | 50 % | |
Seeds C2 | Support vector | 20 | 16 | 10 | |
Missclassified points | 5 | 3 | 2 | 33 % | |
Seeds C3 | Support vector | 24 | 20 | 9 | |
Missclassified points | 8 | 5 | 2 | 60 % |
Dataset | MDS | RMDS | SRMDS | Improvment over RMDS | |
Iris | Support vector | 18 | 13 | 5 | |
Missclassified points | 6 | 3 | 0 | 66 % | |
Cancer | Support vector | 64 | 54 | 47 | |
Missclassified points | 9 | 5 | 1 | 80 % | |
Seeds C1 | Support vector | 42 | 35 | 23 | |
Missclassified points | 12 | 10 | 5 | 50 % | |
Seeds C2 | Support vector | 20 | 16 | 10 | |
Missclassified points | 5 | 3 | 2 | 33 % | |
Seeds C3 | Support vector | 24 | 20 | 9 | |
Missclassified points | 8 | 5 | 2 | 60 % |
Dataset | LDA | SRMDS |
Iris | 0 | 0 |
Cancer | 4 | 2 |
Seeds | 20 | 9 |
Dataset | LDA | SRMDS |
Iris | 0 | 0 |
Cancer | 4 | 2 |
Seeds | 20 | 9 |
[1] |
Ahmad Mousavi, Zheming Gao, Lanshan Han, Alvin Lim. Quadratic surface support vector machine with L1 norm regularization. Journal of Industrial & Management Optimization, 2021 doi: 10.3934/jimo.2021046 |
[2] |
Dmitry Treschev. A locally integrable multi-dimensional billiard system. Discrete & Continuous Dynamical Systems, 2017, 37 (10) : 5271-5284. doi: 10.3934/dcds.2017228 |
[3] |
Kazeem Olalekan Aremu, Chinedu Izuchukwu, Grace Nnenanya Ogwo, Oluwatosin Temitope Mewomo. Multi-step iterative algorithm for minimization and fixed point problems in p-uniformly convex metric spaces. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2161-2180. doi: 10.3934/jimo.2020063 |
[4] |
Dayalal Suthar, Sunil Dutt Purohit, Haile Habenom, Jagdev Singh. Class of integrals and applications of fractional kinetic equation with the generalized multi-index Bessel function. Discrete & Continuous Dynamical Systems - S, 2021 doi: 10.3934/dcdss.2021019 |
[5] |
Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313 |
[6] |
Fernando P. da Costa, João T. Pinto, Rafael Sasportes. On the convergence to critical scaling profiles in submonolayer deposition models. Kinetic & Related Models, 2018, 11 (6) : 1359-1376. doi: 10.3934/krm.2018053 |
[7] |
Haiyan Wang. Existence and nonexistence of positive radial solutions for quasilinear systems. Conference Publications, 2009, 2009 (Special) : 810-817. doi: 10.3934/proc.2009.2009.810 |
[8] |
Ana Rita Nogueira, João Gama, Carlos Abreu Ferreira. Causal discovery in machine learning: Theories and applications. Journal of Dynamics & Games, 2021 doi: 10.3934/jdg.2021008 |
[9] |
Charles Amorim, Miguel Loayza, Marko A. Rojas-Medar. The nonstationary flows of micropolar fluids with thermal convection: An iterative approach. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2509-2535. doi: 10.3934/dcdsb.2020193 |
[10] |
Davi Obata. Symmetries of vector fields: The diffeomorphism centralizer. Discrete & Continuous Dynamical Systems, 2021 doi: 10.3934/dcds.2021063 |
[11] |
Andreas Neubauer. On Tikhonov-type regularization with approximated penalty terms. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021027 |
[12] |
Sara Munday. On the derivative of the $\alpha$-Farey-Minkowski function. Discrete & Continuous Dynamical Systems, 2014, 34 (2) : 709-732. doi: 10.3934/dcds.2014.34.709 |
[13] |
Fatemeh Abtahi, Zeinab Kamali, Maryam Toutounchi. The BSE concepts for vector-valued Lipschitz algebras. Communications on Pure & Applied Analysis, 2021, 20 (3) : 1171-1186. doi: 10.3934/cpaa.2021011 |
[14] |
Yishui Wang, Dongmei Zhang, Peng Zhang, Yong Zhang. Local search algorithm for the squared metric $ k $-facility location problem with linear penalties. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2013-2030. doi: 10.3934/jimo.2020056 |
[15] |
Rafael G. L. D'Oliveira, Marcelo Firer. Minimum dimensional Hamming embeddings. Advances in Mathematics of Communications, 2017, 11 (2) : 359-366. doi: 10.3934/amc.2017029 |
[16] |
Vo Anh Khoa, Thi Kim Thoa Thieu, Ekeoma Rowland Ijioma. On a pore-scale stationary diffusion equation: Scaling effects and correctors for the homogenization limit. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2451-2477. doi: 10.3934/dcdsb.2020190 |
[17] |
Kha Van Huynh, Barbara Kaltenbacher. Some application examples of minimization based formulations of inverse problems and their regularization. Inverse Problems & Imaging, 2021, 15 (3) : 415-443. doi: 10.3934/ipi.2020074 |
[18] |
Ralf Hielscher, Michael Quellmalz. Reconstructing a function on the sphere from its means along vertical slices. Inverse Problems & Imaging, 2016, 10 (3) : 711-739. doi: 10.3934/ipi.2016018 |
[19] |
A. K. Misra, Anupama Sharma, Jia Li. A mathematical model for control of vector borne diseases through media campaigns. Discrete & Continuous Dynamical Systems - B, 2013, 18 (7) : 1909-1927. doi: 10.3934/dcdsb.2013.18.1909 |
[20] |
Wei Xi Li, Chao Jiang Xu. Subellipticity of some complex vector fields related to the Witten Laplacian. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021047 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]