January  2016, 12(1): 251-268. doi: 10.3934/jimo.2016.12.251

A novel discriminant minimum class locality preserving canonical correlation analysis and its applications

1. 

Institute of Metrology and Computational Science, China Jiliang University, Hangzhou, 310018, Zhejiang Province, China, China, China

Received  October 2013 Revised  January 2015 Published  April 2015

Canonical correlation analysis(CCA) is a well-known technique for simultaneously reducing two relevant data sets, and finding maximal correlation between them. However, it fails to preserve the local structure of each data set, as well as the global discriminant ability, which are important in real applications. In this paper, a new CCA model, called discriminant minimum class locality preserving canonical correlation analysis(called as DMPCCA) is proposed. The proposed method introduces locall structure information and global discriminant information into the classical CCA and considers a optimal combination of intra-class locality preserving, global discriminant ability and the maximal correlation between two sets. The experiments on data visualization, web image retrieval and face recognition validate the effectiveness of the proposed method.
Citation: Yubo Yuan, Chenglong Ma, Dongmei Pu. A novel discriminant minimum class locality preserving canonical correlation analysis and its applications. Journal of Industrial & Management Optimization, 2016, 12 (1) : 251-268. doi: 10.3934/jimo.2016.12.251
References:
[1]

B. Abraham, et al., Dimensionality reduction approach to multivariate prediction,, Comput Stat Data Anal, 48 (2005), 5.  doi: 10.1016/j.csda.2003.11.021.  Google Scholar

[2]

N. E. Ayat, et al., Automatic model selection for the optimization of SVM kernels,, Pattern Recognition, 38 (2005), 1733.  doi: 10.1016/j.patcog.2005.03.011.  Google Scholar

[3]

P. N. Belhumeur, et al., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711.  doi: 10.1109/34.598228.  Google Scholar

[4]

Q. N. Chen, et al., Hierarchical multi-view fisher discriminant analysis,, International Conference on Neural Information Processing, 5864 (2009), 289.  doi: 10.1007/978-3-642-10684-2_32.  Google Scholar

[5]

T. Diethe, et al., Constructing nonlinear discriminants from multiple data views,, Machine Learning and Knowledge Discovery in Databases, 6321 (2010), 328.  doi: 10.1007/978-3-642-15880-3_27.  Google Scholar

[6]

J. H. Friedman, et al., Regularized discriminant analysis,, Journal of the American Statistics Association, 84 (1989), 165.  doi: 10.1080/01621459.1989.10478752.  Google Scholar

[7]

D. R. Hardoon, Sparse canonical correlation analysis,, Machine Learning, 83 (2011), 331.  doi: 10.1007/s10994-010-5222-7.  Google Scholar

[8]

D. R. Hardoon, et al., Canonical correlation analysis: An overview with application to learning methods,, Neural Computation, 16 (2004), 2639.  doi: 10.1162/0899766042321814.  Google Scholar

[9]

X. He, et al., Local Preserving Projections,, Advances in Neural Information Processing Systems, (2003).   Google Scholar

[10]

X. He, et al., Face recognition using Laplicianfaces,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), 328.   Google Scholar

[11]

L. Hoegaerts, et al., Subset based least squares subspace regression in RKHS,, Neurocomputing, 63 (2005), 293.  doi: 10.1016/j.neucom.2004.04.013.  Google Scholar

[12]

Y. J. Huang, et al., Protein NMR recall, precision, and F-measure scores (RPF scores): Structure quality assessment measures based on information retrieval statistics,, Journal of the American Chemical Society, 127 (2005), 1665.   Google Scholar

[13]

H. Hotelling, Relations between two sets of variates,, Biometrika, 28 (1936), 312.  doi: 10.2307/2333955.  Google Scholar

[14]

Z. Ji, et al., Rank canonical correlation analysis and its application in visual search reranking,, Signal Processing, 93 (2013), 2352.  doi: 10.1016/j.sigpro.2012.05.006.  Google Scholar

[15]

X. Y. Jing, et al., Color image canonical correlation analysis for face feature extraction and recognition,, Signal Processing, 91 (2011), 2132.  doi: 10.1016/j.sigpro.2011.02.016.  Google Scholar

[16]

E. Kitani, et al., Um Tutorial sobre Analise de Componentes Principais para o Reconhecimento Automatico de Faces [R/OL],, , (2006).   Google Scholar

[17]

P. L. Lai, et al., Kernel and nonlinear canonical correlation analysis,, International Journal of Neural Systems, 10 (2000), 365.  doi: 10.1016/S0129-0657(00)00034-X.  Google Scholar

[18]

Y. Liu, et al., A survey of content-based image retrieval with high-level semantics,, Pattern Recognition, 40 (2007), 262.  doi: 10.1016/j.patcog.2006.04.045.  Google Scholar

[19]

C. D. Manning, et al., Introduction to Information Retrieval,, Cambridge: Cambridge university press, (2008).  doi: 10.1017/CBO9780511809071.  Google Scholar

[20]

T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar

[21]

T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern Recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar

[22]

A. A. Nielsen, et al., Multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data,, IEEE Transactions on Image Processing, 11 (2002), 293.  doi: 10.1109/83.988962.  Google Scholar

[23]

S. Roweis, et al., Nonlinear dimensionality reduction by local linear embedding,, Science, 290 (2000), 2323.  doi: 10.1126/science.290.5500.2323.  Google Scholar

[24]

M. Ortega, et al., Supporting ranked boolean similarity queries in MARS,, IEEE Transaction on Knowledge and Data Engineering, 10 (1998), 905.  doi: 10.1109/69.738357.  Google Scholar

[25]

N. Otopal, et al., Restricted kernel canonical correlation analysis,, Linear Algebra and its Applications, 437 (2012), 1.  doi: 10.1016/j.laa.2012.02.014.  Google Scholar

[26]

O. A. B. Penatti, et al., Comparative study of global color and texture descriptors for web image retrieval,, Journal of Visual Communication and Image Representation, 23 (2012), 359.  doi: 10.1016/j.jvcir.2011.11.002.  Google Scholar

[27]

Y. Peng, et al., Semi-supervised kernel canonical correlation analysis,, Journal of Software, 19 (2008), 2822.   Google Scholar

[28]

Y. Peng, et al., A new canonical correlation analysis algorithm with local discrimination,, Neural Processing Letters, 31 (2010), 1.  doi: 10.1007/s11063-009-9123-3.  Google Scholar

[29]

R. Pless, et al., A Survey of Manifold Learning,, PIPSJ Transactions on Computer Vision and Applications, 1 (2009), 83.   Google Scholar

[30]

F. S. Samaria, et al., Parameterisation of a stochastic model for human face identification,, In Second IEEE Workshop on Applications of Computer Vision, (1994), 138.  doi: 10.1109/ACV.1994.341300.  Google Scholar

[31]

A. Sharma, et al., Generalized Multiview Analysis: A discriminative latent space,, IEEE Conference on Computer Vision and Pattern Recognition, (2012), 2160.  doi: 10.1109/CVPR.2012.6247923.  Google Scholar

[32]

L. Sun, et al., Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 194.   Google Scholar

[33]

Q. Sun, et al., A new method of feature fusion and its application in image recognition,, Pattern Recognition, 38 (2005), 2437.  doi: 10.1016/j.patcog.2004.12.013.  Google Scholar

[34]

T. K. Sun, et al., A novel method of combined feature extraction for recognition,, IEEE Conference on Data Mining, (2008), 1043.  doi: 10.1109/ICDM.2008.28.  Google Scholar

[35]

T. K. Sun, et al., Locality preserving CCA with applications to data visualization and pose estimation,, Image and Vision Computing, 25 (2007), 531.  doi: 10.1016/j.imavis.2006.04.014.  Google Scholar

[36]

M. Turk, et al., Eigenfaces for recognition,, Journal of Cognitive Neuroscience, 3 (1991), 71.  doi: 10.1162/jocn.1991.3.1.71.  Google Scholar

[37]

N. Vlassis, et al., Supervised linear feature extraction for mobile robot localization,, Proceedings of the IEEE international conference on robotics and automation, 3 (2000), 2979.  doi: 10.1109/ROBOT.2000.846480.  Google Scholar

[38]

Y. H. Yan, et al., A novel multiset integrated canonical correlation analysis framework and its application in feature fusion,, Pattern Recognition, 44 (2011), 1031.  doi: 10.1016/j.patcog.2010.11.004.  Google Scholar

[39]

X. Zhu, et al., Dimensionality reduction by mixed kernel canonical correlation analysis,, Pattern Recognition, 45 (2012), 3003.  doi: 10.1016/j.patcog.2012.02.007.  Google Scholar

[40]

J. Yang, et al., Feature fusion: Parallel strategy vs. serial strategy,, Pattern Recognition, 36 (2003), 1369.  doi: 10.1016/S0031-3203(02)00262-5.  Google Scholar

[41]

W. W. Yu, et al., Face recognition using discriminant locality preserving projections,, Image and Vision computing, 24 (2006), 239.  doi: 10.1016/j.imavis.2005.11.006.  Google Scholar

[42]

Y. B. Yuan, Canonical duality solution for alternating support vector machine,, Journal of Industrial and Management Optimization, 8 (2012), 611.  doi: 10.3934/jimo.2012.8.611.  Google Scholar

[43]

X. Zhang, et al., Discriminative locality preserving canonical correlation analysis,, Pattern Recognition, 321 (2012), 341.  doi: 10.1007/978-3-642-33506-8_43.  Google Scholar

[44]

UCI, UCI Repository of machine learning databases,, , ().   Google Scholar

show all references

References:
[1]

B. Abraham, et al., Dimensionality reduction approach to multivariate prediction,, Comput Stat Data Anal, 48 (2005), 5.  doi: 10.1016/j.csda.2003.11.021.  Google Scholar

[2]

N. E. Ayat, et al., Automatic model selection for the optimization of SVM kernels,, Pattern Recognition, 38 (2005), 1733.  doi: 10.1016/j.patcog.2005.03.011.  Google Scholar

[3]

P. N. Belhumeur, et al., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997), 711.  doi: 10.1109/34.598228.  Google Scholar

[4]

Q. N. Chen, et al., Hierarchical multi-view fisher discriminant analysis,, International Conference on Neural Information Processing, 5864 (2009), 289.  doi: 10.1007/978-3-642-10684-2_32.  Google Scholar

[5]

T. Diethe, et al., Constructing nonlinear discriminants from multiple data views,, Machine Learning and Knowledge Discovery in Databases, 6321 (2010), 328.  doi: 10.1007/978-3-642-15880-3_27.  Google Scholar

[6]

J. H. Friedman, et al., Regularized discriminant analysis,, Journal of the American Statistics Association, 84 (1989), 165.  doi: 10.1080/01621459.1989.10478752.  Google Scholar

[7]

D. R. Hardoon, Sparse canonical correlation analysis,, Machine Learning, 83 (2011), 331.  doi: 10.1007/s10994-010-5222-7.  Google Scholar

[8]

D. R. Hardoon, et al., Canonical correlation analysis: An overview with application to learning methods,, Neural Computation, 16 (2004), 2639.  doi: 10.1162/0899766042321814.  Google Scholar

[9]

X. He, et al., Local Preserving Projections,, Advances in Neural Information Processing Systems, (2003).   Google Scholar

[10]

X. He, et al., Face recognition using Laplicianfaces,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), 328.   Google Scholar

[11]

L. Hoegaerts, et al., Subset based least squares subspace regression in RKHS,, Neurocomputing, 63 (2005), 293.  doi: 10.1016/j.neucom.2004.04.013.  Google Scholar

[12]

Y. J. Huang, et al., Protein NMR recall, precision, and F-measure scores (RPF scores): Structure quality assessment measures based on information retrieval statistics,, Journal of the American Chemical Society, 127 (2005), 1665.   Google Scholar

[13]

H. Hotelling, Relations between two sets of variates,, Biometrika, 28 (1936), 312.  doi: 10.2307/2333955.  Google Scholar

[14]

Z. Ji, et al., Rank canonical correlation analysis and its application in visual search reranking,, Signal Processing, 93 (2013), 2352.  doi: 10.1016/j.sigpro.2012.05.006.  Google Scholar

[15]

X. Y. Jing, et al., Color image canonical correlation analysis for face feature extraction and recognition,, Signal Processing, 91 (2011), 2132.  doi: 10.1016/j.sigpro.2011.02.016.  Google Scholar

[16]

E. Kitani, et al., Um Tutorial sobre Analise de Componentes Principais para o Reconhecimento Automatico de Faces [R/OL],, , (2006).   Google Scholar

[17]

P. L. Lai, et al., Kernel and nonlinear canonical correlation analysis,, International Journal of Neural Systems, 10 (2000), 365.  doi: 10.1016/S0129-0657(00)00034-X.  Google Scholar

[18]

Y. Liu, et al., A survey of content-based image retrieval with high-level semantics,, Pattern Recognition, 40 (2007), 262.  doi: 10.1016/j.patcog.2006.04.045.  Google Scholar

[19]

C. D. Manning, et al., Introduction to Information Retrieval,, Cambridge: Cambridge university press, (2008).  doi: 10.1017/CBO9780511809071.  Google Scholar

[20]

T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar

[21]

T. Melzer, et al., Appearance models based on kernel canonical correlation analysis,, Pattern Recognition, 36 (2003), 1961.  doi: 10.1016/S0031-3203(03)00058-X.  Google Scholar

[22]

A. A. Nielsen, et al., Multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data,, IEEE Transactions on Image Processing, 11 (2002), 293.  doi: 10.1109/83.988962.  Google Scholar

[23]

S. Roweis, et al., Nonlinear dimensionality reduction by local linear embedding,, Science, 290 (2000), 2323.  doi: 10.1126/science.290.5500.2323.  Google Scholar

[24]

M. Ortega, et al., Supporting ranked boolean similarity queries in MARS,, IEEE Transaction on Knowledge and Data Engineering, 10 (1998), 905.  doi: 10.1109/69.738357.  Google Scholar

[25]

N. Otopal, et al., Restricted kernel canonical correlation analysis,, Linear Algebra and its Applications, 437 (2012), 1.  doi: 10.1016/j.laa.2012.02.014.  Google Scholar

[26]

O. A. B. Penatti, et al., Comparative study of global color and texture descriptors for web image retrieval,, Journal of Visual Communication and Image Representation, 23 (2012), 359.  doi: 10.1016/j.jvcir.2011.11.002.  Google Scholar

[27]

Y. Peng, et al., Semi-supervised kernel canonical correlation analysis,, Journal of Software, 19 (2008), 2822.   Google Scholar

[28]

Y. Peng, et al., A new canonical correlation analysis algorithm with local discrimination,, Neural Processing Letters, 31 (2010), 1.  doi: 10.1007/s11063-009-9123-3.  Google Scholar

[29]

R. Pless, et al., A Survey of Manifold Learning,, PIPSJ Transactions on Computer Vision and Applications, 1 (2009), 83.   Google Scholar

[30]

F. S. Samaria, et al., Parameterisation of a stochastic model for human face identification,, In Second IEEE Workshop on Applications of Computer Vision, (1994), 138.  doi: 10.1109/ACV.1994.341300.  Google Scholar

[31]

A. Sharma, et al., Generalized Multiview Analysis: A discriminative latent space,, IEEE Conference on Computer Vision and Pattern Recognition, (2012), 2160.  doi: 10.1109/CVPR.2012.6247923.  Google Scholar

[32]

L. Sun, et al., Canonical correlation analysis for multilabel classification: A least-squares formulation, extensions, and analysis,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 194.   Google Scholar

[33]

Q. Sun, et al., A new method of feature fusion and its application in image recognition,, Pattern Recognition, 38 (2005), 2437.  doi: 10.1016/j.patcog.2004.12.013.  Google Scholar

[34]

T. K. Sun, et al., A novel method of combined feature extraction for recognition,, IEEE Conference on Data Mining, (2008), 1043.  doi: 10.1109/ICDM.2008.28.  Google Scholar

[35]

T. K. Sun, et al., Locality preserving CCA with applications to data visualization and pose estimation,, Image and Vision Computing, 25 (2007), 531.  doi: 10.1016/j.imavis.2006.04.014.  Google Scholar

[36]

M. Turk, et al., Eigenfaces for recognition,, Journal of Cognitive Neuroscience, 3 (1991), 71.  doi: 10.1162/jocn.1991.3.1.71.  Google Scholar

[37]

N. Vlassis, et al., Supervised linear feature extraction for mobile robot localization,, Proceedings of the IEEE international conference on robotics and automation, 3 (2000), 2979.  doi: 10.1109/ROBOT.2000.846480.  Google Scholar

[38]

Y. H. Yan, et al., A novel multiset integrated canonical correlation analysis framework and its application in feature fusion,, Pattern Recognition, 44 (2011), 1031.  doi: 10.1016/j.patcog.2010.11.004.  Google Scholar

[39]

X. Zhu, et al., Dimensionality reduction by mixed kernel canonical correlation analysis,, Pattern Recognition, 45 (2012), 3003.  doi: 10.1016/j.patcog.2012.02.007.  Google Scholar

[40]

J. Yang, et al., Feature fusion: Parallel strategy vs. serial strategy,, Pattern Recognition, 36 (2003), 1369.  doi: 10.1016/S0031-3203(02)00262-5.  Google Scholar

[41]

W. W. Yu, et al., Face recognition using discriminant locality preserving projections,, Image and Vision computing, 24 (2006), 239.  doi: 10.1016/j.imavis.2005.11.006.  Google Scholar

[42]

Y. B. Yuan, Canonical duality solution for alternating support vector machine,, Journal of Industrial and Management Optimization, 8 (2012), 611.  doi: 10.3934/jimo.2012.8.611.  Google Scholar

[43]

X. Zhang, et al., Discriminative locality preserving canonical correlation analysis,, Pattern Recognition, 321 (2012), 341.  doi: 10.1007/978-3-642-33506-8_43.  Google Scholar

[44]

UCI, UCI Repository of machine learning databases,, , ().   Google Scholar

[1]

Javier Fernández, Cora Tori, Marcela Zuccalli. Lagrangian reduction of nonholonomic discrete mechanical systems by stages. Journal of Geometric Mechanics, 2020, 12 (4) : 607-639. doi: 10.3934/jgm.2020029

[2]

Shuang Chen, Jinqiao Duan, Ji Li. Effective reduction of a three-dimensional circadian oscillator model. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020349

[3]

George W. Patrick. The geometry of convergence in numerical analysis. Journal of Computational Dynamics, 2021, 8 (1) : 33-58. doi: 10.3934/jcd.2021003

[4]

Min Chen, Olivier Goubet, Shenghao Li. Mathematical analysis of bump to bucket problem. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5567-5580. doi: 10.3934/cpaa.2020251

[5]

Haiyu Liu, Rongmin Zhu, Yuxian Geng. Gorenstein global dimensions relative to balanced pairs. Electronic Research Archive, 2020, 28 (4) : 1563-1571. doi: 10.3934/era.2020082

[6]

Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345

[7]

Bernold Fiedler. Global Hopf bifurcation in networks with fast feedback cycles. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 177-203. doi: 10.3934/dcdss.2020344

[8]

Annegret Glitzky, Matthias Liero, Grigor Nika. Dimension reduction of thermistor models for large-area organic light-emitting diodes. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020460

[9]

Tien-Yu Lin, Bhaba R. Sarker, Chien-Jui Lin. An optimal setup cost reduction and lot size for economic production quantity model with imperfect quality and quantity discounts. Journal of Industrial & Management Optimization, 2021, 17 (1) : 467-484. doi: 10.3934/jimo.2020043

[10]

Qianqian Han, Xiao-Song Yang. Qualitative analysis of a generalized Nosé-Hoover oscillator. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020346

[11]

Laurence Cherfils, Stefania Gatti, Alain Miranville, Rémy Guillevin. Analysis of a model for tumor growth and lactate exchanges in a glioma. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020457

[12]

Vieri Benci, Sunra Mosconi, Marco Squassina. Preface: Applications of mathematical analysis to problems in theoretical physics. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020446

[13]

Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081

[14]

Touria Karite, Ali Boutoulout. Global and regional constrained controllability for distributed parabolic linear systems: RHUM approach. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020055

[15]

Yining Cao, Chuck Jia, Roger Temam, Joseph Tribbia. Mathematical analysis of a cloud resolving model including the ice microphysics. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 131-167. doi: 10.3934/dcds.2020219

[16]

Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026

[17]

Martin Kalousek, Joshua Kortum, Anja Schlömerkemper. Mathematical analysis of weak and strong solutions to an evolutionary model for magnetoviscoelasticity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 17-39. doi: 10.3934/dcdss.2020331

[18]

Feifei Cheng, Ji Li. Geometric singular perturbation analysis of Degasperis-Procesi equation with distributed delay. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 967-985. doi: 10.3934/dcds.2020305

[19]

Illés Horváth, Kristóf Attila Horváth, Péter Kovács, Miklós Telek. Mean-field analysis of a scaling MAC radio protocol. Journal of Industrial & Management Optimization, 2021, 17 (1) : 279-297. doi: 10.3934/jimo.2019111

[20]

Wen Li, Wei-Hui Liu, Seak Weng Vong. Perron vector analysis for irreducible nonnegative tensors and its applications. Journal of Industrial & Management Optimization, 2021, 17 (1) : 29-50. doi: 10.3934/jimo.2019097

2019 Impact Factor: 1.366

Metrics

  • PDF downloads (86)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]