低秩约束的非线性属性选择算法
|
摘要
针对高维的数据中往往存在非线性、低秩形式和属性冗余等问题,提出一种基于核函数的属性自表达无监督属性选择算法——低秩约束的非线性属性选择算法(LRNFS)。首先,将每一维的属性映射到高维的核空间上,通过核空间上的线性属性选择去实现低维空间上的非线性属性选择;然后,对自表达形式引入偏差项并对系数矩阵进行低秩与稀疏处理;最后,引入核矩阵的系数向量的稀疏正则化因子来实现属性选择。所提算法中用核矩阵来体现其非线性关系,低秩考虑数据的全局信息进行子空间学习,自表达形式确定属性的重要程度。实验结果表明,相比于基于重新调整的线性平方回归(RLSR)半监督特征选择算法,所提算法进行属性选择之后作分类的准确率提升了2. 34%。所提算法解决了数据在低维特征空间上线性不可分的问题,提升了属性选择的准确率。
Concerning the problems of high-dimensional data, such as non-linearity, low-rank form, and feature redundancy, an unsupervised feature selection algorithm based on kernel function was proposd, named Low Rank Non-linear Feature Selection algroithm( LRNFS). Firstly, the features of each dimension were mapped to a high-dimensional kernel space, and the non-linear feature selection in the low-dimensional space was achieved through the linear feature selection in the kernel space. Then, the deviation terms were introduced into the self-expression form, and the low rank and sparse processing of coefficient matrix were achieved. Finally, the sparse regularization factor of kernel matrix coefficient vector was introduced to implement the feature selection. In the proposed algorithm, the kernel matrix was used to represent its non-linear relationship, the global information of data was taken into account in low rank to perform subspace learning, and the importance of feature was determined by the self-expression form. The experimental results show that, compared with the semisupervised feature selection algorithm via Rescaled Linear Square Regression( RLSR), the classification accuracy of the proposed algorithm after feature selection is increased by 2. 34 %. The proposed algorithm can solve the problem that the data is linearly inseparable in the low-dimensional feature space, and improve the accuracy of feature selection.
Concerning the problems of high-dimensional data, such as non-linearity, low-rank form, and feature redundancy, an unsupervised feature selection algorithm based on kernel function was proposd, named Low Rank Non-linear Feature Selection algroithm( LRNFS). Firstly, the features of each dimension were mapped to a high-dimensional kernel space, and the non-linear feature selection in the low-dimensional space was achieved through the linear feature selection in the kernel space. Then, the deviation terms were introduced into the self-expression form, and the low rank and sparse processing of coefficient matrix were achieved. Finally, the sparse regularization factor of kernel matrix coefficient vector was introduced to implement the feature selection. In the proposed algorithm, the kernel matrix was used to represent its non-linear relationship, the global information of data was taken into account in low rank to perform subspace learning, and the importance of feature was determined by the self-expression form. The experimental results show that, compared with the semisupervised feature selection algorithm via Rescaled Linear Square Regression( RLSR), the classification accuracy of the proposed algorithm after feature selection is increased by 2. 34 %. The proposed algorithm can solve the problem that the data is linearly inseparable in the low-dimensional feature space, and improve the accuracy of feature selection.
引文
[1]ZHU X F,LI X L,ZHANG S C.Block-row sparse multiview multilabel learning for image classification[J].IEEE Transactions on Cybernetics,2016,46(2):450-461.
[2]YANG Y,ZHA Z J,GAO Y,et al.Exploting web images for semantic video indexing via robust sample-specific loss[J].IEEETransactions on Cybernetics,2014,16(6):1677-1689.
[3]ZHU X F,HUANG Z,SHEN H T,et al.Linear cross-modal hashing for effective multimedia search[C]//Proceedings of the 21st ACM International Conference on Multimedia.New York:ACM,2013:143-152.
[4]ZHU X F,ZHANG S C,JIN Z,et al.Missing value estimation for mixed-attribute data sets[J].IEEE Transactions on Knowledge and Data Engineering,2011,23(1):110-121.
[5]GU Q Q,LI Z H,HAN J W.Joint feature selection and subspace learning[C]//Proceedings of the 2011 22nd International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI,2011:1294-1299.
[6]ZHANG S C,QIN Z,LING C X,et al.Missing is useful:missing values in cost-sensitive decision trees[J].IEEE Transactions on Knowledge and Data Engineering,2005,17(12):1689-1693.
[7]周志华.机器学习[M].北京:清华大学出版社,2016:126-129.(ZHOU Z H.Machine Learning[M].Beijing:Tsinghua University Press,2016:126-129.)
[8]VARMA M,BABU B R.More generally in efficient multiple kernel learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning.New York:ACM,2009:1065-1072.
[9]LI Y D,LEI C,FANG Y,et al.Unsupervised feature selection by combining subspace learning with feature self-representation[J].Pattern Recognition Letters,2017,109:35-43.
[10]GU Q Q,LI Z H,HAN J W.Linear discriminant dimensionality reduction[C]//Proceedings of the 2011 Joint European Conference on Machine Learning and Knowledge Discovery in Databases,LNCS 6911.Berlin:Springer,2011:549-564.
[11]MLLER K R,MIKA S,RTSCH G,et al.An introduction to kernel-based learning algorithm[J].IEEE Transactions on Neural Networks,2001,12(2):181-201.
[12]王华忠,俞金寿.核函数方法及其模型选择[J].江南大学学报(自然科学版),2006,5(4):500-504.(WANG H Z,YU J S.Study on the kernel-based methods and its model selection[J].Journal of Southern Yangtze University(Natural Science Edition),2006,5(4):500-504.)
[13]LU C Y,LIN Z C,YAN S C.Smoothed low rank and sparse matrix recovery by iteratively reweighted least squares minimization[J].IEEE Transactions on Image Processing,2015,24(2):646-654.
[14]DAUBECHIES I,DEVORE R,FORNASIER M,et al.Iteratively reweighted least squares minimization for sparse recovery[J].Communications on Pure and Applied Mathematics,2008,63(1):1-38.
[15]宗鸣,龚永红,文国秋,等.基于稀疏学习的kNN分类[J].广西师范大学学报(自然科学版),2016,34(3):39-45.(ZONG M,GONG Y H,WEN G Q,et al.k NN classification based on sparse learning[J].Journal of Guangxi Normal University(Natural Science Edition),2016,34(3):39-45.)
[16]PARUOLO P.Multivariate reduced-rank regression:theory and applications[J].Journal of the American Statistical Association,1998,95(450):683-685.
[17]UCI.Repository of machine learning data sets[DB/OL].[2018-04-06].http://archive.ics.uci.edu./ml/.
[18]FAN Z Z,XU Y,ZHANG D.Local linear discriminant analysis framework using sample neighbors[J].IEEE Transactions on Neural Networks,2011,22(7):1119-1132.
[19]NIE F P,ZHU W,LI X L.Unsupervised feature selection with structured graph optimization[C]//Proceedings of the 30th AAAIConference on Artificial Intelligence.Menlo Park,CA:AAAI,2016:1302-1308.
[20]CHEN X,YUAN G,NIE F,et al.Semi-supervised feature selection via rescaled linear regression[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI,2017:1525-1531.
[21]ZHU P F,ZUO W M,ZHANG L,et al.Unsupervised feature selection by regularized self-representation[J].Pattern Recognition,2015,48(2):438-446.
[22]YAMADA M,JITKRITTUM W,SIGAL L,et al.High-dimensional feature selection by feature-wise kernelized Lasso[J].Neural Computation,2014,26(1):185-207.
[23]LIBSVM.A library for support vector machines[EB/OL].[2018-04-06].http://www.csie.nu.edu.tw/~cjlin/libsvm.
[2]YANG Y,ZHA Z J,GAO Y,et al.Exploting web images for semantic video indexing via robust sample-specific loss[J].IEEETransactions on Cybernetics,2014,16(6):1677-1689.
[3]ZHU X F,HUANG Z,SHEN H T,et al.Linear cross-modal hashing for effective multimedia search[C]//Proceedings of the 21st ACM International Conference on Multimedia.New York:ACM,2013:143-152.
[4]ZHU X F,ZHANG S C,JIN Z,et al.Missing value estimation for mixed-attribute data sets[J].IEEE Transactions on Knowledge and Data Engineering,2011,23(1):110-121.
[5]GU Q Q,LI Z H,HAN J W.Joint feature selection and subspace learning[C]//Proceedings of the 2011 22nd International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI,2011:1294-1299.
[6]ZHANG S C,QIN Z,LING C X,et al.Missing is useful:missing values in cost-sensitive decision trees[J].IEEE Transactions on Knowledge and Data Engineering,2005,17(12):1689-1693.
[7]周志华.机器学习[M].北京:清华大学出版社,2016:126-129.(ZHOU Z H.Machine Learning[M].Beijing:Tsinghua University Press,2016:126-129.)
[8]VARMA M,BABU B R.More generally in efficient multiple kernel learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning.New York:ACM,2009:1065-1072.
[9]LI Y D,LEI C,FANG Y,et al.Unsupervised feature selection by combining subspace learning with feature self-representation[J].Pattern Recognition Letters,2017,109:35-43.
[10]GU Q Q,LI Z H,HAN J W.Linear discriminant dimensionality reduction[C]//Proceedings of the 2011 Joint European Conference on Machine Learning and Knowledge Discovery in Databases,LNCS 6911.Berlin:Springer,2011:549-564.
[11]MLLER K R,MIKA S,RTSCH G,et al.An introduction to kernel-based learning algorithm[J].IEEE Transactions on Neural Networks,2001,12(2):181-201.
[12]王华忠,俞金寿.核函数方法及其模型选择[J].江南大学学报(自然科学版),2006,5(4):500-504.(WANG H Z,YU J S.Study on the kernel-based methods and its model selection[J].Journal of Southern Yangtze University(Natural Science Edition),2006,5(4):500-504.)
[13]LU C Y,LIN Z C,YAN S C.Smoothed low rank and sparse matrix recovery by iteratively reweighted least squares minimization[J].IEEE Transactions on Image Processing,2015,24(2):646-654.
[14]DAUBECHIES I,DEVORE R,FORNASIER M,et al.Iteratively reweighted least squares minimization for sparse recovery[J].Communications on Pure and Applied Mathematics,2008,63(1):1-38.
[15]宗鸣,龚永红,文国秋,等.基于稀疏学习的kNN分类[J].广西师范大学学报(自然科学版),2016,34(3):39-45.(ZONG M,GONG Y H,WEN G Q,et al.k NN classification based on sparse learning[J].Journal of Guangxi Normal University(Natural Science Edition),2016,34(3):39-45.)
[16]PARUOLO P.Multivariate reduced-rank regression:theory and applications[J].Journal of the American Statistical Association,1998,95(450):683-685.
[17]UCI.Repository of machine learning data sets[DB/OL].[2018-04-06].http://archive.ics.uci.edu./ml/.
[18]FAN Z Z,XU Y,ZHANG D.Local linear discriminant analysis framework using sample neighbors[J].IEEE Transactions on Neural Networks,2011,22(7):1119-1132.
[19]NIE F P,ZHU W,LI X L.Unsupervised feature selection with structured graph optimization[C]//Proceedings of the 30th AAAIConference on Artificial Intelligence.Menlo Park,CA:AAAI,2016:1302-1308.
[20]CHEN X,YUAN G,NIE F,et al.Semi-supervised feature selection via rescaled linear regression[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI,2017:1525-1531.
[21]ZHU P F,ZUO W M,ZHANG L,et al.Unsupervised feature selection by regularized self-representation[J].Pattern Recognition,2015,48(2):438-446.
[22]YAMADA M,JITKRITTUM W,SIGAL L,et al.High-dimensional feature selection by feature-wise kernelized Lasso[J].Neural Computation,2014,26(1):185-207.
[23]LIBSVM.A library for support vector machines[EB/OL].[2018-04-06].http://www.csie.nu.edu.tw/~cjlin/libsvm.