An overview of kernel alignment and its applications

详细信息    查看全文

• 作者：Tinghua Wang (1) (2)
  Dongyan Zhao (1)
  Shengfeng Tian (3)
  
  1. Institute of Computer Science and Technology ; Peking University ; Beijing ; 100871 ; China
  2. School of Mathematics and Computer Science ; Gannan Normal University ; Ganzhou ; 341000 ; China
  3. School of Computer and Information Technology ; Beijing Jiaotong University ; Beijing ; 100044 ; China
  
• 关键词：Kernel alignment ; Kernel evaluation measure ; Learning kernels ; Kernel method ; Model selection
• 刊名：Artificial Intelligence Review
• 出版年：2015
• 出版时间：February 2015
• 年：2015
• 卷：43
• 期：2
• 页码：179-192
• 全文大小：283 KB
• 参考文献：1. Amayri O, Bouguila N (2010) A study of spam filtering using support vector machines. Artif Intell Revi 34(1): 73鈥?08 CrossRef
  2. Baram Y (2005) Learning by kernel polarization. Neural Comput 17(6): 1264鈥?275 CrossRef
  3. Camargo JE, Gonz谩lez FA (2009) A multi-class kernel alignment method for image collection summarization. In: Proceedings of the 14th Iberoamerican conference on pattern recognition: progress in pattern recognition, image analysis, computer vision, and applications, Guadalajara, Mexico, pp 545鈥?52
  4. Chapelle O, Vapnik V, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1): 131鈥?59 CrossRef
  5. Chapelle O, Zien A, Sch枚lkopf B (2006) Semi-supervised learning. MIT Press, Cambridge, MA CrossRef
  6. Chen B, Liu H, Bao Z (2008) A kernel optimization method based on the localized kernel Fisher criterion. Pattern Recognit 41(3): 1098鈥?109 CrossRef
  7. Chudzian P (2012) Evaluation measures for kernel optimization. Pattern Recognit Lett 33(9): 1108鈥?116 CrossRef
  8. Chung KM, Kao WC, Sun T, Wang LL, Lin CJ (2003) Radius margin bounds for support vector machines with the RBF kernel. Neural Comput 15(11): 2463鈥?681 CrossRef
  9. Cortes C, Mohri M, Rostamizadeh A (2012) Algorithms for learning kernels based on centered alignment. J Mach Learn Res 13: 795鈥?28
  10. Cristianini N, Shawe-Taylor J, Elisseeff A, Kandola J (2001) On kernel-target聽alignment. In: Dietterich TG, Becker S, Ghahraman Z (eds) Advances in Neural Information Processing Systems 14, MIT Press, Cambridge, MA, pp 367鈥?73
  11. Ding S, Zhu H, Jia W, Su C (2012) A survey on feature extraction for pattern recognition. Artif Intell Rev 37(3): 169鈥?80 CrossRef
  12. Duan KB, Keerthi SS, Poo AN (2003) Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing 51: 41鈥?9 CrossRef
  13. Girolami M, Rogers S (2005) Hierarchic Bayesian models for kernel learning. In: Proceedings of the 22nd international conference on machine learning, Bonn, Germany, pp 241鈥?48
  14. G枚nen M, Alpay谋n E (2011) Multiple kernel learning algorithms. J Mach Learn Res 12: 2211鈥?268
  15. Guermeur Y, Lifchitz A, Vert R (2004) A kernel for protein secondary structure prediction. In: Sch枚lkopf B, Tsuda K, Vert JP (eds) Kernel Methods in Computational Biology, MIT Press, Cambridge, MA, pp 193鈥?06
  16. Hoi SCH, Lyu MR, Chang EY (2006) Learning the unified kernel machines for classification. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, Philadelphia, USA, pp 187鈥?96
  17. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2): 415鈥?25 CrossRef
  18. Igel C, Glasmachers T, Mersch B, Pfeifer N, Meinicke P (2007) Gradient-based optimization of kernel-target聽alignment for sequence kernels applied to bacterial gene start detection. IEEE/ACM Trans Comput Biol Bioinform 4(2): 216鈥?26 CrossRef
  19. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1): 4鈥?7 CrossRef
  20. Kandola J, Shawe-Taylor J, Cristianini N (2002a) On the extensions of kernel alignment. Technical report 120, Department of Computer Science, University of London
  21. Kandola J, Shawe-Taylor J, Cristianini N (2002b) Optimizing kernel alignment over combinations of kernels. Technical report 121, Department of Computer Science, University of London
  22. Kawanabe M, Nakajima S, Binder A (2009) A procedure of adaptive kernel combination with kernel-target聽alignment for object classification. In: Proceedings of the 8th ACM international conference on image and video retrieval, Santorini Island, Greece
  23. Keerthi SS, Lin CJ (2003) Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Comput 15(7): 1667鈥?689 CrossRef
  24. Lanckriet GRG, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5: 27鈥?2
  25. Lee SW, Bien Z (2010) Representation of Fisher criterion function in a kernel feature space. IEEE Trans Neural Netw 21(2): 333鈥?39 CrossRef
  26. Liu Y, Liao S, Hou Y (2011) Learning kernels with upper bounds of leave-one-out error. In: Proceedings of the 20th ACM conference on information and knowledge management, Glasgow, UK, pp 2205鈥?208
  27. Meila M (2003) Data centering in feature space. In: Proceedings of the 9th international workshop on artificial intelligence and statistics, Key West, USA
  28. Neumann J, Schn枚rr C, Steidl G (2005) Combined SVM-based feature selection and classification. Mach Learn 61(1鈥?): 129鈥?50 CrossRef
  29. Nguyen CH, Ho TB (2008) An efficient kernel matrix evaluation measure. Pattern Recognit 41(11): 3366鈥?372 CrossRef
  30. Ong CS, Williamson RC (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6: 1043鈥?071
  31. Pothin J-B, Richard C (2006) A greedy algorithm for optimizing the kernel alignment and the performance of kernel machines. In: Proceedings of 14th European signal processing conference, Florence, Italy, pp 4鈥?
  32. Pothin J-B, Richard C (2007) Optimal feature representation for kernel machines using kernel-target聽alignment criterion. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing, Honolulu, USA, vol 3, pp 1065鈥?068
  33. Pothin J-B, Richard C (2008) Optimizing kernel alignment by data translation in feature space. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing, Las Vegas, USA, pp 3345鈥?348
  34. Qiu S, Lane T (2009) A framework for multiple kernel support vector regression and its applications to siRNA efficacy prediction. IEEE/ACM Trans Comput Biol Bioinf 6(2): 190鈥?99 CrossRef
  35. Ramona M, Richard G, David B (2012, to appear) Multiclass feature selection with kernel Gram-matrix-based criteria. IEEE Trans Neural Netw Learn Syst
  36. Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Learn Res 5: 101鈥?41
  37. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, New York CrossRef
  38. Tsochantaridis I, Joachims T, Hofmann T, Altun Y (2005) Large margin methods for structured and interdependent output variables. J Mach Learn Res 6: 1453鈥?484
  39. Vapnik V (1998) Statistical learning theory. Wiley, New York
  40. Vert R (2002) Designing a m-SVM kernel for protein secondary structure prediction. Master鈥檚 thesis, DEA Informatique de Lorraine
  41. Wang L (2008) Feature selection with kernel class separability. IEEE Trans Pattern Anal Mach Intell 30(9): 1534鈥?546 CrossRef
  42. Wang L, Xue P, Chan KL (2008) Two criteria for model selection in multiclass support vector machines. IEEE Trans Syst Man Cybern B Cybern 38(6): 1432鈥?448 CrossRef
  43. Wang T, Tian S, Huang H, Deng D (2009) Learning by local kernel polarization. Neurocomputing 72(13鈥?5): 3077鈥?084 CrossRef
  44. Wong WWL, Burkowski FJ (2009) Using kernel alignment to select features of molecular descriptors in a QSAR study. IEEE/ACM Trans Comput Biol Bioinform 8(5): 1373鈥?384 CrossRef
  45. Wu M, Farquhar J (2007) A subspace kernel for nonlinear feature extraction. In: Proceedings of the 20th international joint conference on artificial intelligence, Hyderabad, India, pp 1125鈥?130
  46. Xiong H, Swamy MNS, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2): 460鈥?74 CrossRef
  47. Zhu X, Kandola J, Ghahramani Z, Lafferty J (2004) Nonparametric transforms of graph kernels for semi-supervised learning. In: Saul LK, Weiss Y, Bottou L (eds) Advances in Neural Information Processing Systems 17, MIT Press, Cambridge, MA
  
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Science, general
  Complexity
  
• 出版者：Springer Netherlands
• ISSN：1573-7462

文摘

The success of kernel methods is very much dependent on the choice of kernel. Kernel design and learning a kernel from the data require evaluation measures to assess the quality of the kernel. In recent years, the notion of kernel alignment, which measures the degree of agreement between a kernel and a learning task, is widely used for kernel selection due to its effectiveness and low computational complexity. In this paper, we present an overview of the research progress of kernel alignment and its applications. We introduce the basic idea of kernel alignment and its theoretical properties, as well as the extensions and improvements for specific learning problems. The typical applications, including kernel parameter tuning, multiple kernel learning, spectral kernel learning and feature selection and extraction, are reviewed in the context of classification framework. The relationship between kernel alignment and other evaluation measures is also explored. Finally, concluding remarks and future directions are presented.