增量学习及其在图像识别中的应用
|
摘要
我们生活在一个信息爆炸的时代,增量学习成为处理这些每天都在增加的信息的唯一手段。同时,随着并行计算技术的发展,基于模块化的、并行的增量学习算法成为一个新的研究方向。高斯零交叉函数最小最大模块化网络(M~3-GZC)正是一种具有模块化结构、并行计算能力和增量学习能力的分类器。然而,M~3-GZC网络的模块数量是样本数量的平方级别,导致了平方级别的时间复杂度和空间复杂度,限制了M~3-GZC网络在大规模问题中的应用。另一方面,M~3-GZC网络的增量学习能力是建立在完全实例空间的,这导致了过高的存储空间需求,影响了分类精度。本文通过对M~3-GZC网络的深入分析,提出了去除冗余模块算法和新的增量学习算法,并将它们成功应用到工业图像故障检测、性别识别、手写体输入识别等图像识别问题中去。本文的主要贡献在以下几个方面。
1)本文分析了M~3-GZC网络的特点:高度的模块化结构、一定的增量学习能力、学习的收敛性以及具有给出“不知道”输出的能力。为了更好地理解M~3-GZC网络,本文讨论了它与两种常用模型——最近邻算法以及径向基函数网络的关系。
2)本文通过对M~3-GZC网络接收域特点的分析,提出了一种M~3-GZC网络的结构修剪算法,去除冗余模块,从而减少存储空间需求,加快响应速度。本文在一些公共数据集上验证了该修剪算法的有效性,并将它成功地应用到一个工业图像故障检测项目。
3)为了将M~3-GZC网络的增量学习能力从基于完全的实例空间转变为基于部分的实例空间,本文提出了高门限增量检测算法和有监督的增量聚类算法。前者可以有选择地学习新样本中的代表性样本,并能对已经训练好的M~3-GZC网络进一步去除冗余样本。后者则在学习过程中将训练样本进行聚类。这两种算法使得M~3-GZC网络具有了真正意义上的增量学习能力。
4)为了进一步提高M~3-GZC网络的增量学习能力,本文从概念空间学习的角度,提出了层式支持向量机。它的主要思想是利用先验知识,将大规模复杂问题分解为若干子问题,然后分别求解这些子问题。测试时,首先判断测试样本属于哪个子问题,然后由该子问题对应的分类器决定输出。在层式支持向量机的基础之上,通过合并实例空间与概念空间,本文提出了基于M~3-GZC网络和支持向量机的增量学习算法。它根据每个训练数据子集建立对应的支持向量机,同时不断更新M~3-GZC网络。测试时,首先由M~3-GZC网络判断测试样本属于哪些训练子问题,然后通过概率合并对应支持向量机的输出,给出最终结果。本文成功地将上述两种算法应用到多角度性别识别、手写体输入识别等领域。
5)为了更好地将增量学习算法应用到图像识别领域,本文提出了多尺度边缘增强的图像预处理方法和自适应图像欧氏距离的图像相似性测量方法。由于在图像中,边缘通常反应图像的形状和结构,而非边缘部分通常受照明的影响产生灰度的变化。因此,本文提出一种多尺度边缘增强算法,它可以在图像的预处理阶段强化边缘信息,去除噪声、光照的影响。另一方面,在图像的相似性测量中,常用的欧氏距离忽略了图像结构,并不能正确表达图像间的距离。本文提出的自适应图像欧氏距离则考虑了像素之间的几何相关性和灰度相关性,并且可以很容易地嵌入到各种现有的模式识别算法中去。
In the era of information explosion,incremental learning becomes the only way of processing the information accumulated every day.Moreover,as the development of parallel computing,incremental learning based on modularized structure and parallelization becomes a new research area.The Min-Max Modular Network with Gaussian-Zero-Crossing Functions(M~3-GZC) is a modular classifier which is capable for parallel computing and incremental learning.However,the number of modules in a M~3-GZC network is quadratic complexity with the number of training instances,which results in quadratic time and space complexity and limits the application of M~3-GZC network in large-scale problems.On the other hand,the incremental learning ability of M~3-GZC network is based on full instance memory,which leads to the high requirement in space and limits the classification accuracy. In this dissertation,we analyze the characteristics of M~3-GZC network thoroughly and propose a redundant module removing algorithm and some new incremental learning algorithms. We also apply these algorithms to some image recognition fields,such as industry image fault detection,gender classification and handwritten digital recognition.The main contributions of this dissertation can be described as follows.
1) We reveal that M~3-GZC network has the following attractive features:the highly modular structure,the ability of incremental learning in a certain extent,the guarantee of learning convergence,and the ability of saying 'unknown' to unfamiliar inputs.We also discuss the relationship between M~3-GZC network and two traditional models,the nearest neighbor algorithm and the radius-basis function network for better understanding of M~3-GZC network.
2) We propose a structure pruning algorithm to remove redundant modules based on the analysis of the receptive fields in M~3-GZC network.We validate the algorithm on several benchmark data sets,and apply it successfully to an industry image fault detection project.
3) To change the incremental learning abilities of M~3-GZC network from full instance memory based to partial instance memory based,we propose an enhanced threshold incremental check algorithm and a supervised clustering algorithm.The former can select representative samples from a new training set and prune redundant modules in an already trained M~3-GZC network.While the latter can cluster the training data during learning.The proposed algorithms endow the M~3-GZC network with the truly incremental learning ability.
4) To improve the incremental learning of M~3-GZC network further,we propose a layered support vector machine based on the learning in concept memories at first.The fundamental idea of it is dividing a complicated and large-scale problem into several easy subproblems according to prior knowledge,and then solving these subproblems in parallel.During the test process,it decides which subproblem the test sample belongs to at first,and then gives the final output according to the corresponding support vector machine.Based on the layered support vector machine,we combine the instance memory as well as concept memory, and propose an incremental learning algorithm based on M~3-GZC network and support vector machines.It trains a support vector machine according to each training data subset, and updates a M~3-GZC network at the same time.During the test,the M~3-GZC network decides which subproblem the test sample belongs to,and combines the output of the corresponding support vector machines.We apply the two algorithms successfully to some image recognition fields,such as multi-view gender classification and handwritten digit recognition.
5) We propose a multi-scale edge enhancement algorithm and an adaptive image Euclidean distance for better image recognition.In images,edges are often locate at the boundaries of important image structures and reflect shapes,while non-edge areas are often changed in gray level under the influence of illumination.Therefor,we propose a multiscale edge enhancement algorithm which can intensify the edge information and remove the effects of illumination and noises.While in the image similarity measures,the most commonly used Euclidean distance discards the image structures and is not reasonable for image distance.On the contrary,our proposed adaptive image Euclidean distance considers both the spatial relationship and the gray level relationship between pixels,and can be easily embedded in many existing pattern recognition techniques.
1)本文分析了M~3-GZC网络的特点:高度的模块化结构、一定的增量学习能力、学习的收敛性以及具有给出“不知道”输出的能力。为了更好地理解M~3-GZC网络,本文讨论了它与两种常用模型——最近邻算法以及径向基函数网络的关系。
2)本文通过对M~3-GZC网络接收域特点的分析,提出了一种M~3-GZC网络的结构修剪算法,去除冗余模块,从而减少存储空间需求,加快响应速度。本文在一些公共数据集上验证了该修剪算法的有效性,并将它成功地应用到一个工业图像故障检测项目。
3)为了将M~3-GZC网络的增量学习能力从基于完全的实例空间转变为基于部分的实例空间,本文提出了高门限增量检测算法和有监督的增量聚类算法。前者可以有选择地学习新样本中的代表性样本,并能对已经训练好的M~3-GZC网络进一步去除冗余样本。后者则在学习过程中将训练样本进行聚类。这两种算法使得M~3-GZC网络具有了真正意义上的增量学习能力。
4)为了进一步提高M~3-GZC网络的增量学习能力,本文从概念空间学习的角度,提出了层式支持向量机。它的主要思想是利用先验知识,将大规模复杂问题分解为若干子问题,然后分别求解这些子问题。测试时,首先判断测试样本属于哪个子问题,然后由该子问题对应的分类器决定输出。在层式支持向量机的基础之上,通过合并实例空间与概念空间,本文提出了基于M~3-GZC网络和支持向量机的增量学习算法。它根据每个训练数据子集建立对应的支持向量机,同时不断更新M~3-GZC网络。测试时,首先由M~3-GZC网络判断测试样本属于哪些训练子问题,然后通过概率合并对应支持向量机的输出,给出最终结果。本文成功地将上述两种算法应用到多角度性别识别、手写体输入识别等领域。
5)为了更好地将增量学习算法应用到图像识别领域,本文提出了多尺度边缘增强的图像预处理方法和自适应图像欧氏距离的图像相似性测量方法。由于在图像中,边缘通常反应图像的形状和结构,而非边缘部分通常受照明的影响产生灰度的变化。因此,本文提出一种多尺度边缘增强算法,它可以在图像的预处理阶段强化边缘信息,去除噪声、光照的影响。另一方面,在图像的相似性测量中,常用的欧氏距离忽略了图像结构,并不能正确表达图像间的距离。本文提出的自适应图像欧氏距离则考虑了像素之间的几何相关性和灰度相关性,并且可以很容易地嵌入到各种现有的模式识别算法中去。
In the era of information explosion,incremental learning becomes the only way of processing the information accumulated every day.Moreover,as the development of parallel computing,incremental learning based on modularized structure and parallelization becomes a new research area.The Min-Max Modular Network with Gaussian-Zero-Crossing Functions(M~3-GZC) is a modular classifier which is capable for parallel computing and incremental learning.However,the number of modules in a M~3-GZC network is quadratic complexity with the number of training instances,which results in quadratic time and space complexity and limits the application of M~3-GZC network in large-scale problems.On the other hand,the incremental learning ability of M~3-GZC network is based on full instance memory,which leads to the high requirement in space and limits the classification accuracy. In this dissertation,we analyze the characteristics of M~3-GZC network thoroughly and propose a redundant module removing algorithm and some new incremental learning algorithms. We also apply these algorithms to some image recognition fields,such as industry image fault detection,gender classification and handwritten digital recognition.The main contributions of this dissertation can be described as follows.
1) We reveal that M~3-GZC network has the following attractive features:the highly modular structure,the ability of incremental learning in a certain extent,the guarantee of learning convergence,and the ability of saying 'unknown' to unfamiliar inputs.We also discuss the relationship between M~3-GZC network and two traditional models,the nearest neighbor algorithm and the radius-basis function network for better understanding of M~3-GZC network.
2) We propose a structure pruning algorithm to remove redundant modules based on the analysis of the receptive fields in M~3-GZC network.We validate the algorithm on several benchmark data sets,and apply it successfully to an industry image fault detection project.
3) To change the incremental learning abilities of M~3-GZC network from full instance memory based to partial instance memory based,we propose an enhanced threshold incremental check algorithm and a supervised clustering algorithm.The former can select representative samples from a new training set and prune redundant modules in an already trained M~3-GZC network.While the latter can cluster the training data during learning.The proposed algorithms endow the M~3-GZC network with the truly incremental learning ability.
4) To improve the incremental learning of M~3-GZC network further,we propose a layered support vector machine based on the learning in concept memories at first.The fundamental idea of it is dividing a complicated and large-scale problem into several easy subproblems according to prior knowledge,and then solving these subproblems in parallel.During the test process,it decides which subproblem the test sample belongs to at first,and then gives the final output according to the corresponding support vector machine.Based on the layered support vector machine,we combine the instance memory as well as concept memory, and propose an incremental learning algorithm based on M~3-GZC network and support vector machines.It trains a support vector machine according to each training data subset, and updates a M~3-GZC network at the same time.During the test,the M~3-GZC network decides which subproblem the test sample belongs to,and combines the output of the corresponding support vector machines.We apply the two algorithms successfully to some image recognition fields,such as multi-view gender classification and handwritten digit recognition.
5) We propose a multi-scale edge enhancement algorithm and an adaptive image Euclidean distance for better image recognition.In images,edges are often locate at the boundaries of important image structures and reflect shapes,while non-edge areas are often changed in gray level under the influence of illumination.Therefor,we propose a multiscale edge enhancement algorithm which can intensify the edge information and remove the effects of illumination and noises.While in the image similarity measures,the most commonly used Euclidean distance discards the image structures and is not reasonable for image distance.On the contrary,our proposed adaptive image Euclidean distance considers both the spatial relationship and the gray level relationship between pixels,and can be easily embedded in many existing pattern recognition techniques.
引文
[1]H.W.Coppock and J.E.Freund,"All-or-none versus incremental learning of errorless shock escapes by the rat," Science,vol.135,no.3500,pp.318-319,1962.
[2]R.Polikar,J.Byorick,S.Krause,A.Marino,and M.Moreton,"Learn++:a classifier independent incremental learning algorithmfor supervised neural networks," Proceedings of the International Joint Conference on Neural Networks,vol.2,pp.1742-1747,2002.
[3]D.P.Helmbold and P.M.Long,"Tracking drifting concepts by minimizing disagreements," Machine Learning,vol.14,no.1,pp.27-45,1994.
[4]A.Kuh,T.Petsche,and R.L.Rivest,"Learning time-varying concepts," Proceedings of the conference on Advances in neural information processing,pp.183-189,1990.
[5]R.Klinkenberg and I.Renz,"Adaptive information filtering:Learning in the presence of concept drifts," Learning for Text Categorization,pp.33-40,1998.
[6]G.Widmer and M.Kubat,"Learning in the presence of concept drift and hidden contexts," Machine Learning,vol.23,no.1,pp.69-101,1996.
[7]R.Klinkenberg and T.Joachims,"Detecting concept drift with support vector machines," Proceedings of the seventeenth International Conference on Machine Learning,pp.11-18,2000.
[8]R.Klinkenberg and S.Ruping,"Concept drift and the importance of examples," Text Mining-Theoretical Aspects and Applications,pp.55-77,2003.
[9]R.Klinkenberg,"Learning drifting concepts:Example selection vs.example weighting," Intelligent Data Analysis,vol.8,no.3,pp.281-300,2004.
[10]D.W.Aha,D.Kibler,and M.K.Albert,"Instance-based learning algorithms," Machine Learning,vol.6,no.1,pp.37-66,1991.
[11]M.A.Maloof and R.S.Michalski,"A method for partial-memory incremental learning and its application to computer intrusion detection," Proceedings of the seventh IEEE International Conference on Tools with Artificial Intelligence,pp.392-397,1995.
[12]M.A.Maloof and R.S.Michalski,"Selecting examples for partial memory learning," Machine Learning,vol.41,no.1,pp.27-52,2000.
[13]M.A.Maloof and R.S.Michalski,"Incremental learning with partial instance memory," Artificial Intelligence,vol.154,no.1-2,pp.95-126,2004.
[14]T.Cover and P.Hart,"Nearest neighbor pattern classification," IEEE Transactions on Information Theory,vol.13,no.1,pp.21-27,1967.
[15]J.R.Bunch and C.P.Nielsen,"Updating the singular value decomposition," Numerische Mathematik,vol.31,no.2,pp.111-129,1978.
[16]J.R.Bunch,C.P.Nielsen,and D.C.Sorensen,"Rank-one modification of the symmetric eigenproblem,"Numerische Mathematik,vol.31,no.1,pp.31-48,1978.
[17]R.D.Degroat and R.A.Roberts,"Efficient,numerically stabilized rank-one eigenstructure updating,"IEEE Transactions on Signal Processing,vol.38,no.2,pp.301-316,1990.
[18]H.Murakami and B.Kumar,"Efficient calculation of primary images from a set of images," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.4,pp.511-515,1982.
[19]S.Chaudhuri,S.Sharma,and S.Chatterjee,"Recursive estimation of motion parameters," Computer Vision andlmage Understanding,vol.64,no.3,pp.434-442,1996.
[20]S.Chandrasekaran,B.S.Manjunath,Y.F.Wang,J.Winkeler,and H.Zhang,"An eigenspace update algorithm for image analysis," Graphical Models and Image Processing,vol.59,no.5,pp.321-332,1997.
[21]P.Hall,D.Marshall,and R.Martin,"Incremental eigenanalysis for classification," British Machine Vision Conference,pp.286-295,1998.
[22]D.Skocaj and A.Leonardis,"Weighted and robust incremental method for subspace learning," Proceedings of the ninth IEEE International Conference on Computer Vision,pp.1494-1501,2003.
[23]M.Artac,M.Jogan,and A.Leonardis,"Incremental PCA for on-line visual learning and recognition,"Proceedings of the sixteenth International Conference on Pattern Recognition,vol.3,pp.781-784,2002.
[24]Y.Li,"On incremental and robust subspace learning," Pattern Recognition,vol.37,no.7,pp.1509-1518,2004.
[25]W.Lam,C.K.Keung,and D.Liu,"Discovering useful concept prototypes for classification based on filtering and abstraction," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.24,no.8,pp.1075-1090,2002.
[26]R.M.Cameron-Jones,"Instance selection by encoding length heuristic with random mutation hill climbing," Proceedings of the eighth Australian Joint Conference on Artificial Intelligence,pp.99-106,1995.
[27]G.Gates,"The reduced nearest neighbor rule," IEEE Transactions on Information Theory,vol.18,no.3,pp.431-433,1972.
[28]D.B.Skalak,"Prototype and feature selection by sampling and random mutation hill climbing algorithms," Proceedings of the eleventh International Conference on Machine Learning,pp.293-301,1994.
[29]D.L.Wilson,"Asymptotic properties of nearest neighbor rules using edited data," IEEE Transactions on Systems,Man,and Cybernetics,vol.2,no.3,pp.408-421,1972.
[30]B.E.Boser,I.M.Guyon,and V.N.Vapnik,"A training algorithm for optimal margin classifiers,"Proceedings of the fifth annual workshop on Computational learning theory,pp.144-152,1992.
[31]C.Cortes and V.Vapnik,"Support-vector networks," Machine Learning,vol.20,no.3,pp.273-297,1995.
[32]D.Mattera,F.Palmieri,and S.Haykin,"An explicit algorithm for training support vector machines,"Signal Processing Letters,IEEE,vol.6,no.9,pp.243-245,1999.
[33]萧嵘,王继成,孙正兴,张福炎,“一种SVM增量学习算法,”南京大学学报,vol.38,no.2,pp.152-157,2002.
[34]孔锐,张冰,“一种快速支持向量机增量学习算法,”控制与决策,vol.20,no.10,pp.1129-1136,2005.
[35]安金龙,王正欧,“一种适合于增量学习的支持向量机的快速循环算法,”计算机应用,vol.23,no.10,pp.12-14,2003.
[36]滕月阳,唐焕文,张海霞,“一种新的支持向量机增量学习算法,”计算机工程与应用,vol.40,no.36,pp.77-80,2004.
[37]N.Syed,H.Liu,and K.K.Sung,"Incremental learning with support vector machines," Proceedings of the International Joint Conference on Artificial Intelligence,1999.
[38]P.Mitra,C.A.Murthy,and S.K.Pal,"Data condensation in large databases by incremental learning with support vector machines," Proceedings of the International Conference on Pattern Recognition,vol.2,pp.708-711,2000.
[39]曾嵘,刘建成,蒋新华,“一种基于支持向量机的增量学习算法,”铁道科学与工程学报,vol.2,no.1,pp.94-96,2005.
[40]曾文华,马健,“一种新的支持向量机增量学习算法,”厦门大学学报:自然科学版,vol.41,no.6,pp.687-691,2002.
[41]李凯,黄厚宽,“支持向量机增量学习算法研究,”北方交通大学学报,vol.27,no.5,pp.34-37,2003.
[42]C.Diehl and G.Cauwenberghs,"SVM incremental learning,adaptation and optimization," Proceedings of the International Joint Conference on Neural Networks,pp.2685-2690,2003.
[43]N.A.Syed,H.Liu,and K.K.Sung,"Handling concept drifts in incremental learning with support vector machines," Proceedings of the fifth international conference on Knowledge discovery and data mining,pp.317-321,1999.
[44]R.Polikar,L.Upda,S.S.Upda,and V.Honavar,"Learn++:an incremental learning algorithm for supervised neuralnetworks," IEEE Transactions on Systems,Man and Cybernetics,Part C,vol.31,no.4,pp.497-508,2001.
[45]M.Muhlbaier,A.Topalis,and R.Polikar,"Incremental learning from unbalanced data," Proceedings of the IEEE International Joint Conference on Neural Networks,vol.2,pp.1057-1062,2004.
[46]M.Muhlbaier,A.Topalis,and R.Polikar,"Learn++.MT:a new approach to incremental learning,"Proceedings of the fifth International Workshop on Multiple Classifier Systems,pp.52-61,2004.
[47]Z.Erdem,R.Polikar,F.Gurgen,and N.Yumusak,"Ensemble of SVMs for incremental learning,"Proceedings of the sixth International Workshop on Multiple Classifier Systems,pp.246-256,2005.
[48]A.Gangardiwala and R.Polikar,"Dynamically weighted majority voting for incremental learning and comparison of three boosting based approaches," Proceedings of the IEEE International Joint Conference on Neural Networks,vol.2,pp.1131-1136,2005.
[49]H.S.Mohammed,J.Leander,M.Marbach,and R.Polikar,"Can AdaBoost.M1 learn incrementally?A comparison to Learn++ under different combination rules," Lecture Notes in Computer Science,vol.4131,pp.254-263,2006.
[50]R.Polikar,L.Udpa,S.Udpa,and V.Honavar,"An incremental learning algorithm with confidence estimation for automated identification of NDE signals," IEEE Transactions on Ultrasonics,Ferroelectrics and Frequency Control,vol.51,no.8,pp.990-1001,2004.
[51]W.Buntine,"Theory refinement on Bayesian networks," Proceedings of the seventh Conference on Uncertainty in Artificial Intelligence,pp.52-60,1991.
[52]N.Friedman and M.Goldszmidt,"Sequential update of Bayesian network structure," Proceedings of the thirteenth Conference on Uncertainty in Artificial Intelligence,1997.
[53]J.Route,"An incremental algorithm for tree-shaped Bayesian network learning," Proceedings of the European Conference of Artificial Intelligence,2002.
[54]田凤占,黄丽,于剑,黄厚宽,“包含隐变量的贝叶斯网络增量学习方法,”电子学报,vol.33,no.11,pp.1925-1928,2005.
[55]王飞,刘大有,下淞昕,“基于遗传算法的Bayesian网结构增量学习的研究,”计算机研究与发展,vol.42,no.9,pp.1461-1466,2005.
[56]桑农,曾坤,“基于RBF网络的增量学习,”华中科技大学学报,vol.32,pp.214-216,2004.
[57]K.Okamato,S.Ozawa,and S.Abe,"A fast incremental learning algorithm of RBF networks with long-term memory," Proceedings of the International Joint Conference on Neural Networks,pp.102-107,2003.
[58]骆斌,周志华,“一个增量式判定树学习算法INDUCE,”计算机研究与发展,vol.36,no.5,pp.518-522,1999.
[59]H.Nakayama and A.Hattori,"Incremental learning and forgetting in RBF networks and SVMs with applications to financial problems," vol.2773,pp.1109-1115,2003.
[60]G.Tontini and A.A.De Queiroz,"RBF fuzzy-artmap:a new fuzzy neural network for robust on-linelearning and identification of patterns," IEEE International Conference on Systems,Man,and Cybernetics,vol.2,pp.1364-1369,1996.
[61]C.P.Lim and R.F.Harrison,"Probabilistic Fuzzy ARTMAP:an autonomous neural networkarchitecture for Bayesian probability estimation," Proceedings of the fourth International Conference on Artificial Neural Networks,pp.148-153,1995.
[62]B.L.Lu and M.Ito,"Task decomposition based on class relations:a modular neural network architecture for pattern classification," Lecture Notes in Computer Science,vol.1240,pp.330-339,1997.
[63]B.L.Lu and M.Ito,"Task decomposition and module combination based on class relations:a modular neural network for pattern classification," IEEE Transactions on Neural Networks,vol.10,no.5,pp.1244-1256,1999.
[64]R.E.Jenkins and B.P.Yuhas,"A simplified neural network solution through problem decomposition:the case of the truck backer-upper," IEEE Transactions on Neural Networks,vol.4,no.4,pp.718-720,1993.
[65]C.H.Chen and G.H.You,"Class-sensitive neural network," Neural Parallel Scientific Computing,vol.1,no.1,pp.93-96,1993.
[66]L.Xu,A.Krzyzak,and C.Y.Suen,"Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Transactions on Systems Man and Cybernetics,vol.22,no.3,pp.418-435,1992.
[67]K.Chen,L.Wang,and H.Chi,"Methods of combining multiple classifiers with different features and their applications to text-independent speaker identification," International Journal of Pattern Recognition and Artificial Intelligence,vol.11,no.3,pp.417-445,1997.
[68]J.A.Robert,J.I.Michael,and G.A.Barto,"Task decomposition through competition in a modular connectionist architecture:the what and where vision tasks," Cognitive Science,vol.15,pp.219-250,1991.
[69]R.E.Schapire,"The strength of weak learnability," Machine Learning,vol.5,no.2,pp.197-227,1990.
[70]K.Turner and J.Ghosh,"Analysis of decision boundaries in linearly combined neural classifiers,"Pattern Recognition,vol.29,no.2,pp.341-348,1996.
[71]R.Battiti and A.M.Colla,"Democracy in neural nets:voting schemes for classification," Neural Networks,vol.7,no.4,pp.691-707,1994.
[72]K.Tumer and J.Ghosh,"Order statistics combiners for neural classifiers," Proceedings of the World Congress on Neural Networks,pp.31-34,1995.
[73]W.B.Langdon and B.F.Buxton,"Genetic programming for combining classifiers," Proceedings of the Genetic and Evolutionary Computation Conference,pp.66-73,2001.
[74]B.L.Lu,K.A.Wang,M.Utiyama,and H.Isahara,"A part-versus-pat method for massively parallel training of support vector machines," Proceedings of the IEEE International Joint Conference on Neural Networks,pp.735-740,2004.
[75]K.A.Wang,H.Zhao,and B.L.Lu,"Task decomposition using geometric relation for min-max modular SVMs," Lecture Notes in Computer Science,vol.3496,pp.887-892,2005.
[76]H.C.Lian,B.L.Lu,E.Takikawa,and S.Hosoi,"Gender recognition using a min-max modular support vector machine," Lecture Notes in Computer Science,vol.3611,pp.438-441,2005.
[77]B.L.Lu,Q.Ma,M.Ichikawa,and H.Isahara,"Efficient part-of-speech tagging with a min-max modular neural-network model," Applied Intelligence,vol.19,no.1,pp.65-81,2003.
[78]B.L.Lu,Q.Ma,,and M.Ichikawa,"Massively parallel classification of single-trial LEG signals using a min-max modular neural network," IEEE Transactions on Biomedical Engineering,vol.51,no.3,pp.551-558,2004.
[79]Y.Yang and B.L.Lu,"Prediction of protein subcellular multi-locations with a min-max modular support vector machine," Lecture Notes in Computer Science,vol.3973,pp.667-673,2006.
[80]Z.G.Fan and B.L.Lu,"Multi-view face recognition with min-max modular SVMs," Lecture Notes in Computer Science,vol.3611,pp.396-399,2005.
[81]F.Y.Liu,K.Wu,H.Zhao,and B.L.Lu,"Fast text categorization with min-max modular support ector machines," IJCNN'05.Proceedings of the IEEE International Joint Conference on Neural Networks,vol.1,pp.570-575,2005.
[82]X.L.Chu,C.Ma,L.J.,B.L.Lu,M.Utiyama,and H.Isahara,"Large-scale patent classification with min-max modular support vector machines," International Joint Conference on Neural Networks,vol.1,pp.3972-3979,2008.
[83]B.L.Lu and M.Ichikawa,"A Gaussian zero-crossing discriminant function for min-max modular neural networks," Proceedings of the Fifth international conference on Knowledge-Based Intelligence Information Engineering Systems and Allied Technologies,pp.298-302,2001.
[84]B.L.Lu and M.Ichikawa,"Emergent online learning with a Gaussian zero-crossing discriminantfunction,"Proceedings of the International Joint Conference on Neural Networks,vol.2,pp.1263-1268,2002.
[85]C.Liu and H.Wechsler,"Independent component analysis of Gabor features for face recognition,"IEEE Transactions on Neural Networks,vol.14,no.4,pp.919-928,2003.
[86]C.Liu,"Gabor-based kernel PCA with fractional power polynomial models for face recognition,"IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.26,no.5,pp.572-581,2004.
[87]T.Ojala,M.Pietikainen,and T.Maenpaa,"Mnltiresolntion gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.24,no.7,pp.971-987,2002.
[88]I.H.Witten,T.C.Bell,and A.Moffat,Managing gigabytes:compressing and indexing documents and images.Morgan Kanfmann,1999.
[89]C.C.Aggarwal,A.Hinneburg,and D.Keim,On the surprising behavior of distance metrics in high dimensional space.Springer,2000.
[90]P.Simard,Y.LeCnn,and J.S.Denker,"Efficient pattern recognition using a new transformation distance," Advances in Neural Information Processing Systems,pp.50-58,1992.
[91]D.P.Huttenlocher,G.A.Klanderman,and W.J.Rncklidge,"Comparing images using the Hansdorff distance," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.15,no.9,pp.850-863,1993.
[92]E.P.Vivek and N.Sndha,"Robust Hansdofff distance measure for face recognition," Pattern Recognition,vol.40,no.2,pp.431-442,2007.
[93]C.H.T.Yang,S.H.Lai,and L.W.Chang,"Hybrid image matching combining Hansdofff distance with normalized gradient matching," Pattern Recognition,vol.40,no.4,pp.1173-1181,2007.
[94]S.Santini and R.Jain,"Similarity measures," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.21,no.9,pp.871-883,1999.
[95]D.W.Jacobs,D.Weinshall,and Y.Gdalyahn,"Classification with nonmetric distances:image retrieval and classrepresentation," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.22,no.6,pp.583-600,2000.
[96]X.Tan,S.Chen,J.Li,and Z.H.Zhon,"Learning non-metric partial similarity based on maximal margin criterion," Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,pp.168-145,2006.
[97]J.B.Tenenbanm,V.Silva,and J.C.Langford,"A global geometric framework for nonlinear dimensionality reduction," Science,vol.290,pp.2319-2323,2000.
[98]S.T.Roweis and L.K.Saul,"Nonlinear dimensionality reduction by locally linear embedding,"Science,vol.290,pp.2323-2326,2000.
[99]L.Wang,Y.Zhang,and J.Feng,"On the Euclidean distance of images," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.27,no.8,pp.1334-1339,2005.
[100]J.Chen,R.Wang,S.Shan,X.Chen,and W.Gao,“Isomap based on the image Euclidean distance,”Proceedings of the eighteenth International Conference on Pattern Recognition,vol.2,pp.1110-1113,2006.
[101]K.Han and X.C.Zhu,“Research on face recognition based on IMED and 2DPCA,”Journal of Electronics,vol.23,no.5,pp.786-790,2006.
[102]X.Mei,S.K.Zhou,and H.Wu,“Integrated detection,tracking and recognition for IR video-based vehicle classification,”IEEE International Conference on Acoustics,Speech and Signal Processing,vol.5,pp.745-748,2006.
[103]T.Tangkuampien and D.Suter,“3D object pose inference via kernel principal component analysis with image Euclidian distance(IMED),”Proceedings of the British Machine Vision Association,pp.137-146,2006.
[104]R.Wang,J.Chen,S.Shan,and W Gao,"Enhancing training set for face detection," Proceedings of the eighteenth International Conference on Pattern Recognition,vol.3,pp.477-80,2006.
[105]K.M.Lee and W Nick Street,“Incremental feature weight learning and its application to a shape-based query system,”Pattern Recognition Letters,vol.23,no.7,pp.865-874,2002.
[106]J.Weng,C.H.Evans,and W S.Hwang,“An incremental learning method for face recognition undercontinuous video stream,”Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition,pp.251-256,2000.
[107]R.Sukthankar and R.Stockton,“Argus:the digital doorman,”Intelligent Systems,vol.16,no.2,pp.14-19,2001.
[108]O.Deniz,M.Castrillon,J.Lorenzo,and M.Hernandez,“An incremental learning algorithm for face recognition,”Lecture Notes in Computer Science,vol.2359,pp.1-9,2002.
[109]H.Huifang,“The drifting and positioning of the concept of moral values among contemporary women college students,”Chinese Education And Society,vol.33,no.3,pp.23-28,2000.
[110]A.Pentland,B.Moghaddam,and T.Starner,“View-based and modular eigenspaces for face recognition,”IEEE Computer Society Conference on Computer Vision and Pattern Recognition,pp.84-91,1994.
[111]M.A.Turk and A.P.Pentland,“Face recognition using eigenfaces,”IEEE Computer Society Conference on Computer Vision and Pattern Recognition,pp.586-591,1991.
[112]W Zhu,J.Pei,J.Yin,and Y.Xie,“Granularity adaptive density estimation and on demand clustering of concept-drifting data streams,”Proceedings of the eighth International Conference on Data Warehousing and Knowledge Discovery,vol.4081,pp.322-331,2006.
[113]P.N.Belhumeur,J.P.Hespanha,and D.J.Kriegman,"Eigenfaces vs.Fisherfaces:recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.19,no.7,pp.711-720,1997.
[114]B.Moghaddam and A.Pentland,"Probabilistic visual learning for object representation," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.19,no.7,pp.696-710,1997.
[115]B.Moghaddam,T.Jebara,and A.Pentland,"Bayesian face recognition," Pattern Recognition,vol.33,no.11,pp.1771-1782,2000.
[116]W.Zhao,R.Chellappa,and A.Krishnaswamy,"Discriminant analysis of principal components for face recognition," Face Recognition:From Theory to Applications,pp.73-85,1998.
[117]P.J.Phillips,H.Moon,S.A.Rizvi,and P.J.Rauss,"The FERET evaluation methodology for facerecognition algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.22,no.10,pp.1090-1104,2000.
[118]I.Dagher and R.Nachar,"Face Recognition Using IPCA-ICA Algorithm," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.28,no.6,pp.996-1000,2006.
[119]H.Zhao,P.C.Yuen,J.T.Kwok,and J.Yang,"Incremental PCA based face recognition," Proceedings of the eighth International Conference on Control,Automation,Robotics and Vision,pp.687-691,2004.
[120]S.Ozawa,S.L.Toh,S.Abe,S.Pang,and N.Kasabov,"Incremental learning of feature space and classifier for face recognition," Neural Networks,vol.18,no.5-6,pp.575-584,2005.
[121]S.Ozawa,S.Toh,S.Abe,S.Pang,and N.Kasabov,"Incremental Learning for Online Face Recognition,"Neural Network,vol.18,no.5-6,pp.575-584,2005.
[122]C.L.Blake and C.J.Merz,"UCI repository of machine learning databases," 1998.
[123]B.Huang and B.L.Lu,"Fault diagnosis for industrial images using a min-max modular neural network," Lecture Notes in Computer Science,vol.3316,pp.842-847,2004.
[124]H.C.Lian and B.L.Lu,"An algorithm for pruning redundant modules in min-max modular network,"Proceedings of the 14th National Conference on Neural Network,pp.37-42,2004.
[125]H.C.Lian and B.L.Lu,"An algorithm for pruning redundant modules in min-max modular network,"Proceedings of the IEEE International Joint Conference on Neural Networks,vol.3.
[126]P.Hart,"The condensed nearest neighbor rule(Corresp.)," IEEE Transactions on Information Theory,vol.14,no.3,pp.515-516,1968.
[127]N.Grira,M.Crucianu,and N.Boujemaa,"Unsupervised and semi-supervised clustering:a brief survey," A Review of Machine Learning Techniques for Processing Multimedia Content,Report of the MUSCLE European Network of Excellence,2004.
[128]边肇祺,模式识别.清华大学出版社,2000.
[129]T.Kohonen,Self-organizing maps.Springer,2001.
[130]S.Basu,A.Banerjee,and R.J.Mooney,"Semi-supervised clustering by seeding," Proceedings of ninteenth International Conference on Machine Learning,pp.19-26,2002.
[131]T.Kohonen,"Improved versions of learning vector quantization," Proceedings of International Joint Conference on Neural Networks,pp.545-550,1990.
[132]N.Bansal,A.Blum,and S.Chawla,"Correlation clustering," Machine Learning,vol.56,no.1,pp.89-113,2004.
[133]C.M.Procopiuc,"Clustering problems and their applications(a survey)," Department of Computer Science,Duke University,1997.
[134]X.Li and N.Ye,"Grid and dummy cluster based learning of normal and intrusive clusters for computer intrusion detection," Quality and Reliability Engineering International,vol.18,no.3,pp.231-242,2002.
[135]X.Li and N.Ye,"A supervised clustering algorithm for computer intrusion detection," Knowledge and Information Systems,vol.8,no.4,pp.498-509,2005.
[136]R.F.Sproull,"Refinements to nearest-neighbor searching in k-dimensional trees," Algorithmica,vol.6,no.1,pp.579-589,1991.
[137]S.G.Mallat,"Multifrequency channel decompositions of images and wavelet models," IEEE Transactions on Acoustics,Speech,and Signal Processing,vol.37,no.12,pp.2091-2110,1989.
[138]A.Cohen and J.Kovacevic,"Wavelets:the mathematical background," Proceedings of the IEEE,vol.84,no.4,pp.514-522,1996.
[139]S.Mallat and S.Zhong,"Characterization of signals from multiscale edges," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.14,no.7,pp.710-732,1992.
[140]J.M.Lee,Riemannian manifolds:an introduction to curvature.Springer,1997.
[141]J.Jost,Riemannian geometry and geometric analysis.Springer,1995.
[142]H.S.Seung and D.D.Lee,"Cognition:the manifold ways of perception," Science,vol.290,no.5500,pp.2268-2269,2000.
[143]W.Gao,B.Cao,S.Shan,D.Zhou,X.Zhang,and D.Zhao,"The CAS-PEAL large-scale Chinese face database and baseline evaluations," Technical report of JDL,http://www.jdl.ac.cn/peal/peal tr.pdf,2004.
[144]D.B.Graham and N.M.Allinson,"Characterizing virtual eigensignatures for general purpose face recognition," Face Recognition:From Theory to Applications.NATO ASI Series F,Computer and Systems Sciences,vol.163,pp.446-456,1998.
[145]Y.LeCun,"The MNIST database of handwritten digits," Available online at:http://yann.lecun.com/exdb/mnist/index.html.
[146]B.Moghaddam and M.H.Yang,"Learning gender with support faces," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.24,no.5,pp.707-711,2002.
[147]H.C.Lian and B.L.Lu,"Multi-view gender classification using multi-resolution local binary patterns and support vector machines," International Journal of Neural Systems,vol.17,no.6,pp.479-487,2007.
[148]J.Li and B.L.Lu,"A framework for multi-view gender classification," Lecture Notes in Computer Science,vol.4984,pp.973-982,2007.
[149]C.C.Chang and C.J.Lin,"LIBSVM:a library for support vector machines,"http://www.csie.ntu.edu.tw/cjlin/libsvm,2001.
[150]S.Gutta,J.R.J.Huang,P.Jonathon,and H.Wechsler,"Mixture of experts for classification of gender,ethnic origin,and pose of human faces," IEEE Transactions on Neural Networks,vol.11,no.4,pp.948-960,2000.
[151]A.Khan,"Combination and optimization of classifiers in gender classification using genetic programming,"International Journal of Knowledge-Based and Intelligent Engineering Systems,vol.9,no.1,pp.1-11,2005.
[152]H.C.Lian and B.L.Lu,"Multi-view gender classification using local binary patterns and support vector machines," Lecture Notes in Computer Science,vol.3972,pp.202-209,2006.
[153]J.Luo and B.L.Lu,"Gender recognition using a min-max modular support vector machine with equal clustering," Lecture Notes in Computer Science,vol.3972,pp.210-215,2006.
[154]H.C.Kim,D.Kim,Z.Ghahramani,and S.Y.Bang,"Appearance-based gender classification with Gaussian processes," Pattern Recognition Letters,vol.27,no.6,pp.618-626,2006.
[155]K.Balci and V.Atalay,"PCA for gender estimation:Which eigenvectors contribute," Proceedings of Sixteenth International Conference on Pattern Recognition,vol.3,pp.363-366,2002.
[156]A.Jain and J.Huang,"Integrating independent components and linear discriminant analysis for gender classification," Sixth IEEE International Conference on Automatic Face and Gesture Recognition,pp.159-163,2004.
[157]A.J.O' Toole,K.A.Deffenbacher,D.Valentin,K.McKee,D.Huff,and H.Abdi,"The perception of face gender:The role of stimulus structure in recognition and classification," Memory and Cognition,vol.26,no.1,pp.146-160,1998.
[158]G.W.Cottrell and J.Metcalfe,"EMPATH:face,emotion,and gender recognition using holons,"Proceedings of the Conference On Advances in Neural Information Processing Systems,pp.564-571,1990.
[159]B.Edelman,D.Valentin,and H.Abdi,“Sex classification of face areas:how well can a linear neural network predict human performance,”Journal of Biological System,vol.6,no.3,pp.241-264,1998.
[160]B.A.Golomb,D.T.Lawrence,and T.J.Sejnowski,“SexNet:A neural network identifies sex from human faces,”Proceedings of the 1990 conference on Advances in neural information processing,pp.572-577,1990.
[161]J.C.Platt,“Fast training of support vector machines using sequential minimal optimization,”Advances in kernel methods:support vector learning,pp.185-208,1999.
[162]J.Huang,X.Shao,and H.Wechsler,“Face pose discrimination using support vector machines(SVM),”Fourteenth International Conference on Pattern Recognition,pp.154-156,1998.
[2]R.Polikar,J.Byorick,S.Krause,A.Marino,and M.Moreton,"Learn++:a classifier independent incremental learning algorithmfor supervised neural networks," Proceedings of the International Joint Conference on Neural Networks,vol.2,pp.1742-1747,2002.
[3]D.P.Helmbold and P.M.Long,"Tracking drifting concepts by minimizing disagreements," Machine Learning,vol.14,no.1,pp.27-45,1994.
[4]A.Kuh,T.Petsche,and R.L.Rivest,"Learning time-varying concepts," Proceedings of the conference on Advances in neural information processing,pp.183-189,1990.
[5]R.Klinkenberg and I.Renz,"Adaptive information filtering:Learning in the presence of concept drifts," Learning for Text Categorization,pp.33-40,1998.
[6]G.Widmer and M.Kubat,"Learning in the presence of concept drift and hidden contexts," Machine Learning,vol.23,no.1,pp.69-101,1996.
[7]R.Klinkenberg and T.Joachims,"Detecting concept drift with support vector machines," Proceedings of the seventeenth International Conference on Machine Learning,pp.11-18,2000.
[8]R.Klinkenberg and S.Ruping,"Concept drift and the importance of examples," Text Mining-Theoretical Aspects and Applications,pp.55-77,2003.
[9]R.Klinkenberg,"Learning drifting concepts:Example selection vs.example weighting," Intelligent Data Analysis,vol.8,no.3,pp.281-300,2004.
[10]D.W.Aha,D.Kibler,and M.K.Albert,"Instance-based learning algorithms," Machine Learning,vol.6,no.1,pp.37-66,1991.
[11]M.A.Maloof and R.S.Michalski,"A method for partial-memory incremental learning and its application to computer intrusion detection," Proceedings of the seventh IEEE International Conference on Tools with Artificial Intelligence,pp.392-397,1995.
[12]M.A.Maloof and R.S.Michalski,"Selecting examples for partial memory learning," Machine Learning,vol.41,no.1,pp.27-52,2000.
[13]M.A.Maloof and R.S.Michalski,"Incremental learning with partial instance memory," Artificial Intelligence,vol.154,no.1-2,pp.95-126,2004.
[14]T.Cover and P.Hart,"Nearest neighbor pattern classification," IEEE Transactions on Information Theory,vol.13,no.1,pp.21-27,1967.
[15]J.R.Bunch and C.P.Nielsen,"Updating the singular value decomposition," Numerische Mathematik,vol.31,no.2,pp.111-129,1978.
[16]J.R.Bunch,C.P.Nielsen,and D.C.Sorensen,"Rank-one modification of the symmetric eigenproblem,"Numerische Mathematik,vol.31,no.1,pp.31-48,1978.
[17]R.D.Degroat and R.A.Roberts,"Efficient,numerically stabilized rank-one eigenstructure updating,"IEEE Transactions on Signal Processing,vol.38,no.2,pp.301-316,1990.
[18]H.Murakami and B.Kumar,"Efficient calculation of primary images from a set of images," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.4,pp.511-515,1982.
[19]S.Chaudhuri,S.Sharma,and S.Chatterjee,"Recursive estimation of motion parameters," Computer Vision andlmage Understanding,vol.64,no.3,pp.434-442,1996.
[20]S.Chandrasekaran,B.S.Manjunath,Y.F.Wang,J.Winkeler,and H.Zhang,"An eigenspace update algorithm for image analysis," Graphical Models and Image Processing,vol.59,no.5,pp.321-332,1997.
[21]P.Hall,D.Marshall,and R.Martin,"Incremental eigenanalysis for classification," British Machine Vision Conference,pp.286-295,1998.
[22]D.Skocaj and A.Leonardis,"Weighted and robust incremental method for subspace learning," Proceedings of the ninth IEEE International Conference on Computer Vision,pp.1494-1501,2003.
[23]M.Artac,M.Jogan,and A.Leonardis,"Incremental PCA for on-line visual learning and recognition,"Proceedings of the sixteenth International Conference on Pattern Recognition,vol.3,pp.781-784,2002.
[24]Y.Li,"On incremental and robust subspace learning," Pattern Recognition,vol.37,no.7,pp.1509-1518,2004.
[25]W.Lam,C.K.Keung,and D.Liu,"Discovering useful concept prototypes for classification based on filtering and abstraction," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.24,no.8,pp.1075-1090,2002.
[26]R.M.Cameron-Jones,"Instance selection by encoding length heuristic with random mutation hill climbing," Proceedings of the eighth Australian Joint Conference on Artificial Intelligence,pp.99-106,1995.
[27]G.Gates,"The reduced nearest neighbor rule," IEEE Transactions on Information Theory,vol.18,no.3,pp.431-433,1972.
[28]D.B.Skalak,"Prototype and feature selection by sampling and random mutation hill climbing algorithms," Proceedings of the eleventh International Conference on Machine Learning,pp.293-301,1994.
[29]D.L.Wilson,"Asymptotic properties of nearest neighbor rules using edited data," IEEE Transactions on Systems,Man,and Cybernetics,vol.2,no.3,pp.408-421,1972.
[30]B.E.Boser,I.M.Guyon,and V.N.Vapnik,"A training algorithm for optimal margin classifiers,"Proceedings of the fifth annual workshop on Computational learning theory,pp.144-152,1992.
[31]C.Cortes and V.Vapnik,"Support-vector networks," Machine Learning,vol.20,no.3,pp.273-297,1995.
[32]D.Mattera,F.Palmieri,and S.Haykin,"An explicit algorithm for training support vector machines,"Signal Processing Letters,IEEE,vol.6,no.9,pp.243-245,1999.
[33]萧嵘,王继成,孙正兴,张福炎,“一种SVM增量学习算法,”南京大学学报,vol.38,no.2,pp.152-157,2002.
[34]孔锐,张冰,“一种快速支持向量机增量学习算法,”控制与决策,vol.20,no.10,pp.1129-1136,2005.
[35]安金龙,王正欧,“一种适合于增量学习的支持向量机的快速循环算法,”计算机应用,vol.23,no.10,pp.12-14,2003.
[36]滕月阳,唐焕文,张海霞,“一种新的支持向量机增量学习算法,”计算机工程与应用,vol.40,no.36,pp.77-80,2004.
[37]N.Syed,H.Liu,and K.K.Sung,"Incremental learning with support vector machines," Proceedings of the International Joint Conference on Artificial Intelligence,1999.
[38]P.Mitra,C.A.Murthy,and S.K.Pal,"Data condensation in large databases by incremental learning with support vector machines," Proceedings of the International Conference on Pattern Recognition,vol.2,pp.708-711,2000.
[39]曾嵘,刘建成,蒋新华,“一种基于支持向量机的增量学习算法,”铁道科学与工程学报,vol.2,no.1,pp.94-96,2005.
[40]曾文华,马健,“一种新的支持向量机增量学习算法,”厦门大学学报:自然科学版,vol.41,no.6,pp.687-691,2002.
[41]李凯,黄厚宽,“支持向量机增量学习算法研究,”北方交通大学学报,vol.27,no.5,pp.34-37,2003.
[42]C.Diehl and G.Cauwenberghs,"SVM incremental learning,adaptation and optimization," Proceedings of the International Joint Conference on Neural Networks,pp.2685-2690,2003.
[43]N.A.Syed,H.Liu,and K.K.Sung,"Handling concept drifts in incremental learning with support vector machines," Proceedings of the fifth international conference on Knowledge discovery and data mining,pp.317-321,1999.
[44]R.Polikar,L.Upda,S.S.Upda,and V.Honavar,"Learn++:an incremental learning algorithm for supervised neuralnetworks," IEEE Transactions on Systems,Man and Cybernetics,Part C,vol.31,no.4,pp.497-508,2001.
[45]M.Muhlbaier,A.Topalis,and R.Polikar,"Incremental learning from unbalanced data," Proceedings of the IEEE International Joint Conference on Neural Networks,vol.2,pp.1057-1062,2004.
[46]M.Muhlbaier,A.Topalis,and R.Polikar,"Learn++.MT:a new approach to incremental learning,"Proceedings of the fifth International Workshop on Multiple Classifier Systems,pp.52-61,2004.
[47]Z.Erdem,R.Polikar,F.Gurgen,and N.Yumusak,"Ensemble of SVMs for incremental learning,"Proceedings of the sixth International Workshop on Multiple Classifier Systems,pp.246-256,2005.
[48]A.Gangardiwala and R.Polikar,"Dynamically weighted majority voting for incremental learning and comparison of three boosting based approaches," Proceedings of the IEEE International Joint Conference on Neural Networks,vol.2,pp.1131-1136,2005.
[49]H.S.Mohammed,J.Leander,M.Marbach,and R.Polikar,"Can AdaBoost.M1 learn incrementally?A comparison to Learn++ under different combination rules," Lecture Notes in Computer Science,vol.4131,pp.254-263,2006.
[50]R.Polikar,L.Udpa,S.Udpa,and V.Honavar,"An incremental learning algorithm with confidence estimation for automated identification of NDE signals," IEEE Transactions on Ultrasonics,Ferroelectrics and Frequency Control,vol.51,no.8,pp.990-1001,2004.
[51]W.Buntine,"Theory refinement on Bayesian networks," Proceedings of the seventh Conference on Uncertainty in Artificial Intelligence,pp.52-60,1991.
[52]N.Friedman and M.Goldszmidt,"Sequential update of Bayesian network structure," Proceedings of the thirteenth Conference on Uncertainty in Artificial Intelligence,1997.
[53]J.Route,"An incremental algorithm for tree-shaped Bayesian network learning," Proceedings of the European Conference of Artificial Intelligence,2002.
[54]田凤占,黄丽,于剑,黄厚宽,“包含隐变量的贝叶斯网络增量学习方法,”电子学报,vol.33,no.11,pp.1925-1928,2005.
[55]王飞,刘大有,下淞昕,“基于遗传算法的Bayesian网结构增量学习的研究,”计算机研究与发展,vol.42,no.9,pp.1461-1466,2005.
[56]桑农,曾坤,“基于RBF网络的增量学习,”华中科技大学学报,vol.32,pp.214-216,2004.
[57]K.Okamato,S.Ozawa,and S.Abe,"A fast incremental learning algorithm of RBF networks with long-term memory," Proceedings of the International Joint Conference on Neural Networks,pp.102-107,2003.
[58]骆斌,周志华,“一个增量式判定树学习算法INDUCE,”计算机研究与发展,vol.36,no.5,pp.518-522,1999.
[59]H.Nakayama and A.Hattori,"Incremental learning and forgetting in RBF networks and SVMs with applications to financial problems," vol.2773,pp.1109-1115,2003.
[60]G.Tontini and A.A.De Queiroz,"RBF fuzzy-artmap:a new fuzzy neural network for robust on-linelearning and identification of patterns," IEEE International Conference on Systems,Man,and Cybernetics,vol.2,pp.1364-1369,1996.
[61]C.P.Lim and R.F.Harrison,"Probabilistic Fuzzy ARTMAP:an autonomous neural networkarchitecture for Bayesian probability estimation," Proceedings of the fourth International Conference on Artificial Neural Networks,pp.148-153,1995.
[62]B.L.Lu and M.Ito,"Task decomposition based on class relations:a modular neural network architecture for pattern classification," Lecture Notes in Computer Science,vol.1240,pp.330-339,1997.
[63]B.L.Lu and M.Ito,"Task decomposition and module combination based on class relations:a modular neural network for pattern classification," IEEE Transactions on Neural Networks,vol.10,no.5,pp.1244-1256,1999.
[64]R.E.Jenkins and B.P.Yuhas,"A simplified neural network solution through problem decomposition:the case of the truck backer-upper," IEEE Transactions on Neural Networks,vol.4,no.4,pp.718-720,1993.
[65]C.H.Chen and G.H.You,"Class-sensitive neural network," Neural Parallel Scientific Computing,vol.1,no.1,pp.93-96,1993.
[66]L.Xu,A.Krzyzak,and C.Y.Suen,"Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Transactions on Systems Man and Cybernetics,vol.22,no.3,pp.418-435,1992.
[67]K.Chen,L.Wang,and H.Chi,"Methods of combining multiple classifiers with different features and their applications to text-independent speaker identification," International Journal of Pattern Recognition and Artificial Intelligence,vol.11,no.3,pp.417-445,1997.
[68]J.A.Robert,J.I.Michael,and G.A.Barto,"Task decomposition through competition in a modular connectionist architecture:the what and where vision tasks," Cognitive Science,vol.15,pp.219-250,1991.
[69]R.E.Schapire,"The strength of weak learnability," Machine Learning,vol.5,no.2,pp.197-227,1990.
[70]K.Turner and J.Ghosh,"Analysis of decision boundaries in linearly combined neural classifiers,"Pattern Recognition,vol.29,no.2,pp.341-348,1996.
[71]R.Battiti and A.M.Colla,"Democracy in neural nets:voting schemes for classification," Neural Networks,vol.7,no.4,pp.691-707,1994.
[72]K.Tumer and J.Ghosh,"Order statistics combiners for neural classifiers," Proceedings of the World Congress on Neural Networks,pp.31-34,1995.
[73]W.B.Langdon and B.F.Buxton,"Genetic programming for combining classifiers," Proceedings of the Genetic and Evolutionary Computation Conference,pp.66-73,2001.
[74]B.L.Lu,K.A.Wang,M.Utiyama,and H.Isahara,"A part-versus-pat method for massively parallel training of support vector machines," Proceedings of the IEEE International Joint Conference on Neural Networks,pp.735-740,2004.
[75]K.A.Wang,H.Zhao,and B.L.Lu,"Task decomposition using geometric relation for min-max modular SVMs," Lecture Notes in Computer Science,vol.3496,pp.887-892,2005.
[76]H.C.Lian,B.L.Lu,E.Takikawa,and S.Hosoi,"Gender recognition using a min-max modular support vector machine," Lecture Notes in Computer Science,vol.3611,pp.438-441,2005.
[77]B.L.Lu,Q.Ma,M.Ichikawa,and H.Isahara,"Efficient part-of-speech tagging with a min-max modular neural-network model," Applied Intelligence,vol.19,no.1,pp.65-81,2003.
[78]B.L.Lu,Q.Ma,,and M.Ichikawa,"Massively parallel classification of single-trial LEG signals using a min-max modular neural network," IEEE Transactions on Biomedical Engineering,vol.51,no.3,pp.551-558,2004.
[79]Y.Yang and B.L.Lu,"Prediction of protein subcellular multi-locations with a min-max modular support vector machine," Lecture Notes in Computer Science,vol.3973,pp.667-673,2006.
[80]Z.G.Fan and B.L.Lu,"Multi-view face recognition with min-max modular SVMs," Lecture Notes in Computer Science,vol.3611,pp.396-399,2005.
[81]F.Y.Liu,K.Wu,H.Zhao,and B.L.Lu,"Fast text categorization with min-max modular support ector machines," IJCNN'05.Proceedings of the IEEE International Joint Conference on Neural Networks,vol.1,pp.570-575,2005.
[82]X.L.Chu,C.Ma,L.J.,B.L.Lu,M.Utiyama,and H.Isahara,"Large-scale patent classification with min-max modular support vector machines," International Joint Conference on Neural Networks,vol.1,pp.3972-3979,2008.
[83]B.L.Lu and M.Ichikawa,"A Gaussian zero-crossing discriminant function for min-max modular neural networks," Proceedings of the Fifth international conference on Knowledge-Based Intelligence Information Engineering Systems and Allied Technologies,pp.298-302,2001.
[84]B.L.Lu and M.Ichikawa,"Emergent online learning with a Gaussian zero-crossing discriminantfunction,"Proceedings of the International Joint Conference on Neural Networks,vol.2,pp.1263-1268,2002.
[85]C.Liu and H.Wechsler,"Independent component analysis of Gabor features for face recognition,"IEEE Transactions on Neural Networks,vol.14,no.4,pp.919-928,2003.
[86]C.Liu,"Gabor-based kernel PCA with fractional power polynomial models for face recognition,"IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.26,no.5,pp.572-581,2004.
[87]T.Ojala,M.Pietikainen,and T.Maenpaa,"Mnltiresolntion gray-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.24,no.7,pp.971-987,2002.
[88]I.H.Witten,T.C.Bell,and A.Moffat,Managing gigabytes:compressing and indexing documents and images.Morgan Kanfmann,1999.
[89]C.C.Aggarwal,A.Hinneburg,and D.Keim,On the surprising behavior of distance metrics in high dimensional space.Springer,2000.
[90]P.Simard,Y.LeCnn,and J.S.Denker,"Efficient pattern recognition using a new transformation distance," Advances in Neural Information Processing Systems,pp.50-58,1992.
[91]D.P.Huttenlocher,G.A.Klanderman,and W.J.Rncklidge,"Comparing images using the Hansdorff distance," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.15,no.9,pp.850-863,1993.
[92]E.P.Vivek and N.Sndha,"Robust Hansdofff distance measure for face recognition," Pattern Recognition,vol.40,no.2,pp.431-442,2007.
[93]C.H.T.Yang,S.H.Lai,and L.W.Chang,"Hybrid image matching combining Hansdofff distance with normalized gradient matching," Pattern Recognition,vol.40,no.4,pp.1173-1181,2007.
[94]S.Santini and R.Jain,"Similarity measures," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.21,no.9,pp.871-883,1999.
[95]D.W.Jacobs,D.Weinshall,and Y.Gdalyahn,"Classification with nonmetric distances:image retrieval and classrepresentation," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.22,no.6,pp.583-600,2000.
[96]X.Tan,S.Chen,J.Li,and Z.H.Zhon,"Learning non-metric partial similarity based on maximal margin criterion," Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,pp.168-145,2006.
[97]J.B.Tenenbanm,V.Silva,and J.C.Langford,"A global geometric framework for nonlinear dimensionality reduction," Science,vol.290,pp.2319-2323,2000.
[98]S.T.Roweis and L.K.Saul,"Nonlinear dimensionality reduction by locally linear embedding,"Science,vol.290,pp.2323-2326,2000.
[99]L.Wang,Y.Zhang,and J.Feng,"On the Euclidean distance of images," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.27,no.8,pp.1334-1339,2005.
[100]J.Chen,R.Wang,S.Shan,X.Chen,and W.Gao,“Isomap based on the image Euclidean distance,”Proceedings of the eighteenth International Conference on Pattern Recognition,vol.2,pp.1110-1113,2006.
[101]K.Han and X.C.Zhu,“Research on face recognition based on IMED and 2DPCA,”Journal of Electronics,vol.23,no.5,pp.786-790,2006.
[102]X.Mei,S.K.Zhou,and H.Wu,“Integrated detection,tracking and recognition for IR video-based vehicle classification,”IEEE International Conference on Acoustics,Speech and Signal Processing,vol.5,pp.745-748,2006.
[103]T.Tangkuampien and D.Suter,“3D object pose inference via kernel principal component analysis with image Euclidian distance(IMED),”Proceedings of the British Machine Vision Association,pp.137-146,2006.
[104]R.Wang,J.Chen,S.Shan,and W Gao,"Enhancing training set for face detection," Proceedings of the eighteenth International Conference on Pattern Recognition,vol.3,pp.477-80,2006.
[105]K.M.Lee and W Nick Street,“Incremental feature weight learning and its application to a shape-based query system,”Pattern Recognition Letters,vol.23,no.7,pp.865-874,2002.
[106]J.Weng,C.H.Evans,and W S.Hwang,“An incremental learning method for face recognition undercontinuous video stream,”Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition,pp.251-256,2000.
[107]R.Sukthankar and R.Stockton,“Argus:the digital doorman,”Intelligent Systems,vol.16,no.2,pp.14-19,2001.
[108]O.Deniz,M.Castrillon,J.Lorenzo,and M.Hernandez,“An incremental learning algorithm for face recognition,”Lecture Notes in Computer Science,vol.2359,pp.1-9,2002.
[109]H.Huifang,“The drifting and positioning of the concept of moral values among contemporary women college students,”Chinese Education And Society,vol.33,no.3,pp.23-28,2000.
[110]A.Pentland,B.Moghaddam,and T.Starner,“View-based and modular eigenspaces for face recognition,”IEEE Computer Society Conference on Computer Vision and Pattern Recognition,pp.84-91,1994.
[111]M.A.Turk and A.P.Pentland,“Face recognition using eigenfaces,”IEEE Computer Society Conference on Computer Vision and Pattern Recognition,pp.586-591,1991.
[112]W Zhu,J.Pei,J.Yin,and Y.Xie,“Granularity adaptive density estimation and on demand clustering of concept-drifting data streams,”Proceedings of the eighth International Conference on Data Warehousing and Knowledge Discovery,vol.4081,pp.322-331,2006.
[113]P.N.Belhumeur,J.P.Hespanha,and D.J.Kriegman,"Eigenfaces vs.Fisherfaces:recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.19,no.7,pp.711-720,1997.
[114]B.Moghaddam and A.Pentland,"Probabilistic visual learning for object representation," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.19,no.7,pp.696-710,1997.
[115]B.Moghaddam,T.Jebara,and A.Pentland,"Bayesian face recognition," Pattern Recognition,vol.33,no.11,pp.1771-1782,2000.
[116]W.Zhao,R.Chellappa,and A.Krishnaswamy,"Discriminant analysis of principal components for face recognition," Face Recognition:From Theory to Applications,pp.73-85,1998.
[117]P.J.Phillips,H.Moon,S.A.Rizvi,and P.J.Rauss,"The FERET evaluation methodology for facerecognition algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.22,no.10,pp.1090-1104,2000.
[118]I.Dagher and R.Nachar,"Face Recognition Using IPCA-ICA Algorithm," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.28,no.6,pp.996-1000,2006.
[119]H.Zhao,P.C.Yuen,J.T.Kwok,and J.Yang,"Incremental PCA based face recognition," Proceedings of the eighth International Conference on Control,Automation,Robotics and Vision,pp.687-691,2004.
[120]S.Ozawa,S.L.Toh,S.Abe,S.Pang,and N.Kasabov,"Incremental learning of feature space and classifier for face recognition," Neural Networks,vol.18,no.5-6,pp.575-584,2005.
[121]S.Ozawa,S.Toh,S.Abe,S.Pang,and N.Kasabov,"Incremental Learning for Online Face Recognition,"Neural Network,vol.18,no.5-6,pp.575-584,2005.
[122]C.L.Blake and C.J.Merz,"UCI repository of machine learning databases," 1998.
[123]B.Huang and B.L.Lu,"Fault diagnosis for industrial images using a min-max modular neural network," Lecture Notes in Computer Science,vol.3316,pp.842-847,2004.
[124]H.C.Lian and B.L.Lu,"An algorithm for pruning redundant modules in min-max modular network,"Proceedings of the 14th National Conference on Neural Network,pp.37-42,2004.
[125]H.C.Lian and B.L.Lu,"An algorithm for pruning redundant modules in min-max modular network,"Proceedings of the IEEE International Joint Conference on Neural Networks,vol.3.
[126]P.Hart,"The condensed nearest neighbor rule(Corresp.)," IEEE Transactions on Information Theory,vol.14,no.3,pp.515-516,1968.
[127]N.Grira,M.Crucianu,and N.Boujemaa,"Unsupervised and semi-supervised clustering:a brief survey," A Review of Machine Learning Techniques for Processing Multimedia Content,Report of the MUSCLE European Network of Excellence,2004.
[128]边肇祺,模式识别.清华大学出版社,2000.
[129]T.Kohonen,Self-organizing maps.Springer,2001.
[130]S.Basu,A.Banerjee,and R.J.Mooney,"Semi-supervised clustering by seeding," Proceedings of ninteenth International Conference on Machine Learning,pp.19-26,2002.
[131]T.Kohonen,"Improved versions of learning vector quantization," Proceedings of International Joint Conference on Neural Networks,pp.545-550,1990.
[132]N.Bansal,A.Blum,and S.Chawla,"Correlation clustering," Machine Learning,vol.56,no.1,pp.89-113,2004.
[133]C.M.Procopiuc,"Clustering problems and their applications(a survey)," Department of Computer Science,Duke University,1997.
[134]X.Li and N.Ye,"Grid and dummy cluster based learning of normal and intrusive clusters for computer intrusion detection," Quality and Reliability Engineering International,vol.18,no.3,pp.231-242,2002.
[135]X.Li and N.Ye,"A supervised clustering algorithm for computer intrusion detection," Knowledge and Information Systems,vol.8,no.4,pp.498-509,2005.
[136]R.F.Sproull,"Refinements to nearest-neighbor searching in k-dimensional trees," Algorithmica,vol.6,no.1,pp.579-589,1991.
[137]S.G.Mallat,"Multifrequency channel decompositions of images and wavelet models," IEEE Transactions on Acoustics,Speech,and Signal Processing,vol.37,no.12,pp.2091-2110,1989.
[138]A.Cohen and J.Kovacevic,"Wavelets:the mathematical background," Proceedings of the IEEE,vol.84,no.4,pp.514-522,1996.
[139]S.Mallat and S.Zhong,"Characterization of signals from multiscale edges," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.14,no.7,pp.710-732,1992.
[140]J.M.Lee,Riemannian manifolds:an introduction to curvature.Springer,1997.
[141]J.Jost,Riemannian geometry and geometric analysis.Springer,1995.
[142]H.S.Seung and D.D.Lee,"Cognition:the manifold ways of perception," Science,vol.290,no.5500,pp.2268-2269,2000.
[143]W.Gao,B.Cao,S.Shan,D.Zhou,X.Zhang,and D.Zhao,"The CAS-PEAL large-scale Chinese face database and baseline evaluations," Technical report of JDL,http://www.jdl.ac.cn/peal/peal tr.pdf,2004.
[144]D.B.Graham and N.M.Allinson,"Characterizing virtual eigensignatures for general purpose face recognition," Face Recognition:From Theory to Applications.NATO ASI Series F,Computer and Systems Sciences,vol.163,pp.446-456,1998.
[145]Y.LeCun,"The MNIST database of handwritten digits," Available online at:http://yann.lecun.com/exdb/mnist/index.html.
[146]B.Moghaddam and M.H.Yang,"Learning gender with support faces," IEEE Transactions on Pattern Analysis and Machine Intelligence,vol.24,no.5,pp.707-711,2002.
[147]H.C.Lian and B.L.Lu,"Multi-view gender classification using multi-resolution local binary patterns and support vector machines," International Journal of Neural Systems,vol.17,no.6,pp.479-487,2007.
[148]J.Li and B.L.Lu,"A framework for multi-view gender classification," Lecture Notes in Computer Science,vol.4984,pp.973-982,2007.
[149]C.C.Chang and C.J.Lin,"LIBSVM:a library for support vector machines,"http://www.csie.ntu.edu.tw/cjlin/libsvm,2001.
[150]S.Gutta,J.R.J.Huang,P.Jonathon,and H.Wechsler,"Mixture of experts for classification of gender,ethnic origin,and pose of human faces," IEEE Transactions on Neural Networks,vol.11,no.4,pp.948-960,2000.
[151]A.Khan,"Combination and optimization of classifiers in gender classification using genetic programming,"International Journal of Knowledge-Based and Intelligent Engineering Systems,vol.9,no.1,pp.1-11,2005.
[152]H.C.Lian and B.L.Lu,"Multi-view gender classification using local binary patterns and support vector machines," Lecture Notes in Computer Science,vol.3972,pp.202-209,2006.
[153]J.Luo and B.L.Lu,"Gender recognition using a min-max modular support vector machine with equal clustering," Lecture Notes in Computer Science,vol.3972,pp.210-215,2006.
[154]H.C.Kim,D.Kim,Z.Ghahramani,and S.Y.Bang,"Appearance-based gender classification with Gaussian processes," Pattern Recognition Letters,vol.27,no.6,pp.618-626,2006.
[155]K.Balci and V.Atalay,"PCA for gender estimation:Which eigenvectors contribute," Proceedings of Sixteenth International Conference on Pattern Recognition,vol.3,pp.363-366,2002.
[156]A.Jain and J.Huang,"Integrating independent components and linear discriminant analysis for gender classification," Sixth IEEE International Conference on Automatic Face and Gesture Recognition,pp.159-163,2004.
[157]A.J.O' Toole,K.A.Deffenbacher,D.Valentin,K.McKee,D.Huff,and H.Abdi,"The perception of face gender:The role of stimulus structure in recognition and classification," Memory and Cognition,vol.26,no.1,pp.146-160,1998.
[158]G.W.Cottrell and J.Metcalfe,"EMPATH:face,emotion,and gender recognition using holons,"Proceedings of the Conference On Advances in Neural Information Processing Systems,pp.564-571,1990.
[159]B.Edelman,D.Valentin,and H.Abdi,“Sex classification of face areas:how well can a linear neural network predict human performance,”Journal of Biological System,vol.6,no.3,pp.241-264,1998.
[160]B.A.Golomb,D.T.Lawrence,and T.J.Sejnowski,“SexNet:A neural network identifies sex from human faces,”Proceedings of the 1990 conference on Advances in neural information processing,pp.572-577,1990.
[161]J.C.Platt,“Fast training of support vector machines using sequential minimal optimization,”Advances in kernel methods:support vector learning,pp.185-208,1999.
[162]J.Huang,X.Shao,and H.Wechsler,“Face pose discrimination using support vector machines(SVM),”Fourteenth International Conference on Pattern Recognition,pp.154-156,1998.