核机器学习方法若干问题研究
|
摘要
论文主要对机器学习问题、核机器方法、小波核机器、模糊小波核机器等内容进行探讨和分析,构建了几种核机器模型,理论分析和实验结果验证了它们的可行性和有效性。主要研究内容如下:
处理大规模样本时,支持向量机难以满足实时性要求,针对这一问题,提出了一种支持向量预提取方法:先用核感知机模块提取准支持向量,然后将所得结果输入到支持向量机中进行二次处理。核感知机采用的是简单的迭代结构,即使在样本规模较大时,花费的时间也很少;此外,准支持向量的数目可以通过设定阈值进行控制。在一定精度要求下,能从很大程度上提高数据的处理效率。由于核函数和误分界的引入,在综合使用支持向量机的基础上,能处理线性可分、非线性可分、非线性不可分带噪声数据以及回归等问题,理论分析和实验结果较好地验证了这一结论。
对非平稳信号进行处理时,信号细微特征的提取非常关键。论文尝试将小波技术、主分量分析及核方法相结合,用于处理这类信号。对采用小波基构建核函数的可行性进行了探讨,证明了它满足Mercy条件及其在Hilbert空间具有再生性的命题,以此为基础,结合主分量分析,探讨了小波核机器的构建方法,构造出一种核机器模型,并作了实例仿真。实验结果表明,复Gaussian小波核和复Morlet小波核的性能大致相当,它们都优于常规的高斯核和多项式核,初步展示出该方法的可行性和优越性。
对模糊逻辑和小波技术的相关理论进行探讨和分析,构建了一种模糊小波容许核函数,并与支持向量机结合,构造出一种核机器模型,对该模型的一致逼近性作了证明。在此基础上,提出了一种模糊小波支持向量核机器方法FW-SVKM,对参数的选择与预测结果的内在关系作了较为详细的分析,与三层神经网络ANN进行短期峰值负荷预测的对比实验,结果表明FW-SVKM优于ANN,具有较大的实用价值和较好的应用前景。
针对学习机器在参数较多时,优化时间过长、效率过低,不利于工程应用的问题,提出了一种多参数同步优化策略。实验结果表明,该方法在实际应用中是行之有效的,能大幅减少多参数模型的优化时间,增强核机器方法的实用性和有效性。
This paper dealed with machine learning, kernel machine method, wavelet kernel machine and fuzzy wavelet kernel machine technology detailedly. Several kinds of kernel machine models were constructed, and have been applied to non-stationary signals processing. Theoretical analysis and implementation results show their validity and feasibility. Some important issues have been discussed in this paper as follows.It is well known that general SVM (Support Vector Machine) costs too much time on large scale data sets. As a valid solution, support vectors pre-extraction method has been discussed in this paper. Kernel perceptron firstly has been used to extract quasi-support vectors. And then, quasi-support vectors were input to standard SVM to process accurately. This method takes advantages of the high speed of perceptron for its simple iterative structure. Perceptron costs fewer time than general SVM, especially on large scale data sets, and can control number of quasi-support vectors easily by a threshold variable, much time will be economized in latter process. Some special technologies, such as kernel function and error boundary, etc, have been adopted to conduct linear separable, nonlinear separable, nonlinear unseparated model recognition and regression questions effectively.Generally, imperceptible features are very important in non-stationary signal processing. Some kinds of complex methods were discussed in this paper, which combined wavelet, Principal Component Analysis (PCA) and kernel function teconology. Wavelet kernel function was constructed after proofs of propositions, that it can meet Mercy condition needs and has reproduction feature in Hilbert space. A kind of kernel machine model was presented and some numerical simulation experiments were applied to validate its correctness. Experiment results show that complex Gaussian wavelet kernel almost has the approximate performance as complex Morlet wavelet kernel, however, excel than general Gaussian kernel and polynomial kernel.Fuzzy and wavelet technology were adopted to construct a kind of fuzzy wavelet kernel function. After that, a kind of kernel machine model based on SVM was built and proofs of consistant approximation were shown immediately. Based upon these theories foundation, Fuzzy Wavelet Support Vector Kernel Machine (FW-SVKR) was formed. The close relationship between parameters and forecast
results was expatiated later. Contrast experiments between FW-SVKR and Artificial Neurial Network (ANN) show that the former was superior to the latter in electric power system load forcasting, and seems to have more applied value in this domain.Theory analysis suggested that much more time should be cost when training a multi-parameters model, which formed the mainly obstacle in application. Aimed at this question, a kind of new technology named multi-parametes synchronous optimization method was proposed. It can save a lot of time when training parameters, and enhance applied value remarkably. Experiment results show its advantages in application.Along with the boost of country economy, city traffic block should be solved urgently. Based on analysis of features of city traffic flow, some kind of kernels such as general kernels, compound kernels and fuzzy wavelet kernels, have been adopted to do realtime traffic flow forecast. Contrast experiment results show the different performance of those kernels, which can help to improve the city traffic control power effectively. Another application of kernel machine method was discussed in this paper, complex kernel function method and support vectors preextraction technology have been adopted to retrieve the Oceanic Chlorophyll-a Concentration in SeaWIFS data sets. Further more, ANN altorithm and twelve kinds of empirical algorithms have been adopted, too. Contrast experiment results show that, retrieve precision with complex kernel function method is higher than that of other algorithms;it seems to be more suitable in this domain.
处理大规模样本时,支持向量机难以满足实时性要求,针对这一问题,提出了一种支持向量预提取方法:先用核感知机模块提取准支持向量,然后将所得结果输入到支持向量机中进行二次处理。核感知机采用的是简单的迭代结构,即使在样本规模较大时,花费的时间也很少;此外,准支持向量的数目可以通过设定阈值进行控制。在一定精度要求下,能从很大程度上提高数据的处理效率。由于核函数和误分界的引入,在综合使用支持向量机的基础上,能处理线性可分、非线性可分、非线性不可分带噪声数据以及回归等问题,理论分析和实验结果较好地验证了这一结论。
对非平稳信号进行处理时,信号细微特征的提取非常关键。论文尝试将小波技术、主分量分析及核方法相结合,用于处理这类信号。对采用小波基构建核函数的可行性进行了探讨,证明了它满足Mercy条件及其在Hilbert空间具有再生性的命题,以此为基础,结合主分量分析,探讨了小波核机器的构建方法,构造出一种核机器模型,并作了实例仿真。实验结果表明,复Gaussian小波核和复Morlet小波核的性能大致相当,它们都优于常规的高斯核和多项式核,初步展示出该方法的可行性和优越性。
对模糊逻辑和小波技术的相关理论进行探讨和分析,构建了一种模糊小波容许核函数,并与支持向量机结合,构造出一种核机器模型,对该模型的一致逼近性作了证明。在此基础上,提出了一种模糊小波支持向量核机器方法FW-SVKM,对参数的选择与预测结果的内在关系作了较为详细的分析,与三层神经网络ANN进行短期峰值负荷预测的对比实验,结果表明FW-SVKM优于ANN,具有较大的实用价值和较好的应用前景。
针对学习机器在参数较多时,优化时间过长、效率过低,不利于工程应用的问题,提出了一种多参数同步优化策略。实验结果表明,该方法在实际应用中是行之有效的,能大幅减少多参数模型的优化时间,增强核机器方法的实用性和有效性。
This paper dealed with machine learning, kernel machine method, wavelet kernel machine and fuzzy wavelet kernel machine technology detailedly. Several kinds of kernel machine models were constructed, and have been applied to non-stationary signals processing. Theoretical analysis and implementation results show their validity and feasibility. Some important issues have been discussed in this paper as follows.It is well known that general SVM (Support Vector Machine) costs too much time on large scale data sets. As a valid solution, support vectors pre-extraction method has been discussed in this paper. Kernel perceptron firstly has been used to extract quasi-support vectors. And then, quasi-support vectors were input to standard SVM to process accurately. This method takes advantages of the high speed of perceptron for its simple iterative structure. Perceptron costs fewer time than general SVM, especially on large scale data sets, and can control number of quasi-support vectors easily by a threshold variable, much time will be economized in latter process. Some special technologies, such as kernel function and error boundary, etc, have been adopted to conduct linear separable, nonlinear separable, nonlinear unseparated model recognition and regression questions effectively.Generally, imperceptible features are very important in non-stationary signal processing. Some kinds of complex methods were discussed in this paper, which combined wavelet, Principal Component Analysis (PCA) and kernel function teconology. Wavelet kernel function was constructed after proofs of propositions, that it can meet Mercy condition needs and has reproduction feature in Hilbert space. A kind of kernel machine model was presented and some numerical simulation experiments were applied to validate its correctness. Experiment results show that complex Gaussian wavelet kernel almost has the approximate performance as complex Morlet wavelet kernel, however, excel than general Gaussian kernel and polynomial kernel.Fuzzy and wavelet technology were adopted to construct a kind of fuzzy wavelet kernel function. After that, a kind of kernel machine model based on SVM was built and proofs of consistant approximation were shown immediately. Based upon these theories foundation, Fuzzy Wavelet Support Vector Kernel Machine (FW-SVKR) was formed. The close relationship between parameters and forecast
results was expatiated later. Contrast experiments between FW-SVKR and Artificial Neurial Network (ANN) show that the former was superior to the latter in electric power system load forcasting, and seems to have more applied value in this domain.Theory analysis suggested that much more time should be cost when training a multi-parameters model, which formed the mainly obstacle in application. Aimed at this question, a kind of new technology named multi-parametes synchronous optimization method was proposed. It can save a lot of time when training parameters, and enhance applied value remarkably. Experiment results show its advantages in application.Along with the boost of country economy, city traffic block should be solved urgently. Based on analysis of features of city traffic flow, some kind of kernels such as general kernels, compound kernels and fuzzy wavelet kernels, have been adopted to do realtime traffic flow forecast. Contrast experiment results show the different performance of those kernels, which can help to improve the city traffic control power effectively. Another application of kernel machine method was discussed in this paper, complex kernel function method and support vectors preextraction technology have been adopted to retrieve the Oceanic Chlorophyll-a Concentration in SeaWIFS data sets. Further more, ANN altorithm and twelve kinds of empirical algorithms have been adopted, too. Contrast experiment results show that, retrieve precision with complex kernel function method is higher than that of other algorithms;it seems to be more suitable in this domain.
引文
[1] Fisher R A. Contributions to Mathematical Statistics[M]. J. Wiley, New York, 1952
[2] Rosenblatt F. Principles of Neurodinamics: Perceptron and Theory of Brain Mechanisms[M]. Spartan Books, Washington D. C. , 1962
[3] Novikoff A B J. On Convergence Proofs on Perceptrons[A]. Proceedings of the Symposium on the Mathematical Theory of Automata[C] , Poltechnic Institute of Brooklyn, 1962, Ⅻ: 615～622
[4] Vapnik V N, Chervonenkis A J. On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities[J]. Doklady Akademii Nauk USSR, 1968, 181(4): 173～187
[5] Ivanov V V. On Linear Problems Which are not Well-posed[J]. Soviet Math Dokl. , 1962, 3(4): 981～983
[6] Philips D Z. A Technique for Numerical Solution of Certain Integral Equation of the First Kind[J]. Journal of Association with Computer Machine, 1962, 9:84～96
[7] Tikhonov A N. On Solving ill-posed problem and method of regularization[J]. Doklady Akademii Nauk USSR, 1963, 153:501～504
[8] Vapnik V N, Stefanyuk A R. Nonparametric Methods for Estimation Probability Densities[J]. Automation and Remote Control, 1978, 8:27～35
[9] Solomonoff R J. A Preliminary Report on General Theory of Inductive Inference[R]. Technical Report ZTB-138, Zator Company, Cambridge, 1960
[10] Kolmogoroff A N. Three Approaches to the Quantitative Definitions of Information[J]. Problem of Information Transmission, 1965, 1(1): 1～7
[11] Chaitin G J. On the Length of Programs for Computing Finite Binary Sequences[J]. Journal of Association with Computer Machine, 1966, 13:547～569
[12] Rissanen J. Modeling by Shortest Data Description[J]. Automatica, 1978, 14: 465～471
[13] LeCun Y. Learning Processes in an Asymmetric Threshold Network[J]. Disordered Systems and Biological Organizations, Les. Houches, France, Springer, 1986, 9: 233～240
[14] Rumelhart D E, Hinton G E, Williams R J. Learning Internal Representations by Error Propagation[M]. Parallel Distributed Processing: Explorations in Macrostructure of coganition, 1986, Ⅰ, Badford Books, Cambridge, MA. :318～362
[15] Vapnik V N. Three Fundamental Concepts of the Capacity of Learning Machines[J]. Physica A, 1993, 200:538～544
[16] Vapnik V N, Bottou L. Local Algorithms for Pattern Recognition and Dependencies Estimation[J]. Neural Computation, 1993, 5(6): 893～908
[17] Vapnik V N. Statistical Learning Theory[M]. J. Wiley, New York, 1998
[18] Mercy J. Functions of Positive and negative type and their connection with the theory of integral equations[J]. Philosophical Transactions of the Royal Society, London, 1909, A, 209:415～446
[19] Boser B E, Guyon I M, Vapnik V N. A Training Algorithm for Optimal Margin Classifiers[J]. In D. Haussler Editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, ACM Press, 1992:144～152
[20] Scholkoph B A, Smola A J, Muller K R. Kernel Principal Component Analysis, Advances in Kernel Methods-Support Vector Learning[M]. MIT Press, Cambridge, MA, 1999:327～352
[21] Scholkoph B A, Smola A J, Bartlett P L, New Support Vector Algorithms[J]. Neural Computation, 2000, 12:1207～1245
[22] Suykens J A, Branbanter J K, Lukas L, et al. , Weighted Least Squares Support Vector Machine: Robustness and Spare Approximation[J]. Neurocomputing, 2002, 48(1): 85～105
[23] Lin C F, Wang S D. Fuzzy Support Vector Machines[J]. IEEE Transaction on Neural Networks, 2002, 48(1): 85～105
[24] Keoman V, Hadzic I, Support Vectors Selection by Linear Programming[A]. Proceedings of the IEEE- INNS-ENNS International Joint Conference on Networks[C] , Como. Italy, 2000, 5:193～198
[25] Laskov P, Feasible Direction Decomposition Algorithms for Training Support Vector Machine[J] , Machine Learning, 2002, 46(1): 315～349
[26] Smola A J, Scholkoph B A, A Tutorial on Support Vector Regression[R]. Royal Holloway College, NeuroCOLT Technical Report TR- 1998-030, 1998
[27] Carozza M, Rampone S. Towards an Incremental SVM for Regression[A]. Proceedings of the IEEE-ENNS Int Joint Conference on Neural Networks, 2000, 6: 405～410
[28] Gestel T V. Financial Time Series Prediction Using Least Squares Support Vector Machines within the Evidence Framework[J]. IEEE Transaction on Neural Networks, 2001, 12(4): 809～821
[29] Gretton A. Support Vector Regression for Black-box System Identification[A]. Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing[C] , Singapore, 2001, 341～344
[30] Chan W C, Chan C W, Cheung K C, et al. On the Modeling of Nonlinear Dynamic Systems using Support Vector Neural Networks[J]. Engineering Application of Artificial Intelligence, 2001, 14(2): 105～113
[31] Kruif B J, Vries T J. On Using a Support Vector Machine in Learning feed-forward control[A]. Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics[C] , Coma, Italy, 2001, 1:272～277
[32] 陈念贻,陆文聪,武海顺等.支持向量机算法在氮化铝薄生长过程控制中的应用[J].计算机与应用化学,2002,19(6):726～728
[33] 潘继斌.回归函数的支持向量机估计法[J].湖北师范学院学报,2003(4),23:9～13
[34] 李良敏,屈梁生.基于遗传编程和支持向量机的故障诊断模型[J].西安交通大学学报,2004(3),38:239～242
[35] 刘洪,赵金洲,胡永全等,支持向量机在重复压裂中的应用研究[J],天然气工业, 2004(3), 24:75-77
[36] Gabor D. Theory of Communication [J]. Journal of IEE, 1946, 93: 429-457
[37] Ville J. Theorie et Applications de la Notion de Signal Analytique [J]. Cables et Transmission, 1948,2A: 61-74
[38] Page C H. Instantaneous Power Spectra [J]. Journal of Physical Application, 1952,23:103-106
[39] Namias V. The Fractional Fourier Transform and Its Application in Quantum Mechanics [J]. Journal of Institute of Math, and Its Application, 1980,25: 241-265
[40] Mann S, Haykin S. "Chirplets and Wavelets": Novel Time-frequency Methods [J]. Electronic Letters, 1992, 28: 114-116
[41] Mann S, Haykin S. Adaptive "Chirplet" Transform: an Adaptive Generalization of the Wavelet Transform [J]. Optical England, 1992, 31: 1243-1256
[42] Mann S, Haykin S. The Chirplet Transform: Physical Considerations [J]. IEEE Transaction on Signal Processing, 1995,43: 2745-2761
[43] Mihovilovic D, Bracewell R N. Adative Chirplet Representation of Signals on Time-frequency Plane [J]. Electronics Letters, 1991, 27: 1159-1161
[44] Mihovilovic D, Bracewell R N. Whistler Analysis in the Time-frequency Plane Using Chirplets [J]. Journal of Geophysical Research, 1992, 97: 17,199-204
[45] Grosses A., Morlet J. Decomposition of Hardy Function into Square Wavelets of Constant Shapes [J]. SIAM J Math. Anal, 1984(15):723~726
[46] Meyer Y. Principle D'incertitude bases Hilbertiennes et Algebra D'Operataur [J]. Bourbaki Seminaire, Asterisque (Societe Mathematique de France), 1985, 2: 662-690
[47] Meyer Y. Ondelettes, Functions Splines et Analyses Graduees [R]. Italy: Lectures Given at the University of Torino, 1986
[48] Mallat S. A Theory for Multiresolution Signal Decomposition: The Wavelet Represention. IEEE Trans. Pattern Analysis and Machine Interlligence, 1989,11(7):674~693
[49] Mallat S. Multifrequency Channel Decomposition of Images and Wavelet Model. IEEE Trans. Acoust. Speech Signal Processing, 1989, 37:2091-2110
[50] Daubechies I. Orthonormal Bases of Compactly Supported Wavelets. Commun. Pure Appl, 1988,41:990-996
[51 ] Vapnik V N, Chervonenkis A J. On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities [J]. Theory Probab. Apl., 1971, 16: 264-280
[52] Vapnik V N. Estimation of Dependencies Based on Empirical Data [M]. Nauka,Moscow, 1979 (English Edition: Vapnik V N. Estimation of Depen- dencies Based on Empirical Data [M], Springer, New York, 1982)
[53] Robbins H, Monroe H. A Stochastic Approximation Method [J]. Annalsof Mathematical Statistics, 1951, 22: 400-407
[54] Aizerman M A, Braveman E M, Rozonoer L I. Theoretical Foundation of Potential Function Method in Pattern Recognition Learning [J]. Automation and Remote Control, 1964,25:821-837
[55] Aizerman M A, Braveman E M, Rozonoer L I. The Robbince-Monroe Process and the Method of Potential Functions [J]. Automation and Reote Control, 1965, 1882-1885
[56] Amari S. A Theory of Adaptive Pattern Classifiers[J]. IEEE Trans. Elect. Comp. , 1967, EC_16:299～307
[57] Tsypkin Y Z. Adaptation and Learning in Automatic Systems[M]. Academic, New York, 1971
[58] Tsypkin Y Z. Foundation of the Theory of Learning Systems[M]. Academic, New York, 1973
[59] Vapnik V N, Che rvonenkis A J. Theory of Pattern Recognition[M]. Nauka, Moscow, 1974
[60] Cristianini N, Taylor J S. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods[M]. England: Cambridge University Press, 2000. 4
[61] Huber P. Robust Estimation of Location Parameter[J]. Ann. Math. Stat. , 1964, 35(1)
[62] Vapnik V N. The Nature of Statistical Learning Theory[M]. 2nd Edition, New York: Springer verlag, 1999:147～156
[63] 许建华,张学工,李衍达.一种基于核函数的非线性感知机算法[J].计算机学报,2002,7(25):689～695
[64] 张亭禄,贺明霞.基于人工神经网络的一类水域叶绿素a浓度反演方法[J].遥感学报,2001,6(1):40～44
[65] 黄海清,何贤强,王迪峰,等.神经网络法反演海水叶绿素浓度的分析[J].地球信息科学,2004,6(2):31～37
[66] 冯春晶,赵朝方.基于人工神经网络研究海水中叶绿素浓度的垂直分布[J].中国海洋大学学报,2004,34(3):497～505
[67] 裴洪平,罗妮娜,蒋勇.利用BP神经网络方法预测西湖叶绿素a的浓度[J].生态学报,2004,24(2):246～251
[68] Slade W H, Richard J, Miller L, et al. Ensemble Neural Network Methods for Satellite-Derived Estimation of Chlorophyll a[J]. IEEE Transaction on Geoscience and Remote Sensing, 2003:547～552
[69] Davide D' Alimonte, Giuseppe Zibordi. Phytoplankton Determination in an Optically Complex Coastal Region Using a Multilayer Perception Neural Network[J]. IEEE Transaction on Geoscience and Remote Sensing, 2003, 41 (12): 1861～1868
[70] 边肇祺,张学工.模式识别(第二版)[M].北京:清华大学出版社,2000.1
[71] 杜平,张燕昆,刘重庆.基于不变矩的人脸识别方法的研究[J].计算机仿真,2002,19(3):78～81
[72] 祝海龙,屈梁生,张海军.基于小波变换和支持向量机的人脸检测系统[J].西安交通大学学报,2002,36(9):947～950
[73] Caldenon A P. Intermediate Spaces and Interpolation, the Complex Method[J]. Study on Math. , 1964, 24:113～190
[74] Aslaksen E W, Klauder J R. Unitary Representations of the Affine Group[J]. J. Math. Phys. , 1968, 9:206～211
[75] Paul T. Ondelettes et Mecanique Quantique[D]. Ph. D. Thesis, France: Universite de Marseille, 1985
[76] Smith M J T, Barnwell T P. Exact Reconstruction Techniques for Treestructured Subband Coders[J]. IEEE Trans. Acoust. Signal Speech Process, 1986, 34: 434～441
[77] Vetterli M. Filter Banks Allowing Perfect Reconstruction[J]. Signal Process. , 1986, 10:219～244
[78] 崔锦泰著,程正兴译,小波分析导论[M],陕西:西安交通大学出版社,1997.1
[79] Daubechies I. Ten Lectures on Wavelets[M]. Philadephia: the Society for Industrial and Applied Mathematics, 1992
[80] Holschneider M, Tchamitchian P G. Regularite Local de la Function Nondifferentiable'de Riemann[J] , Lemarie, 1990:102～124
[81] Grossmann A, Morlet J, Paul T. Transforms Associated to Square Integrable Group Representations[J] , Part Ⅰ: General Result. Jour. Math. Phys. , 1985, 27:2473～2479
[82] Grochenig G K. Describing Functions: Atomic Decompositions Versus Frames[J]. Monatsh: Math. , 1991, 112:1～42
[83] Hirchoren G A. Estimation of fractional brownian motion with multi- resolution kalman filter banks[J] , IEEE Transaction on Signal Processing, 1999, 47(5): 1431～1434
[84] 李元诚,李波,方廷健.基于小波支持向量机的非线性组合预测方法研究[J].信息与控制,2004.6,33(3):303～307
[85] Savaresi S M, Previdi F, Dester A, et al. Modeling, Identification, and Analysis of Limit-Cycling Pitch and Heave Dynamics in an ROV[J] , IEEE Journal of Oceanic Engineering, 2004. 4, 29(2): 407～417
[86] Helvoirt J V, Jager B D, Steinbuch M, et al. Stability Parameter Identification for a Centrifugal Compression System[A] , 43rd IEEE Conference on Decision and Control[C]. Atlantis, Paradise Island, Bahamas, 2004. 12:3400～3405
[87] 王雷,陈宗海.一种多神经网络混合模型的学习算法研究[J].系统仿真学报,2004.12,16(12):2680～2683
[88] Jing Xu, Qing-chun Meng, Dong-ming Zhou, et al. A Model Parameters Identification Method Based on Recurrent Neural Networks[A] , Proceedings of 3rd International Conf. on Machine Learning and Cybernetics[C]. Shanghai, 2004. 8:3207～3212
[89] Dong-hai Zhai, Li Li, Fan Jin. Fuzzy Neural Network for Nonlinear Systems Model Identification[A] , Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation[C]. Kobe, Japan, 2003. 7: 1282～1287
[90] Nomikos P, MacGregor J F. Monitoring Batch Process Using Multi-way Principle Component Analysis[J]. Journal of American Institute of Chemical Engineer, 1994, 40(8): 1361～1369
[91] MacGregor J F, Jaeckle C, Kiparissides C, et al, Process Monitoring and Diagnosis by Multiblock PLS Methods[J]. Journal of American Institute of Chemical Engineer, 1994, 40(5): 826～838
[92] 王军,肖建.模糊小波网络及其在永磁同步电机控制中的应用[D].博士学位论文.成都:西南交通大学,2005
[93] 苏金泷,欧阳钟辉.基于小波分解和FNN的图像数据可控有损压缩算法研究[J].计算机应用,2005,25(S1):221～224
[94] 刘琳,沈颂华,刘强.基于小波模糊网络的电厂汽轮发电机组故障诊断[J].电网技 术,2005,29(16):11～15,32
[95] 李波,覃征,石美红.利用小波变换和FCM算法进行多特征纹理分割[J].计算机工程,2005,31(24):148～150
[96] Zadeh LA. Fuzzy Sets[J]. Information and Control, 1965, 8:338-353
[97] Cantelli P. Sulla Determinazione Empirica Della Leggi di Probabilita[J]. G. Inst. Ital. Attuari, 1933, 4:1～19
[98] Chih-Wei Hsu, Chih-Jen Lin. A Comparison of Methods for Multi-class Support Vector Machines[J]. IEEE Transactions on Neural Networks, 2002. 3, 13(2): 415～425
[99] Liu K. Comparison of Very Short-term Load Forecasting Technique[J]. IEEE Trans. Power Systems, 1996, 11(2): 877～882
[100] 胡政,柳进,胡林献.电网高峰负荷分析决策平台的设计与实现[J].电网技术,2005.3,29(6):58～62
[101] 李端超,谢恒,江山立等.安徽电网实时发电控制系统设计及实现[J].电网技术,2001,25(1):62～66
[102] 招海丹,吴捷,杨苹等.一个综合智能化电力短期负荷预测系统的研究[J].电网技术,2000,4(12):45～48
[103] 孟宪生,余铭正,武寒等.用神经网络法预测特大型电网短期负荷的初探[J].华东电力,2000(6):4～6
[104] 柳进,于继来,唐降龙.基于数据挖掘的电网高峰负荷预测系统[J].计算机工程,2005,31(1):9～11
[105] Franklin P. W. A Theoretical Study of the Three Phase Salient Pole Type Generator with Simultaneous AC and Bridge Rectified DC output, Part Ⅰ and Part Ⅱ[J]. IEEE Transactions on Power Apparatus and Systems, 1973, 92(2): 543～557
[106] Schiferl R F. Six Phase Synchronous Machine with AC and DC Stator Connections, Part Ⅰ and Part Ⅱ[J]. IEEE Transactions on Power Apparatus and Systems, 1983, 102(8): 2685～2701
[107] MA Wei-ming, ZHANG Gai-fan, LIU De-zhi, et al. A Synchronous Machine with Simultaneous AD/DC output[P]. The Patent of China, ZL 94107628. 8, 1999. 09. 11
[108] 尤勇,盛万兴,王孙安.基于人工免疫网络的短期负荷预测模型[J].中国电机工程学报,2003.3,23(3):26～30
[109] 谢宏,程浩忠,张国立等.基于粗糙集理论建立短期电力负荷神经网络预测模型[J].中国电机工程学报,2003,23(11):1～4
[110] 田晓宇,李明干,刘沛.基于Kalman滤波的神经网络学习算法及其应用[J].计算机与数字工程,2005,33(2):40～43
[111] 赵晓煜,汪定伟.供应链中二级分销网络优化设计的模糊机会约束规划模型[J].控制理论与应用,2002,19(2):249～253
[112] 李驰宇,李远富,梁东.基于人工神经网络的交通运量预测[J].交通与安全,2005,8:130～132
[113] 许玮珑,马林才,王巍.BP神经网络在交通流量预测中的应用[J].交通与安全,2005,11:42～46
[114] 张毅,罗元.基于人工神经网络城市交通流量智能预测的研究[J].重庆邮电学院学报,2005,17(2):241～243
[115] 吴浩勇,丛玉良,王宏志.基于神经网络的交通参数预测方法[J].吉林大学学报, 2005,23(6):569～573
[116] 张朝元,胡光华,徐天泽.基于LS-SVM的交通流量时间序列预测[J].云南大学学报,2004,26(增刊):19～22
[117] 张朝元,胡光华,徐天泽等.支持向量机改进的神经网络的函数逼近[J].昆明理工大学学报,2004,29(6):148～152
[118] 殷英,张朝元,胡光华等.基于SVM的实时交通流模拟与预测系统设计[J].计算机工程与应用,2005,10,197～199
[119] 王继生,高宝成,时良平.支持向量机在交通量预测中的应用[J].信息技术,2004,28(4):8～10
[120] 徐启华,杨瑞.支持向量机在交通流量实时预测中的应用[J].公路交通科技,2005,22(12):131～134
[2] Rosenblatt F. Principles of Neurodinamics: Perceptron and Theory of Brain Mechanisms[M]. Spartan Books, Washington D. C. , 1962
[3] Novikoff A B J. On Convergence Proofs on Perceptrons[A]. Proceedings of the Symposium on the Mathematical Theory of Automata[C] , Poltechnic Institute of Brooklyn, 1962, Ⅻ: 615～622
[4] Vapnik V N, Chervonenkis A J. On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities[J]. Doklady Akademii Nauk USSR, 1968, 181(4): 173～187
[5] Ivanov V V. On Linear Problems Which are not Well-posed[J]. Soviet Math Dokl. , 1962, 3(4): 981～983
[6] Philips D Z. A Technique for Numerical Solution of Certain Integral Equation of the First Kind[J]. Journal of Association with Computer Machine, 1962, 9:84～96
[7] Tikhonov A N. On Solving ill-posed problem and method of regularization[J]. Doklady Akademii Nauk USSR, 1963, 153:501～504
[8] Vapnik V N, Stefanyuk A R. Nonparametric Methods for Estimation Probability Densities[J]. Automation and Remote Control, 1978, 8:27～35
[9] Solomonoff R J. A Preliminary Report on General Theory of Inductive Inference[R]. Technical Report ZTB-138, Zator Company, Cambridge, 1960
[10] Kolmogoroff A N. Three Approaches to the Quantitative Definitions of Information[J]. Problem of Information Transmission, 1965, 1(1): 1～7
[11] Chaitin G J. On the Length of Programs for Computing Finite Binary Sequences[J]. Journal of Association with Computer Machine, 1966, 13:547～569
[12] Rissanen J. Modeling by Shortest Data Description[J]. Automatica, 1978, 14: 465～471
[13] LeCun Y. Learning Processes in an Asymmetric Threshold Network[J]. Disordered Systems and Biological Organizations, Les. Houches, France, Springer, 1986, 9: 233～240
[14] Rumelhart D E, Hinton G E, Williams R J. Learning Internal Representations by Error Propagation[M]. Parallel Distributed Processing: Explorations in Macrostructure of coganition, 1986, Ⅰ, Badford Books, Cambridge, MA. :318～362
[15] Vapnik V N. Three Fundamental Concepts of the Capacity of Learning Machines[J]. Physica A, 1993, 200:538～544
[16] Vapnik V N, Bottou L. Local Algorithms for Pattern Recognition and Dependencies Estimation[J]. Neural Computation, 1993, 5(6): 893～908
[17] Vapnik V N. Statistical Learning Theory[M]. J. Wiley, New York, 1998
[18] Mercy J. Functions of Positive and negative type and their connection with the theory of integral equations[J]. Philosophical Transactions of the Royal Society, London, 1909, A, 209:415～446
[19] Boser B E, Guyon I M, Vapnik V N. A Training Algorithm for Optimal Margin Classifiers[J]. In D. Haussler Editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, ACM Press, 1992:144～152
[20] Scholkoph B A, Smola A J, Muller K R. Kernel Principal Component Analysis, Advances in Kernel Methods-Support Vector Learning[M]. MIT Press, Cambridge, MA, 1999:327～352
[21] Scholkoph B A, Smola A J, Bartlett P L, New Support Vector Algorithms[J]. Neural Computation, 2000, 12:1207～1245
[22] Suykens J A, Branbanter J K, Lukas L, et al. , Weighted Least Squares Support Vector Machine: Robustness and Spare Approximation[J]. Neurocomputing, 2002, 48(1): 85～105
[23] Lin C F, Wang S D. Fuzzy Support Vector Machines[J]. IEEE Transaction on Neural Networks, 2002, 48(1): 85～105
[24] Keoman V, Hadzic I, Support Vectors Selection by Linear Programming[A]. Proceedings of the IEEE- INNS-ENNS International Joint Conference on Networks[C] , Como. Italy, 2000, 5:193～198
[25] Laskov P, Feasible Direction Decomposition Algorithms for Training Support Vector Machine[J] , Machine Learning, 2002, 46(1): 315～349
[26] Smola A J, Scholkoph B A, A Tutorial on Support Vector Regression[R]. Royal Holloway College, NeuroCOLT Technical Report TR- 1998-030, 1998
[27] Carozza M, Rampone S. Towards an Incremental SVM for Regression[A]. Proceedings of the IEEE-ENNS Int Joint Conference on Neural Networks, 2000, 6: 405～410
[28] Gestel T V. Financial Time Series Prediction Using Least Squares Support Vector Machines within the Evidence Framework[J]. IEEE Transaction on Neural Networks, 2001, 12(4): 809～821
[29] Gretton A. Support Vector Regression for Black-box System Identification[A]. Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing[C] , Singapore, 2001, 341～344
[30] Chan W C, Chan C W, Cheung K C, et al. On the Modeling of Nonlinear Dynamic Systems using Support Vector Neural Networks[J]. Engineering Application of Artificial Intelligence, 2001, 14(2): 105～113
[31] Kruif B J, Vries T J. On Using a Support Vector Machine in Learning feed-forward control[A]. Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics[C] , Coma, Italy, 2001, 1:272～277
[32] 陈念贻,陆文聪,武海顺等.支持向量机算法在氮化铝薄生长过程控制中的应用[J].计算机与应用化学,2002,19(6):726～728
[33] 潘继斌.回归函数的支持向量机估计法[J].湖北师范学院学报,2003(4),23:9～13
[34] 李良敏,屈梁生.基于遗传编程和支持向量机的故障诊断模型[J].西安交通大学学报,2004(3),38:239～242
[35] 刘洪,赵金洲,胡永全等,支持向量机在重复压裂中的应用研究[J],天然气工业, 2004(3), 24:75-77
[36] Gabor D. Theory of Communication [J]. Journal of IEE, 1946, 93: 429-457
[37] Ville J. Theorie et Applications de la Notion de Signal Analytique [J]. Cables et Transmission, 1948,2A: 61-74
[38] Page C H. Instantaneous Power Spectra [J]. Journal of Physical Application, 1952,23:103-106
[39] Namias V. The Fractional Fourier Transform and Its Application in Quantum Mechanics [J]. Journal of Institute of Math, and Its Application, 1980,25: 241-265
[40] Mann S, Haykin S. "Chirplets and Wavelets": Novel Time-frequency Methods [J]. Electronic Letters, 1992, 28: 114-116
[41] Mann S, Haykin S. Adaptive "Chirplet" Transform: an Adaptive Generalization of the Wavelet Transform [J]. Optical England, 1992, 31: 1243-1256
[42] Mann S, Haykin S. The Chirplet Transform: Physical Considerations [J]. IEEE Transaction on Signal Processing, 1995,43: 2745-2761
[43] Mihovilovic D, Bracewell R N. Adative Chirplet Representation of Signals on Time-frequency Plane [J]. Electronics Letters, 1991, 27: 1159-1161
[44] Mihovilovic D, Bracewell R N. Whistler Analysis in the Time-frequency Plane Using Chirplets [J]. Journal of Geophysical Research, 1992, 97: 17,199-204
[45] Grosses A., Morlet J. Decomposition of Hardy Function into Square Wavelets of Constant Shapes [J]. SIAM J Math. Anal, 1984(15):723~726
[46] Meyer Y. Principle D'incertitude bases Hilbertiennes et Algebra D'Operataur [J]. Bourbaki Seminaire, Asterisque (Societe Mathematique de France), 1985, 2: 662-690
[47] Meyer Y. Ondelettes, Functions Splines et Analyses Graduees [R]. Italy: Lectures Given at the University of Torino, 1986
[48] Mallat S. A Theory for Multiresolution Signal Decomposition: The Wavelet Represention. IEEE Trans. Pattern Analysis and Machine Interlligence, 1989,11(7):674~693
[49] Mallat S. Multifrequency Channel Decomposition of Images and Wavelet Model. IEEE Trans. Acoust. Speech Signal Processing, 1989, 37:2091-2110
[50] Daubechies I. Orthonormal Bases of Compactly Supported Wavelets. Commun. Pure Appl, 1988,41:990-996
[51 ] Vapnik V N, Chervonenkis A J. On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities [J]. Theory Probab. Apl., 1971, 16: 264-280
[52] Vapnik V N. Estimation of Dependencies Based on Empirical Data [M]. Nauka,Moscow, 1979 (English Edition: Vapnik V N. Estimation of Depen- dencies Based on Empirical Data [M], Springer, New York, 1982)
[53] Robbins H, Monroe H. A Stochastic Approximation Method [J]. Annalsof Mathematical Statistics, 1951, 22: 400-407
[54] Aizerman M A, Braveman E M, Rozonoer L I. Theoretical Foundation of Potential Function Method in Pattern Recognition Learning [J]. Automation and Remote Control, 1964,25:821-837
[55] Aizerman M A, Braveman E M, Rozonoer L I. The Robbince-Monroe Process and the Method of Potential Functions [J]. Automation and Reote Control, 1965, 1882-1885
[56] Amari S. A Theory of Adaptive Pattern Classifiers[J]. IEEE Trans. Elect. Comp. , 1967, EC_16:299～307
[57] Tsypkin Y Z. Adaptation and Learning in Automatic Systems[M]. Academic, New York, 1971
[58] Tsypkin Y Z. Foundation of the Theory of Learning Systems[M]. Academic, New York, 1973
[59] Vapnik V N, Che rvonenkis A J. Theory of Pattern Recognition[M]. Nauka, Moscow, 1974
[60] Cristianini N, Taylor J S. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods[M]. England: Cambridge University Press, 2000. 4
[61] Huber P. Robust Estimation of Location Parameter[J]. Ann. Math. Stat. , 1964, 35(1)
[62] Vapnik V N. The Nature of Statistical Learning Theory[M]. 2nd Edition, New York: Springer verlag, 1999:147～156
[63] 许建华,张学工,李衍达.一种基于核函数的非线性感知机算法[J].计算机学报,2002,7(25):689～695
[64] 张亭禄,贺明霞.基于人工神经网络的一类水域叶绿素a浓度反演方法[J].遥感学报,2001,6(1):40～44
[65] 黄海清,何贤强,王迪峰,等.神经网络法反演海水叶绿素浓度的分析[J].地球信息科学,2004,6(2):31～37
[66] 冯春晶,赵朝方.基于人工神经网络研究海水中叶绿素浓度的垂直分布[J].中国海洋大学学报,2004,34(3):497～505
[67] 裴洪平,罗妮娜,蒋勇.利用BP神经网络方法预测西湖叶绿素a的浓度[J].生态学报,2004,24(2):246～251
[68] Slade W H, Richard J, Miller L, et al. Ensemble Neural Network Methods for Satellite-Derived Estimation of Chlorophyll a[J]. IEEE Transaction on Geoscience and Remote Sensing, 2003:547～552
[69] Davide D' Alimonte, Giuseppe Zibordi. Phytoplankton Determination in an Optically Complex Coastal Region Using a Multilayer Perception Neural Network[J]. IEEE Transaction on Geoscience and Remote Sensing, 2003, 41 (12): 1861～1868
[70] 边肇祺,张学工.模式识别(第二版)[M].北京:清华大学出版社,2000.1
[71] 杜平,张燕昆,刘重庆.基于不变矩的人脸识别方法的研究[J].计算机仿真,2002,19(3):78～81
[72] 祝海龙,屈梁生,张海军.基于小波变换和支持向量机的人脸检测系统[J].西安交通大学学报,2002,36(9):947～950
[73] Caldenon A P. Intermediate Spaces and Interpolation, the Complex Method[J]. Study on Math. , 1964, 24:113～190
[74] Aslaksen E W, Klauder J R. Unitary Representations of the Affine Group[J]. J. Math. Phys. , 1968, 9:206～211
[75] Paul T. Ondelettes et Mecanique Quantique[D]. Ph. D. Thesis, France: Universite de Marseille, 1985
[76] Smith M J T, Barnwell T P. Exact Reconstruction Techniques for Treestructured Subband Coders[J]. IEEE Trans. Acoust. Signal Speech Process, 1986, 34: 434～441
[77] Vetterli M. Filter Banks Allowing Perfect Reconstruction[J]. Signal Process. , 1986, 10:219～244
[78] 崔锦泰著,程正兴译,小波分析导论[M],陕西:西安交通大学出版社,1997.1
[79] Daubechies I. Ten Lectures on Wavelets[M]. Philadephia: the Society for Industrial and Applied Mathematics, 1992
[80] Holschneider M, Tchamitchian P G. Regularite Local de la Function Nondifferentiable'de Riemann[J] , Lemarie, 1990:102～124
[81] Grossmann A, Morlet J, Paul T. Transforms Associated to Square Integrable Group Representations[J] , Part Ⅰ: General Result. Jour. Math. Phys. , 1985, 27:2473～2479
[82] Grochenig G K. Describing Functions: Atomic Decompositions Versus Frames[J]. Monatsh: Math. , 1991, 112:1～42
[83] Hirchoren G A. Estimation of fractional brownian motion with multi- resolution kalman filter banks[J] , IEEE Transaction on Signal Processing, 1999, 47(5): 1431～1434
[84] 李元诚,李波,方廷健.基于小波支持向量机的非线性组合预测方法研究[J].信息与控制,2004.6,33(3):303～307
[85] Savaresi S M, Previdi F, Dester A, et al. Modeling, Identification, and Analysis of Limit-Cycling Pitch and Heave Dynamics in an ROV[J] , IEEE Journal of Oceanic Engineering, 2004. 4, 29(2): 407～417
[86] Helvoirt J V, Jager B D, Steinbuch M, et al. Stability Parameter Identification for a Centrifugal Compression System[A] , 43rd IEEE Conference on Decision and Control[C]. Atlantis, Paradise Island, Bahamas, 2004. 12:3400～3405
[87] 王雷,陈宗海.一种多神经网络混合模型的学习算法研究[J].系统仿真学报,2004.12,16(12):2680～2683
[88] Jing Xu, Qing-chun Meng, Dong-ming Zhou, et al. A Model Parameters Identification Method Based on Recurrent Neural Networks[A] , Proceedings of 3rd International Conf. on Machine Learning and Cybernetics[C]. Shanghai, 2004. 8:3207～3212
[89] Dong-hai Zhai, Li Li, Fan Jin. Fuzzy Neural Network for Nonlinear Systems Model Identification[A] , Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation[C]. Kobe, Japan, 2003. 7: 1282～1287
[90] Nomikos P, MacGregor J F. Monitoring Batch Process Using Multi-way Principle Component Analysis[J]. Journal of American Institute of Chemical Engineer, 1994, 40(8): 1361～1369
[91] MacGregor J F, Jaeckle C, Kiparissides C, et al, Process Monitoring and Diagnosis by Multiblock PLS Methods[J]. Journal of American Institute of Chemical Engineer, 1994, 40(5): 826～838
[92] 王军,肖建.模糊小波网络及其在永磁同步电机控制中的应用[D].博士学位论文.成都:西南交通大学,2005
[93] 苏金泷,欧阳钟辉.基于小波分解和FNN的图像数据可控有损压缩算法研究[J].计算机应用,2005,25(S1):221～224
[94] 刘琳,沈颂华,刘强.基于小波模糊网络的电厂汽轮发电机组故障诊断[J].电网技 术,2005,29(16):11～15,32
[95] 李波,覃征,石美红.利用小波变换和FCM算法进行多特征纹理分割[J].计算机工程,2005,31(24):148～150
[96] Zadeh LA. Fuzzy Sets[J]. Information and Control, 1965, 8:338-353
[97] Cantelli P. Sulla Determinazione Empirica Della Leggi di Probabilita[J]. G. Inst. Ital. Attuari, 1933, 4:1～19
[98] Chih-Wei Hsu, Chih-Jen Lin. A Comparison of Methods for Multi-class Support Vector Machines[J]. IEEE Transactions on Neural Networks, 2002. 3, 13(2): 415～425
[99] Liu K. Comparison of Very Short-term Load Forecasting Technique[J]. IEEE Trans. Power Systems, 1996, 11(2): 877～882
[100] 胡政,柳进,胡林献.电网高峰负荷分析决策平台的设计与实现[J].电网技术,2005.3,29(6):58～62
[101] 李端超,谢恒,江山立等.安徽电网实时发电控制系统设计及实现[J].电网技术,2001,25(1):62～66
[102] 招海丹,吴捷,杨苹等.一个综合智能化电力短期负荷预测系统的研究[J].电网技术,2000,4(12):45～48
[103] 孟宪生,余铭正,武寒等.用神经网络法预测特大型电网短期负荷的初探[J].华东电力,2000(6):4～6
[104] 柳进,于继来,唐降龙.基于数据挖掘的电网高峰负荷预测系统[J].计算机工程,2005,31(1):9～11
[105] Franklin P. W. A Theoretical Study of the Three Phase Salient Pole Type Generator with Simultaneous AC and Bridge Rectified DC output, Part Ⅰ and Part Ⅱ[J]. IEEE Transactions on Power Apparatus and Systems, 1973, 92(2): 543～557
[106] Schiferl R F. Six Phase Synchronous Machine with AC and DC Stator Connections, Part Ⅰ and Part Ⅱ[J]. IEEE Transactions on Power Apparatus and Systems, 1983, 102(8): 2685～2701
[107] MA Wei-ming, ZHANG Gai-fan, LIU De-zhi, et al. A Synchronous Machine with Simultaneous AD/DC output[P]. The Patent of China, ZL 94107628. 8, 1999. 09. 11
[108] 尤勇,盛万兴,王孙安.基于人工免疫网络的短期负荷预测模型[J].中国电机工程学报,2003.3,23(3):26～30
[109] 谢宏,程浩忠,张国立等.基于粗糙集理论建立短期电力负荷神经网络预测模型[J].中国电机工程学报,2003,23(11):1～4
[110] 田晓宇,李明干,刘沛.基于Kalman滤波的神经网络学习算法及其应用[J].计算机与数字工程,2005,33(2):40～43
[111] 赵晓煜,汪定伟.供应链中二级分销网络优化设计的模糊机会约束规划模型[J].控制理论与应用,2002,19(2):249～253
[112] 李驰宇,李远富,梁东.基于人工神经网络的交通运量预测[J].交通与安全,2005,8:130～132
[113] 许玮珑,马林才,王巍.BP神经网络在交通流量预测中的应用[J].交通与安全,2005,11:42～46
[114] 张毅,罗元.基于人工神经网络城市交通流量智能预测的研究[J].重庆邮电学院学报,2005,17(2):241～243
[115] 吴浩勇,丛玉良,王宏志.基于神经网络的交通参数预测方法[J].吉林大学学报, 2005,23(6):569～573
[116] 张朝元,胡光华,徐天泽.基于LS-SVM的交通流量时间序列预测[J].云南大学学报,2004,26(增刊):19～22
[117] 张朝元,胡光华,徐天泽等.支持向量机改进的神经网络的函数逼近[J].昆明理工大学学报,2004,29(6):148～152
[118] 殷英,张朝元,胡光华等.基于SVM的实时交通流模拟与预测系统设计[J].计算机工程与应用,2005,10,197～199
[119] 王继生,高宝成,时良平.支持向量机在交通量预测中的应用[J].信息技术,2004,28(4):8～10
[120] 徐启华,杨瑞.支持向量机在交通流量实时预测中的应用[J].公路交通科技,2005,22(12):131～134