不确定空间上基于复样本的统计学习理论基础
|
摘要
统计学习理论是处理小样本学习问题的重要理论,但该理论是建立在概率空间上基于实随机样本的,它难以处理现实世界中客观存在的非概率空间上基于非实随机样本的小样本统计学习问题。不确定空间是比概率空间更广的空间,本文讨论了不确定空间上基于复样本的统计学习理论。首先,给出了复不确定变量及其分布函数、期望和方差的概念,证明了不确定空间上基于复样本的Markov不等式、Chebyshev不等式和Khintchine大数定律;其次,引入了不确定空间上基于复样本的复经验风险泛函、复期望风险泛函、复经验风险最小化原则以及复严格一致收敛的定义,证明了不确定空间上基于复样本的学习理论的关键定理;最后,讨论了不确定空间上基于复样本的学习过程一致收敛速度的界。
Statistical Learning Theory (SLT) is one of the most important theories to deal with small samples. However, the theory is built on probability space and based on real random samples, so it can hardly handle statistical learning problems built on non-probability space and based on non-real random samples in real world. Uncertainty space is wider than probability space. In this dissertation, SLT built on uncertainty space and based on complex random samples is discussed. Firstly, the definitions of complex uncertain variable together with its distribution function, expected value, and variance are presented. Then Markov inequality, Chebyshev inequality and Khintchine law of large numbers built on uncertainty space and based on complex samples are also proved. Secondly, some new concepts, such as complex empirical risk functional, complex expected risk functional, and strict consistency of the complex empirical risk minimization principle built on uncertainty space and based on complex samples, are introduced. The key theorem of learning theory built on uncertainty space and based on complex samples is proved. At last, the bounds on the rate of convergence of learning process built on uncertainty space and based on complex random samples are given.
Statistical Learning Theory (SLT) is one of the most important theories to deal with small samples. However, the theory is built on probability space and based on real random samples, so it can hardly handle statistical learning problems built on non-probability space and based on non-real random samples in real world. Uncertainty space is wider than probability space. In this dissertation, SLT built on uncertainty space and based on complex random samples is discussed. Firstly, the definitions of complex uncertain variable together with its distribution function, expected value, and variance are presented. Then Markov inequality, Chebyshev inequality and Khintchine law of large numbers built on uncertainty space and based on complex samples are also proved. Secondly, some new concepts, such as complex empirical risk functional, complex expected risk functional, and strict consistency of the complex empirical risk minimization principle built on uncertainty space and based on complex samples, are introduced. The key theorem of learning theory built on uncertainty space and based on complex samples is proved. At last, the bounds on the rate of convergence of learning process built on uncertainty space and based on complex random samples are given.
引文
[1] V. N. Vapnik. The nature of statistical learning theory. New York: Springer-Verlag, 1995.
[2] V. N. Vapnik. Statistical learning theory. New York: A Wiley-Interscience Publication, 1998.
[3] V. N. Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 1999, 10(5): 988-999.
[4] N. Cristinanini and J. Shawe-Taylor. An introduction to support vector machines. Cambridge: Cambridge University Press, 2000.
[5]边肇祺,张学工.模式识别(第二版).北京:清华大学出版社, 2000.
[6]邓乃扬,田英杰,等.数据挖掘中的新方法—支持向量机.北京:科学出版社, 2004.
[7] C. Hwang, D. H. Hong and K. H. Seok. Support vector interval regression machine for crisp input and output data. Fuzzy Sets and Systems, 2006, 157(8): 1114-1125.
[8]孙大瑞,吴乐南.基于非线性特征提取和SVM的人脸识别算法.电子与信息学报, 2004, 26(2): 307-311.
[9] J. M. Li, B. Zhang and F. Z. Lin. Nonlinear speech model based on support vector machine and wavelet transform. Proceedings of the 15th IEEE International Conference on Tools with Artificial Intelligence. Washington, USA, June 2003, 259-264.
[10] H. Wechsler and F. Li. Motion estimation using statistical learning theory. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(4): 466-478.
[11]高学,金连文,尹俊勋,等.一种基于支持向量机的手写汉字识别方法.电子学报, 2002, 30(5): 651-654.
[12]刘广利,邓乃扬.基于SVM分类的预警系统.中国农业大学学报, 2002, 7(6): 97-100.
[13] Y. Q. Wang, S. Y. Wang and K .K. Lai. A new fuzzy support vector machine to evaluate credit risk. IEEE Transactions on Fuzzy Systems, 2005, 13(6): 820-931.
[14] C. Su and C. H. Yang. Feature selection for the SVM: An application to hypertension diagnosis. Journal of Hydrology, 2008, 34(1): 754-763.
[15]马潜云,张学工.支持向量机函数拟合在分形插值中的应用.清华大学学报(自然科学版), 2000, 40(3): 76-78.
[16]杨利民,刘广利.不确定性支持向量机原理及应用.北京:科学出版社, 2007.
[17]哈明虎,周彩丽.基于模糊样本的学习理论的关键定理.华北科技学院学报, 2004(1): 11- 15.
[18]田静.模糊学习过程一致收敛速度的界.保定:河北大学数学与计算机学院(理学硕士学位论文), 2005.
[19]周彩丽,哈明虎,鲍俊艳,等.基于模糊数的模糊学习理论的关键定理.河北大学学报(自然科学版), 2008, 28(5): 449-451,455.
[20]张植明.基于随机粗糙样本的统计学习理论研究.计算机工程与应用. 2008, 44(31): 43-63.
[21]张植明,田景峰.基于双重粗糙样本的统计学习理论的理论基础.应用数学学报, 2009, 32(4): 608 -619.
[22]田景峰,张植明.受噪声影响的复gλ样本的学习理论的关键定理.计算机工程与应用, 2009, 45(5): 59-63.
[23]张植明,田景峰,哈明虎.基于复拟随机样本的统计学习理论的理论基础.计算机工程与应用, 2008, 44(9): 82-86.
[24]张植明,田景峰. Sugeno测度空间基于复样本的统计学习理论.计算机工程与应用, 2009, 45(7): 59-64.
[25] M. H. Ha, W. Pedrycz, Z. M. Zhang, et al. The theoretical foundations of statistical learning theory of complex random samples. Far East Journal of Applied Mathematics, 2009, 34(3): 315-336.
[26]白云超.可能性空间上统计学习理论的的关键定理.保定:河北大学数学与计算机学院(理学硕士学位论文), 2004.
[27]哈明虎,王鹏.可能性空间中学习过程一致收敛速度的界.河北大学学报, 2004, 24(1): 1-6.
[28] M. H. Ha, Y. C. Bai and W. G. Tang. The sub-key theorem on credibility measure space. Proceeding of 2003 International Conference on Machine Learning and Cybernetics, Xi’an, China, 2003, 5: 3264-3268.
[29] M. H. Ha, Y. C. Bai, P. Wang, et al. The key theorem and the bounds on the rate of uniform convergence of statistical learning theory on a credibility space. Advances in Fuzzy Sets and Systems, 2006, 1(2): 143-172.
[30] M. H. Ha, Y. Li, J. Li, et al. The key theorem and the bounds on the rate of uniform convergence of learning theory on Sugeno measure space. Science in China, Series F Information Sciences, 2006, 49(3): 372-385.
[31]哈明虎,李颜,李嘉,等. Sugeno测度空间上学习理论的关键定理和一致收敛速度的界.中国科学, 2006, 36⑷: 398-410.
[32]哈明虎,冯志芳,宋士吉,等.拟概率空间上学习理论的关键定理和学习过程一致收敛速度的界.计算机学报, 2008, 31(3): 476-485.
[33] X. K. Zhang, M. H. Ha, J. Wu, et al. The bounds on the rate of uniform convergence of learning process on uncertainty space. The Sixth International Symposium on Neural Networks, Wuhan, China, May 2009, 110-117.
[34] S. J. Yan, M. H. Ha, X. K. Zhang, et al. The key theorem of learning theory on uncertainty space .The Sixth International Symposium on Neural Networks, Wuhan, China, May 2009, 699-706.
[35] B. D. Liu. Uncertainty theory, 3rd ed.. http://orsc.edu.cn/liu/ut.pdf, 2008.
[36]严士健,王隽骧,刘秀芳.概率论基础.北京:科学出版社, 1999.
[37]定光桂.巴拿赫空间引论.北京:科学出版社, 2001.
[2] V. N. Vapnik. Statistical learning theory. New York: A Wiley-Interscience Publication, 1998.
[3] V. N. Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 1999, 10(5): 988-999.
[4] N. Cristinanini and J. Shawe-Taylor. An introduction to support vector machines. Cambridge: Cambridge University Press, 2000.
[5]边肇祺,张学工.模式识别(第二版).北京:清华大学出版社, 2000.
[6]邓乃扬,田英杰,等.数据挖掘中的新方法—支持向量机.北京:科学出版社, 2004.
[7] C. Hwang, D. H. Hong and K. H. Seok. Support vector interval regression machine for crisp input and output data. Fuzzy Sets and Systems, 2006, 157(8): 1114-1125.
[8]孙大瑞,吴乐南.基于非线性特征提取和SVM的人脸识别算法.电子与信息学报, 2004, 26(2): 307-311.
[9] J. M. Li, B. Zhang and F. Z. Lin. Nonlinear speech model based on support vector machine and wavelet transform. Proceedings of the 15th IEEE International Conference on Tools with Artificial Intelligence. Washington, USA, June 2003, 259-264.
[10] H. Wechsler and F. Li. Motion estimation using statistical learning theory. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(4): 466-478.
[11]高学,金连文,尹俊勋,等.一种基于支持向量机的手写汉字识别方法.电子学报, 2002, 30(5): 651-654.
[12]刘广利,邓乃扬.基于SVM分类的预警系统.中国农业大学学报, 2002, 7(6): 97-100.
[13] Y. Q. Wang, S. Y. Wang and K .K. Lai. A new fuzzy support vector machine to evaluate credit risk. IEEE Transactions on Fuzzy Systems, 2005, 13(6): 820-931.
[14] C. Su and C. H. Yang. Feature selection for the SVM: An application to hypertension diagnosis. Journal of Hydrology, 2008, 34(1): 754-763.
[15]马潜云,张学工.支持向量机函数拟合在分形插值中的应用.清华大学学报(自然科学版), 2000, 40(3): 76-78.
[16]杨利民,刘广利.不确定性支持向量机原理及应用.北京:科学出版社, 2007.
[17]哈明虎,周彩丽.基于模糊样本的学习理论的关键定理.华北科技学院学报, 2004(1): 11- 15.
[18]田静.模糊学习过程一致收敛速度的界.保定:河北大学数学与计算机学院(理学硕士学位论文), 2005.
[19]周彩丽,哈明虎,鲍俊艳,等.基于模糊数的模糊学习理论的关键定理.河北大学学报(自然科学版), 2008, 28(5): 449-451,455.
[20]张植明.基于随机粗糙样本的统计学习理论研究.计算机工程与应用. 2008, 44(31): 43-63.
[21]张植明,田景峰.基于双重粗糙样本的统计学习理论的理论基础.应用数学学报, 2009, 32(4): 608 -619.
[22]田景峰,张植明.受噪声影响的复gλ样本的学习理论的关键定理.计算机工程与应用, 2009, 45(5): 59-63.
[23]张植明,田景峰,哈明虎.基于复拟随机样本的统计学习理论的理论基础.计算机工程与应用, 2008, 44(9): 82-86.
[24]张植明,田景峰. Sugeno测度空间基于复样本的统计学习理论.计算机工程与应用, 2009, 45(7): 59-64.
[25] M. H. Ha, W. Pedrycz, Z. M. Zhang, et al. The theoretical foundations of statistical learning theory of complex random samples. Far East Journal of Applied Mathematics, 2009, 34(3): 315-336.
[26]白云超.可能性空间上统计学习理论的的关键定理.保定:河北大学数学与计算机学院(理学硕士学位论文), 2004.
[27]哈明虎,王鹏.可能性空间中学习过程一致收敛速度的界.河北大学学报, 2004, 24(1): 1-6.
[28] M. H. Ha, Y. C. Bai and W. G. Tang. The sub-key theorem on credibility measure space. Proceeding of 2003 International Conference on Machine Learning and Cybernetics, Xi’an, China, 2003, 5: 3264-3268.
[29] M. H. Ha, Y. C. Bai, P. Wang, et al. The key theorem and the bounds on the rate of uniform convergence of statistical learning theory on a credibility space. Advances in Fuzzy Sets and Systems, 2006, 1(2): 143-172.
[30] M. H. Ha, Y. Li, J. Li, et al. The key theorem and the bounds on the rate of uniform convergence of learning theory on Sugeno measure space. Science in China, Series F Information Sciences, 2006, 49(3): 372-385.
[31]哈明虎,李颜,李嘉,等. Sugeno测度空间上学习理论的关键定理和一致收敛速度的界.中国科学, 2006, 36⑷: 398-410.
[32]哈明虎,冯志芳,宋士吉,等.拟概率空间上学习理论的关键定理和学习过程一致收敛速度的界.计算机学报, 2008, 31(3): 476-485.
[33] X. K. Zhang, M. H. Ha, J. Wu, et al. The bounds on the rate of uniform convergence of learning process on uncertainty space. The Sixth International Symposium on Neural Networks, Wuhan, China, May 2009, 110-117.
[34] S. J. Yan, M. H. Ha, X. K. Zhang, et al. The key theorem of learning theory on uncertainty space .The Sixth International Symposium on Neural Networks, Wuhan, China, May 2009, 699-706.
[35] B. D. Liu. Uncertainty theory, 3rd ed.. http://orsc.edu.cn/liu/ut.pdf, 2008.
[36]严士健,王隽骧,刘秀芳.概率论基础.北京:科学出版社, 1999.
[37]定光桂.巴拿赫空间引论.北京:科学出版社, 2001.