基于深层自适应平衡自编码机的手写数字分类
|
摘要
针对深层自动编码机(DAE)内部协变量迁移问题,提出一种深层自适应平衡自动编码机(DSBAE),用于手写数字的分类(HDC)。重点研究了自适应平衡层对内部协变量迁移的纠正方法,构建了DSBAE网络的分类模型,根据平衡网络参数体系原理,制定了自适应参数更新策略。实验在MNIST,USPS以及PENDIGITS三个公开手写数据集上对DSBAE以及深度学习中其他分类算法进行比较,证明DSBAE能有效解决深层网络的内部协变量迁移问题,并在手写数字分类准确率上占有明显优势。
To solve the problem of internal covariate migration for deep auto encoder( DAE),a deep self-adaptive balance auto encoder( DSBAE) is proposed for handwritten digital classification( HDC). Beside constructing the DSBAE network classification model,correcting method of the self-adaptive balance layer to the internal covariate migration is also focused on. Updating strategy of self-adaptive parameters is worked out according to principle of balanced network parameter system. The experiment compares DSBAE and other classification algorithms of deep learning in MNIST,USPS and PENDIGITS datasets,which proves that DSBAE can effectively solve the problem of internal covariate migration in deep network and obtain a clear advantage in accuracy of HDC.
To solve the problem of internal covariate migration for deep auto encoder( DAE),a deep self-adaptive balance auto encoder( DSBAE) is proposed for handwritten digital classification( HDC). Beside constructing the DSBAE network classification model,correcting method of the self-adaptive balance layer to the internal covariate migration is also focused on. Updating strategy of self-adaptive parameters is worked out according to principle of balanced network parameter system. The experiment compares DSBAE and other classification algorithms of deep learning in MNIST,USPS and PENDIGITS datasets,which proves that DSBAE can effectively solve the problem of internal covariate migration in deep network and obtain a clear advantage in accuracy of HDC.
引文
[1] Ekachai P,Rapeepol C. Post-processing of unsupervised dictionary learning in handwritten digit recognition[C]∥2014 International Symposium on Communications and Information Technologies(ISCIT),2014:166-170.
[2] Bengio Y,Delalleau O. On the expressive power of deep architectures[C]∥Proc of the 14th International Conference on Discovery Science,Berlin,Springer Verlag,2011:18-36.
[3] Bengio Y. Learning deep architectures for AI[J]. Foundations and Trends in Machine Learning,2009,2(1):1-12.
[4] Sankiaran A,Pandey P,Vatsa M,et al. On latent fingerprint minutiae extraction using stacked denoising sparse autoencoders[C]∥Proceedings of the 2014 IEEE International Joint Conference on Biometrics(IJCB),Clearwater,2014:1-7.
[5]张彦,彭华.基于深度自编码器的单样本人脸识别[J].模式识别与人工智能,2017,30(4):343-352.
[6]郭晓洁,陈良,沈长青,等.自适应卷积神经网络在人脸识别上的应用[J].自动化技术与应用,2017,36(7):72-77.
[7]马鸿飞,赵月娇,刘珂,等.栈自动编码机语音分类算法[J].西安电子科技大学学报,2017,44(5):13-18.
[8] Ioffe S,Szegedy C. Batch normalization:Accelerating deep network training by reducing internal covariate shift[C]∥International Conference on Machine Learning,2015.
[9] Song Z J,Yu R. Research on the classification of handwriting number based on deep learning[J]. Chongqing Technol Business Univ,2015,32(8):49-53.
[10] Chandra B,Rajesh K Sharma. Fast learning in deep neural networks[J]. Neurocomputing,2016,171:1205-1215.
[11]董晴,宋威.基于粒子群优化的深度神经网络分类算法[J].传感器与微系统,2017,36(9):143-146.
[12]刘云龙,谢寿生,郑晓飞,等.基于深度学习的航空发动机传感器故障检测[J].传感器与微系统,2017,36(9):147-151.
[13] Salakhutdinov R,Hinton G. Semantic Hashing[J]. International Journal of Approximate Reasoning,2009,50(7):969-978.
[14] Glorot X,Bengio Y. Understanding the difficulty of training deep feedforward neural networks[J]. Journal of Machine Learning Research,2015,9:249-256.
[2] Bengio Y,Delalleau O. On the expressive power of deep architectures[C]∥Proc of the 14th International Conference on Discovery Science,Berlin,Springer Verlag,2011:18-36.
[3] Bengio Y. Learning deep architectures for AI[J]. Foundations and Trends in Machine Learning,2009,2(1):1-12.
[4] Sankiaran A,Pandey P,Vatsa M,et al. On latent fingerprint minutiae extraction using stacked denoising sparse autoencoders[C]∥Proceedings of the 2014 IEEE International Joint Conference on Biometrics(IJCB),Clearwater,2014:1-7.
[5]张彦,彭华.基于深度自编码器的单样本人脸识别[J].模式识别与人工智能,2017,30(4):343-352.
[6]郭晓洁,陈良,沈长青,等.自适应卷积神经网络在人脸识别上的应用[J].自动化技术与应用,2017,36(7):72-77.
[7]马鸿飞,赵月娇,刘珂,等.栈自动编码机语音分类算法[J].西安电子科技大学学报,2017,44(5):13-18.
[8] Ioffe S,Szegedy C. Batch normalization:Accelerating deep network training by reducing internal covariate shift[C]∥International Conference on Machine Learning,2015.
[9] Song Z J,Yu R. Research on the classification of handwriting number based on deep learning[J]. Chongqing Technol Business Univ,2015,32(8):49-53.
[10] Chandra B,Rajesh K Sharma. Fast learning in deep neural networks[J]. Neurocomputing,2016,171:1205-1215.
[11]董晴,宋威.基于粒子群优化的深度神经网络分类算法[J].传感器与微系统,2017,36(9):143-146.
[12]刘云龙,谢寿生,郑晓飞,等.基于深度学习的航空发动机传感器故障检测[J].传感器与微系统,2017,36(9):147-151.
[13] Salakhutdinov R,Hinton G. Semantic Hashing[J]. International Journal of Approximate Reasoning,2009,50(7):969-978.
[14] Glorot X,Bengio Y. Understanding the difficulty of training deep feedforward neural networks[J]. Journal of Machine Learning Research,2015,9:249-256.