MD-RBM神经网络模型及其在材料微结构中聚类研究
|
摘要
在传统的RBM神经网络的基础上提出一种新颖的MD-RBM神经网络模型用于超高碳钢微结构高维图像数据的特征学习。该模型利用新的乘法距离(MD)取代欧式距离以计算高维图像数据之间的距离关系,有效缓解欧式距离在高维数据中的不稳定性问题。MD-RBM神经网络模型利用少量的成对约束监督信息引导其编码过程,使得一部分图像数据的隐藏层特征更加聚集在一起,而且同时使得一部分图像数据的隐藏层特征更加分散,由此得到高维图像数据的隐藏层特征表现出很好的聚类性能。实验选择两种经典聚类算法Affinity Propagation(AP)和Spectral Clustering(SC)作为对比,结果显示,基于MD-RBM模型的聚类识别算法比原始聚类算法、半监督算法以及基于RBM模型的聚类算法都表现出更优的聚类性能。
This paper proposed a novel MD-RBM neural network model based on traditional RBM for feature learning of high-dimensional ultra-high carbon steel(UHCS) microstructure images data. This model used a new multiplicative distance(MD) to replace the Euclidean distance to calculate the distance relationship between high-dimensional image data, and effectively relieved the instability of Euclidean distance in high-dimensional data. The MD-RBM neural network model used a small amount of pairwise constraint supervisory information to guide its coding process, which made the hidden layer features of some image data closer and the other part more remote. Therefore, the hidden layer features of high-dimensional image data showed better clustering performance. In the experiment, we chose two classical clustering algorithms Affinity Propagation(AP) and Spectral Clustering(SC) as comparison. The results show that the algorithm based on MD-RBM model has better clustering performance than the original clustering algorithm, semi-supervised algorithm and clustering algorithm based on RBM model.
This paper proposed a novel MD-RBM neural network model based on traditional RBM for feature learning of high-dimensional ultra-high carbon steel(UHCS) microstructure images data. This model used a new multiplicative distance(MD) to replace the Euclidean distance to calculate the distance relationship between high-dimensional image data, and effectively relieved the instability of Euclidean distance in high-dimensional data. The MD-RBM neural network model used a small amount of pairwise constraint supervisory information to guide its coding process, which made the hidden layer features of some image data closer and the other part more remote. Therefore, the hidden layer features of high-dimensional image data showed better clustering performance. In the experiment, we chose two classical clustering algorithms Affinity Propagation(AP) and Spectral Clustering(SC) as comparison. The results show that the algorithm based on MD-RBM model has better clustering performance than the original clustering algorithm, semi-supervised algorithm and clustering algorithm based on RBM model.
引文
[1] Azimi S M,Britz D,Engstler M,et al.Advanced Steel Microstructural Classification by Deep Learning Methods[J].Scientific Reports,2018,8:2128.
[2] Gola J,Britz D,Staudt T,et al.Advanced microstructure classification by data mining methods[J].Computational Materials Science,2018,148:324-335.
[3] Hinton G E.Training products of experts by minimizing contrastive divergence[J].Neural Compute,2002,14(8):1771-1800.
[4] Sarikaya R,Hinton G E,Deoras A.Application of Deep Belief Networks for Natural Language Understanding[J].IEEE/ACM Transactions on Audio Speech & Language Processing,2014,22(4):778-784.
[5] Yuan M,Tang H,Li H.Real-time keypoint recognition using restricted Boltzmann machine[J].IEEE Transactions on Neural Networks & Learning Systems,2014,25(11):2119-2126.
[6] Bengio Y,Courville A,Vincent P.Representation Learning:A Review and New Perspectives[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(8):1798-1828.
[7] Nie S,Wang Z,Ji Q.A Generative Restricted Boltzmann Machine Based Method for High-Dimensional Motion Data Modeling[J].Computer Vision & Image Understanding,2015,136(C):14-22.
[8] Teh Y W,Hinton G E.Rate-coded restricted Boltzmann machines for face recognition[C]// International Conference on Neural Information Processing Systems.MIT Press,2000:872-878.
[9] Fink O,Zio E,Weidmann U.Fuzzy Classification With Restricted Boltzman Machines and Echo-State Networks for Predicting Potential Railway Door System Failures[J].IEEE Transactions on Reliability,2015,64(3):861-868.
[10] Chen Y,Zhao X,Jia X.Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing,2017,8(6):2381-2392.
[11] Chen Y,Zhao X,Jia X.Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing,2017,8(6):2381-2392.
[12] Kuremoto T,Kimura S,Kobayashi K,et al.Time Series Forecasting Using Restricted Boltzmann Machine[M]// Emerging Intelligent Computing Technology and Applications.Springer Berlin Heidelberg,2012:17-22.
[13] Hirata T,Kuremoto T,Obayashi M,et al.Time Series Prediction Using DBN and ARIMA[C]// International Conference on Computer Application Technologies.IEEE,2016:24-29.
[14] Nakashika T,Takiguchi T,Ariki Y.Voice conversion using RNN pre-trained by recurrent temporal restricted boltzmann machines[J].IEEE/ACM Transactions on Audio Speech & Language Processing,2015,23(3):580-587.
[15] Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[16] Chu J,Wang H,Meng H,et al.Restricted Boltzmann Machines with Gaussian Visible Units Guided by Pairwise Constraints[J].IEEE Transactions on Cybernetics(Early Access),2018.doi:10.1109/TCYB.2018.286360.
[17] Zhang D,Chen S,Zhou Z H,et al.Constraint Projections for Ensemble Learning.[C]// AAAI Conference on Artificial Intelligence,AAAI 2008,Chicago,Illinois,USA,July.DBLP,2008:758-763.
[18] Mansouri J,Khademi M.Multiplicative distance:a method to alleviate distance instability for high-dimensional data[J].Knowledge & Information Systems,2015,45(3):783-805.
[19] Frey B J,Dueck D.Clustering by passing messages between data points[J].Science,2007,315(5814):972-976.
[20] Ng A Y,Jordan M I,Weiss Y.On Spectral Clustering:Analysis and an Algorithm[C]// Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic.MIT Press,2001:849-856.
[21] Xiao Y,Yu J.Semi-Supervised Clustering Based on Affinity Propagation Algorithm[J].Journal of Software,2008,19(11):2803-2813.
[22] Rangapuram S S,Hein M.Constrained 1-Spectral Clustering[EB].arXiv:1505.06485,2015.
[23] Cai D,He X,Han J.Document Clustering Using Locality Preserving Indexing[J].IEEE Transactions on Knowledge & Data Engineering,2005,17(12):1624-1637.
[24] Ding C,Li T,Peng W,et al.Orthogonal nonnegative matrix t-factorizations for clustering[C]// Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining,2006:126-135.
[25] Campello R J G B.A fuzzy extension of the Rand index and other related indexes for clustering and classification assessment[J].Pattern Recognition Letters,2007,28(7):833-841.
[2] Gola J,Britz D,Staudt T,et al.Advanced microstructure classification by data mining methods[J].Computational Materials Science,2018,148:324-335.
[3] Hinton G E.Training products of experts by minimizing contrastive divergence[J].Neural Compute,2002,14(8):1771-1800.
[4] Sarikaya R,Hinton G E,Deoras A.Application of Deep Belief Networks for Natural Language Understanding[J].IEEE/ACM Transactions on Audio Speech & Language Processing,2014,22(4):778-784.
[5] Yuan M,Tang H,Li H.Real-time keypoint recognition using restricted Boltzmann machine[J].IEEE Transactions on Neural Networks & Learning Systems,2014,25(11):2119-2126.
[6] Bengio Y,Courville A,Vincent P.Representation Learning:A Review and New Perspectives[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(8):1798-1828.
[7] Nie S,Wang Z,Ji Q.A Generative Restricted Boltzmann Machine Based Method for High-Dimensional Motion Data Modeling[J].Computer Vision & Image Understanding,2015,136(C):14-22.
[8] Teh Y W,Hinton G E.Rate-coded restricted Boltzmann machines for face recognition[C]// International Conference on Neural Information Processing Systems.MIT Press,2000:872-878.
[9] Fink O,Zio E,Weidmann U.Fuzzy Classification With Restricted Boltzman Machines and Echo-State Networks for Predicting Potential Railway Door System Failures[J].IEEE Transactions on Reliability,2015,64(3):861-868.
[10] Chen Y,Zhao X,Jia X.Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing,2017,8(6):2381-2392.
[11] Chen Y,Zhao X,Jia X.Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network[J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing,2017,8(6):2381-2392.
[12] Kuremoto T,Kimura S,Kobayashi K,et al.Time Series Forecasting Using Restricted Boltzmann Machine[M]// Emerging Intelligent Computing Technology and Applications.Springer Berlin Heidelberg,2012:17-22.
[13] Hirata T,Kuremoto T,Obayashi M,et al.Time Series Prediction Using DBN and ARIMA[C]// International Conference on Computer Application Technologies.IEEE,2016:24-29.
[14] Nakashika T,Takiguchi T,Ariki Y.Voice conversion using RNN pre-trained by recurrent temporal restricted boltzmann machines[J].IEEE/ACM Transactions on Audio Speech & Language Processing,2015,23(3):580-587.
[15] Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[16] Chu J,Wang H,Meng H,et al.Restricted Boltzmann Machines with Gaussian Visible Units Guided by Pairwise Constraints[J].IEEE Transactions on Cybernetics(Early Access),2018.doi:10.1109/TCYB.2018.286360.
[17] Zhang D,Chen S,Zhou Z H,et al.Constraint Projections for Ensemble Learning.[C]// AAAI Conference on Artificial Intelligence,AAAI 2008,Chicago,Illinois,USA,July.DBLP,2008:758-763.
[18] Mansouri J,Khademi M.Multiplicative distance:a method to alleviate distance instability for high-dimensional data[J].Knowledge & Information Systems,2015,45(3):783-805.
[19] Frey B J,Dueck D.Clustering by passing messages between data points[J].Science,2007,315(5814):972-976.
[20] Ng A Y,Jordan M I,Weiss Y.On Spectral Clustering:Analysis and an Algorithm[C]// Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic.MIT Press,2001:849-856.
[21] Xiao Y,Yu J.Semi-Supervised Clustering Based on Affinity Propagation Algorithm[J].Journal of Software,2008,19(11):2803-2813.
[22] Rangapuram S S,Hein M.Constrained 1-Spectral Clustering[EB].arXiv:1505.06485,2015.
[23] Cai D,He X,Han J.Document Clustering Using Locality Preserving Indexing[J].IEEE Transactions on Knowledge & Data Engineering,2005,17(12):1624-1637.
[24] Ding C,Li T,Peng W,et al.Orthogonal nonnegative matrix t-factorizations for clustering[C]// Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining,2006:126-135.
[25] Campello R J G B.A fuzzy extension of the Rand index and other related indexes for clustering and classification assessment[J].Pattern Recognition Letters,2007,28(7):833-841.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |