基于混合评价指标的自组织模糊神经网络设计研究
|
摘要
针对在无增长和修剪阈值时模糊神经网络结构难以自适应问题,提出一种基于混合评价指标(hybrid evaluation index, HEI)的结构设计方法。首先,通过模糊C均值聚类算法(fuzzy C-means clustering, FCM)确定初始规则层神经元数目及其中心与宽度。其次,基于戴维森堡丁指数(Davies bouldin index, DBI)和邓恩指数(Dunn index, DI)提出一种新的相关性评价指标(relevance evaluation index, REI)来计算规则层各神经元输出之间的相关性,同时根据训练过程中网络输出均方根误差(root mean square error, RMSE)的变化情况来确定网络的学习能力,然后基于REI和RMSE提出了HEI。通过HEI来调整模糊神经网络的拓扑结构,有效解决了在无增长和修剪阈值时网络结构难以动态自调整的问题且避免了网络结构冗余。最后,通过对Mackey-Glass时间序列预测、非线性系统辨识和大气中PM_(2.5)浓度预测,证明了该结构设计方法的可行性和有效性。
Aiming at the problem that the fuzzy neural network structure is difficult to adapt when there is no growth and pruning thresholds, this paper proposes a structure design method based on hybrid evaluation index(HEI).First, the initial number, centers and widths of rule neurons are determined by the fuzzy C-means clustering algorithm. Next, a novel relevance evaluation index(REI), which is composed of the Davies bouldin index(DBI) and the Dunn index(DI), is presented to calculate the correlation among the outputs of rule neurons. The learning ability of neural network will be determined by the change of root mean square error(RMSE) during the training process.Then, the HEI is presented based on REI and RMSE. The topology structure of the fuzzy neural network is adjusted according to the HEI. Finally, the feasibility and effectiveness of the structure design method are proved by using the Mackey-Glass time series prediction, nonlinear system identification and PM_(2.5) concentration prediction.
Aiming at the problem that the fuzzy neural network structure is difficult to adapt when there is no growth and pruning thresholds, this paper proposes a structure design method based on hybrid evaluation index(HEI).First, the initial number, centers and widths of rule neurons are determined by the fuzzy C-means clustering algorithm. Next, a novel relevance evaluation index(REI), which is composed of the Davies bouldin index(DBI) and the Dunn index(DI), is presented to calculate the correlation among the outputs of rule neurons. The learning ability of neural network will be determined by the change of root mean square error(RMSE) during the training process.Then, the HEI is presented based on REI and RMSE. The topology structure of the fuzzy neural network is adjusted according to the HEI. Finally, the feasibility and effectiveness of the structure design method are proved by using the Mackey-Glass time series prediction, nonlinear system identification and PM_(2.5) concentration prediction.
引文
[1]Luo G Z,Zhang R,Chen Z,et al.A novel nonlinear modeling method for permanent-magnet synchronous motors[J].IEEETransactions Industrial Electronics,2016,63(10):6490-6498.
[2]Wang N,Er M J,Han M.Dynamic tanker steering control using generalized ellipsoidal-basis-function-based fuzzy neural networks[J].IEEE Transactions on Fuzzy Systems,2015,23(5):1414-1427.
[3]Geng J,Huang M L,Li M W,et al.Hybridization of seasonal chaotic cloud simulated annealing algorithm in a SVR-based load forecasting model[J].Neurocomputing,2015,151:1362-1373.
[4]Coyle D,Prasad G,McGinnity T M.Faster self-organizing fuzzy neural network training and a hyperparameter analysis for a brain-computer interface[J].IEEE Transactions on Systems,Man,and Cybernetics,Part B(Cybernetics),2009,39(6):1458-1471.
[5]Zhou Y M,Dexter A.Off-line identification of nonlinear,dynamic systems using a neuro-fuzzy modelling technique[J].Fuzzy Sets and Systems,2013,225:74-92.
[6]Wu S,Er M J,Gao Y.A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks[J].IEEETransactions on Fuzzy Systems,2001,9(4):578-594.
[7]Malek H,Ebadzadeh M M,Rahmati M.Three new fuzzy neural networks learning algorithms based on clustering,training error and genetic algorithm[J].Applied Intelligence,2012,37(2):280-289.
[8]Juang C F,Chen T C,Cheng W Y.Speedup of implementing fuzzy neural networks with high-dimensional inputs through parallel processing on graphic processing units[J].IEEE Transactions on Fuzzy Systems,2011,19(4):717-728.
[9]Wang N,Er M J,Meng X.A fast and accurate online selforganizing scheme for parsimonious fuzzy neural networks[J].Neurocomputing,2009,72(16/17/18):3818-3829.
[10]Ebadzadeh M M,Salimi-Badr A.CFNN:correlated fuzzy neural network[J].Neurocomputing,2015,148:430-444.
[11]Wang N.A generalized ellipsoidal basis function based online self-constructing fuzzy neural network[J].Neural Processing Letters,2011,34(1):13-37.
[12]Prasad M,Lin Y Y,Lin C T,et al.A new data-driven neural fuzzy system with collaborative fuzzy clustering mechanism[J].Neurocomputing,2015,167:558-568.
[13]McDonald S,Kerr D,Coleman S,et al.Modelling retinal ganglion cells using self-organising fuzzy neural networks[C]//Neural Networks(IJCNN),2015 International Joint Conference.IEEE,2015:1-8.
[14]de Jesús Rubio J.SOFMLS:online self-organizing fuzzy modified least-squares network[J].IEEE Transactions on Fuzzy Systems,2009,17(6):1296-1309.
[15]Lin C J,Chen C H.Nonlinear system control using self-evolving neural fuzzy inference networks with reinforcement evolutionary learning[J].Applied Soft Computing,2011,11(8):5463-5476.
[16]Chen C,Wang F Y.A self-organizing neuro-fuzzy network based on first order effect sensitivity analysis[J].Neurocomputing,2013,118(11):21-32.
[17]Han H G,Lin Z L,Qiao J F.Modeling of nonlinear systems using the self-organizing fuzzy neural network with adaptive gradient algorithm[J].Neurocomputing,2017,266:566-578.
[18]乔俊飞,张力,李文静.基于尖峰自组织模糊神经网络的需水量预测[J].控制与决策,2018,33(12):2197-2202.Qiao J F,Zhang L,Li W J.Prediction of water demand based on spiking self-organizing fuzzy neural network[J].Control and Decision,2018,33(12):2197-2202.
[19]Juang C F,Hsieh C D.A fuzzy system constructed by rule generation and iterative linear SVR for antecedent and consequent parameter optimization[J].IEEE Transactions on Fuzzy Systems,2012,20(2):372-384.
[20]Khayat O,Ebadzadeh M M,Shahdoosti H R,et al.A novel hybrid algorithm for creating self-organizing fuzzy neural networks[J].Neurocomputing,2009,73(1/2/3):517-524.
[21]Izakian H,Pedrycz W,Jamal I.Fuzzy clustering of time series data using dynamic time warping distance[J].Engineering Applications of Artificial Intelligence,2015,39:235-244.
[22]Izakian H,Pedrycz W.Agreement-based fuzzy C-means for clustering data with blocks of features[J].Neurocomputing,2014,127:266-280.
[23]Dunn J C.A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters[J].Journal of Cybernetics,1973,3:32-57.
[24]Nasr M B,Chtourou M.A self-organizing map-based initialization for hybrid training of feedforward neural networks[J].Applied Soft Computing,2011,11(8):4458-4464.
[25]Liu Y,Zhou C F,Chen Y W.Weight initialization of feedforward neural networks by means of partial least squares[C]//Machine Learning and Cybernetics,2006 International Conference.IEEE,2006:3119-3122.
[26]Song Q.Robust initialization of a Jordan network with recurrent constrained learning[J].IEEE Transactions on Neural Networks,2011,22(12):2460-2473.
[27]Tala?ka T,Kolasa M,D?ugosz R,et al.An efficient initialization mechanism of neurons for winner takes all neural network implemented in the CMOS technology[J].Applied Mathematics and Computation,2015,267:119-138.
[28]Qiao J F,Li S Y,Li W J.Mutual information based weight initialization method for sigmoidal feedforward neural networks[J].Neurocomputing,2016,207:676-683.
[29]Wu J J,Liu H F,Xiong H,et al.K-means-based consensus clustering:a unified view[J].IEEE Transactions on Knowledge and Data Engineering,2015,27(1):155-169.
[30]Liu Y C,Li Z M,Xiong H,et al.Understanding of internal clustering validation measures[C]//Data Mining(ICDM),2010IEEE 10th International Conference.IEEE,2010:911-916.
[31]Azid A,Juahir H,Toriman M E,et al.Prediction of the level of air pollution using principal component analysis and artificial neural network techniques:a case study in Malaysia[J].Water,Air&Soil Pollution,2014,225(8):2063.
[32]Qiao J F,Han H G.Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach[J].Automatica,2012,48(8):1729-1734.
[33]Islam M M,Sattar M A,Amin M F,et al.A new adaptive merging and growing algorithm for designing artificial neural networks[J].IEEE Transactions on Systems,Man,and Cybernetics,2009,39(3):705-722.
[34]Han H G,Qiao J F.A self-organizing fuzzy neural network based on a growing-and-pruning algorithm[J].IEEE Transactions on Fuzzy Systems,2010,18(6):1129-1143.
[35]Qiao J F,Cai J,Han H G,et al.Predicting PM2.5 concentrations at a regional background station using second order selforganizing fuzzy neural network[J].Atmosphere,2017,8(1):10.
[36]He Z Z,Ye X D,Gu K,et al.Learn to predict PM2.5concentration with image contrast-sensitive features[C]//201837th Chinese Control Conference(CCC).IEEE,2018:4102-4106.
[2]Wang N,Er M J,Han M.Dynamic tanker steering control using generalized ellipsoidal-basis-function-based fuzzy neural networks[J].IEEE Transactions on Fuzzy Systems,2015,23(5):1414-1427.
[3]Geng J,Huang M L,Li M W,et al.Hybridization of seasonal chaotic cloud simulated annealing algorithm in a SVR-based load forecasting model[J].Neurocomputing,2015,151:1362-1373.
[4]Coyle D,Prasad G,McGinnity T M.Faster self-organizing fuzzy neural network training and a hyperparameter analysis for a brain-computer interface[J].IEEE Transactions on Systems,Man,and Cybernetics,Part B(Cybernetics),2009,39(6):1458-1471.
[5]Zhou Y M,Dexter A.Off-line identification of nonlinear,dynamic systems using a neuro-fuzzy modelling technique[J].Fuzzy Sets and Systems,2013,225:74-92.
[6]Wu S,Er M J,Gao Y.A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks[J].IEEETransactions on Fuzzy Systems,2001,9(4):578-594.
[7]Malek H,Ebadzadeh M M,Rahmati M.Three new fuzzy neural networks learning algorithms based on clustering,training error and genetic algorithm[J].Applied Intelligence,2012,37(2):280-289.
[8]Juang C F,Chen T C,Cheng W Y.Speedup of implementing fuzzy neural networks with high-dimensional inputs through parallel processing on graphic processing units[J].IEEE Transactions on Fuzzy Systems,2011,19(4):717-728.
[9]Wang N,Er M J,Meng X.A fast and accurate online selforganizing scheme for parsimonious fuzzy neural networks[J].Neurocomputing,2009,72(16/17/18):3818-3829.
[10]Ebadzadeh M M,Salimi-Badr A.CFNN:correlated fuzzy neural network[J].Neurocomputing,2015,148:430-444.
[11]Wang N.A generalized ellipsoidal basis function based online self-constructing fuzzy neural network[J].Neural Processing Letters,2011,34(1):13-37.
[12]Prasad M,Lin Y Y,Lin C T,et al.A new data-driven neural fuzzy system with collaborative fuzzy clustering mechanism[J].Neurocomputing,2015,167:558-568.
[13]McDonald S,Kerr D,Coleman S,et al.Modelling retinal ganglion cells using self-organising fuzzy neural networks[C]//Neural Networks(IJCNN),2015 International Joint Conference.IEEE,2015:1-8.
[14]de Jesús Rubio J.SOFMLS:online self-organizing fuzzy modified least-squares network[J].IEEE Transactions on Fuzzy Systems,2009,17(6):1296-1309.
[15]Lin C J,Chen C H.Nonlinear system control using self-evolving neural fuzzy inference networks with reinforcement evolutionary learning[J].Applied Soft Computing,2011,11(8):5463-5476.
[16]Chen C,Wang F Y.A self-organizing neuro-fuzzy network based on first order effect sensitivity analysis[J].Neurocomputing,2013,118(11):21-32.
[17]Han H G,Lin Z L,Qiao J F.Modeling of nonlinear systems using the self-organizing fuzzy neural network with adaptive gradient algorithm[J].Neurocomputing,2017,266:566-578.
[18]乔俊飞,张力,李文静.基于尖峰自组织模糊神经网络的需水量预测[J].控制与决策,2018,33(12):2197-2202.Qiao J F,Zhang L,Li W J.Prediction of water demand based on spiking self-organizing fuzzy neural network[J].Control and Decision,2018,33(12):2197-2202.
[19]Juang C F,Hsieh C D.A fuzzy system constructed by rule generation and iterative linear SVR for antecedent and consequent parameter optimization[J].IEEE Transactions on Fuzzy Systems,2012,20(2):372-384.
[20]Khayat O,Ebadzadeh M M,Shahdoosti H R,et al.A novel hybrid algorithm for creating self-organizing fuzzy neural networks[J].Neurocomputing,2009,73(1/2/3):517-524.
[21]Izakian H,Pedrycz W,Jamal I.Fuzzy clustering of time series data using dynamic time warping distance[J].Engineering Applications of Artificial Intelligence,2015,39:235-244.
[22]Izakian H,Pedrycz W.Agreement-based fuzzy C-means for clustering data with blocks of features[J].Neurocomputing,2014,127:266-280.
[23]Dunn J C.A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters[J].Journal of Cybernetics,1973,3:32-57.
[24]Nasr M B,Chtourou M.A self-organizing map-based initialization for hybrid training of feedforward neural networks[J].Applied Soft Computing,2011,11(8):4458-4464.
[25]Liu Y,Zhou C F,Chen Y W.Weight initialization of feedforward neural networks by means of partial least squares[C]//Machine Learning and Cybernetics,2006 International Conference.IEEE,2006:3119-3122.
[26]Song Q.Robust initialization of a Jordan network with recurrent constrained learning[J].IEEE Transactions on Neural Networks,2011,22(12):2460-2473.
[27]Tala?ka T,Kolasa M,D?ugosz R,et al.An efficient initialization mechanism of neurons for winner takes all neural network implemented in the CMOS technology[J].Applied Mathematics and Computation,2015,267:119-138.
[28]Qiao J F,Li S Y,Li W J.Mutual information based weight initialization method for sigmoidal feedforward neural networks[J].Neurocomputing,2016,207:676-683.
[29]Wu J J,Liu H F,Xiong H,et al.K-means-based consensus clustering:a unified view[J].IEEE Transactions on Knowledge and Data Engineering,2015,27(1):155-169.
[30]Liu Y C,Li Z M,Xiong H,et al.Understanding of internal clustering validation measures[C]//Data Mining(ICDM),2010IEEE 10th International Conference.IEEE,2010:911-916.
[31]Azid A,Juahir H,Toriman M E,et al.Prediction of the level of air pollution using principal component analysis and artificial neural network techniques:a case study in Malaysia[J].Water,Air&Soil Pollution,2014,225(8):2063.
[32]Qiao J F,Han H G.Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach[J].Automatica,2012,48(8):1729-1734.
[33]Islam M M,Sattar M A,Amin M F,et al.A new adaptive merging and growing algorithm for designing artificial neural networks[J].IEEE Transactions on Systems,Man,and Cybernetics,2009,39(3):705-722.
[34]Han H G,Qiao J F.A self-organizing fuzzy neural network based on a growing-and-pruning algorithm[J].IEEE Transactions on Fuzzy Systems,2010,18(6):1129-1143.
[35]Qiao J F,Cai J,Han H G,et al.Predicting PM2.5 concentrations at a regional background station using second order selforganizing fuzzy neural network[J].Atmosphere,2017,8(1):10.
[36]He Z Z,Ye X D,Gu K,et al.Learn to predict PM2.5concentration with image contrast-sensitive features[C]//201837th Chinese Control Conference(CCC).IEEE,2018:4102-4106.