分布式网络多层次数据挖掘改进方法研究
|
摘要
传统数据挖掘方法对数据挖掘时必须为高速通信网络,而且还导致系统响应时间延长,对数据安全性产生威胁。文中以分布式环境为背景,提出基于熵值思想的聚类挖掘改进方法,实现网络多层次数据挖掘。设定网络多层次数据聚类参数,计算产生新聚类数,将该数据值作为聚类搜索范围的上限值kmax,选取合适的有效性Silhouette指标,结合最大最小距离理论设置的聚类中心,获得最佳聚类数目;运用熵值理论及动态规划思想形成改进聚类挖掘方法,运用熵值理论判定数据属性权重值,并获取多层次数据对象与邻近数据间的权重关系,将欧氏距离当作数据相似度衡量依据;利用动态规划思想计算获得最大k个数据对象,确定多层次数据挖掘聚类中心。实验证明,利用文中改进数据挖掘方法可有效挖掘网络多层次数据中的有价值信息。
Traditional data mining methods must be high-speed communication networks for data mining,but also lead to longer response time and threat to data security. Based on the distributed environment,this paper proposes an improved clustering mining method based on entropy value to realize multi-level data mining. According to the network data of multi level clustering parameters set in advance,produce new clustering number by calculating the data value as clustering the search range of the upper limit of kmax,select the appropriate indicators of the effectiveness of Silhouette,with the maximum and minimum distance clustering center set theory,obtain the optimal number of clusters; using the entropy theory and dynamic programming form improvement clustering mining method,determine the data value of attribute weight by entropy theory,and obtain the weight hierarchy data object and the adjacent data between the Euclidean distance as a similarity measure based on the maximum data; k data object is calculated by using the dynamic programming to determine the multi-level data mining clustering center. Experimental results show that the improved data mining method can effectively mine valuable information in multi-level data.
Traditional data mining methods must be high-speed communication networks for data mining,but also lead to longer response time and threat to data security. Based on the distributed environment,this paper proposes an improved clustering mining method based on entropy value to realize multi-level data mining. According to the network data of multi level clustering parameters set in advance,produce new clustering number by calculating the data value as clustering the search range of the upper limit of kmax,select the appropriate indicators of the effectiveness of Silhouette,with the maximum and minimum distance clustering center set theory,obtain the optimal number of clusters; using the entropy theory and dynamic programming form improvement clustering mining method,determine the data value of attribute weight by entropy theory,and obtain the weight hierarchy data object and the adjacent data between the Euclidean distance as a similarity measure based on the maximum data; k data object is calculated by using the dynamic programming to determine the multi-level data mining clustering center. Experimental results show that the improved data mining method can effectively mine valuable information in multi-level data.
引文
[1]邓芳.大型物联网电子设备的海量数据高效挖掘方法研究[J].现代电子技术,2016,39(4):159-162.
[2]孔翔宇,毕秀春,张曙光.财经新闻与股市预测——基于数据挖掘技术的实证分析[J].数理统计与管理,2016,35(2):215-224
[3]刘应波,王锋,季凯帆,等.基于压缩-字对齐位图的天文海量数据实时索引[J].计算机工程与应用,2016,52(1):37-41.
[4]滕翠,梁川,梁碧珍.海量数据环境下大型ISP网络流量卸载方法研究[J].科学技术与工程,2017,17(13):44-48.
[5]李远宁,刘森,张诗军,等.分布式数据质量管理系统在电力企业的实践和应用[J].电信科学,2016,32(4):169-174.
[6]逄琳,刘方爱.基于层次划分的密度优化聚类算法[J].计算机应用,2016,36(6):1634-1638.
[7]韩萌,王志海,丁剑.一种频繁模式决策树处理可变数据流[J].计算机学报,2016,39(8):1541-1554.
[8]张春生.大数据环境下相容数据集的关联规则数据挖掘[J].微电子学与计算机,2016,33(8):34-39.
[2]孔翔宇,毕秀春,张曙光.财经新闻与股市预测——基于数据挖掘技术的实证分析[J].数理统计与管理,2016,35(2):215-224
[3]刘应波,王锋,季凯帆,等.基于压缩-字对齐位图的天文海量数据实时索引[J].计算机工程与应用,2016,52(1):37-41.
[4]滕翠,梁川,梁碧珍.海量数据环境下大型ISP网络流量卸载方法研究[J].科学技术与工程,2017,17(13):44-48.
[5]李远宁,刘森,张诗军,等.分布式数据质量管理系统在电力企业的实践和应用[J].电信科学,2016,32(4):169-174.
[6]逄琳,刘方爱.基于层次划分的密度优化聚类算法[J].计算机应用,2016,36(6):1634-1638.
[7]韩萌,王志海,丁剑.一种频繁模式决策树处理可变数据流[J].计算机学报,2016,39(8):1541-1554.
[8]张春生.大数据环境下相容数据集的关联规则数据挖掘[J].微电子学与计算机,2016,33(8):34-39.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |