基于大数据HDFS改进的心跳模型
|
摘要
分布式文件存储可有效缓解庞大数据对传统文件存储方式以及数据挖掘带来的压力,而心跳机制则是保证分布式文件存储主节点与子节点之间通信、正常运行的关键部分。针对传统心跳机制无法快速判断网络拥堵与宕机两种连接状态的问题,提出了一种子节点互联通信的改进心跳模型,旨在通过多子节点互相监控降低主节点误判率。通过仿真验证,该心跳模型减少了分布式存储系统在网络繁忙情况下对于两种通信状态的误判,提高了数据通信处理速度,同时增强了系统通信的可靠性。
Distributed file storage can effectively relieve the pressure of huge data on traditional file storage and data mining. Heartbeat mechanism is a key part to ensure the communication and operation of distributed file storage between primary and secondary nodes. Aiming at the problem that the traditional heartbeat mechanism can not quickly determine the connection state of network congestion and downtime, an improved heartbeat model of sub node interconnection communication is proposed, which aims to reduce the false positive rate of the master node by monitoring each other with sub nodes. Through the simulation of stand-alone model, this model reduces the misjudgment between the two communication states in distributed storage system to a certain extent, improves the data communication processing speed and enhances the reliability of system communication.
Distributed file storage can effectively relieve the pressure of huge data on traditional file storage and data mining. Heartbeat mechanism is a key part to ensure the communication and operation of distributed file storage between primary and secondary nodes. Aiming at the problem that the traditional heartbeat mechanism can not quickly determine the connection state of network congestion and downtime, an improved heartbeat model of sub node interconnection communication is proposed, which aims to reduce the false positive rate of the master node by monitoring each other with sub nodes. Through the simulation of stand-alone model, this model reduces the misjudgment between the two communication states in distributed storage system to a certain extent, improves the data communication processing speed and enhances the reliability of system communication.
引文
[1]游会迪,张振友.基于Hadoop大数据平台的搭建及其测试研究[J].电脑知识与技术,2017,13(19):211-213.
[2]范素娟,田军锋.基于Hadoop的云计算平台研究与实现[J].计算机技术与发展,2016,26(07):127-132.
[3]黄静,牛鸽.基于心跳机制的农业物联网数据采集系统设计[J].传感器与微系统,2016,35(6):95-97.
[4]何龙,陈晋川.一种面向HDFS的多层索引技术[J].软件学报,2017,28(3):502-513.
[5]王来,翟健宏.基于HDFS的分布式存储策略分析[J].智能计算机与应用,2016,6(01):5-8.
[6]关国栋.基于心跳超时机制的Hadoop实时容错技术[J].计算机应用,2015,35(10):2784-2788.
[7]郑晓薇,项明,张大为,等.基于节点能力的Hadoop集群任务自适应调度方法[J].计算机研究与发展,2014, 51(3):618-626.
[8]吴方超.基于分布式Pi-演算的Name Node单点故障研究与实现[D].西安电子科技大学,2014.)
[9]饶磊.基于分布式Name Node节点的HDFS优化研究[A].第八届全国信号和智能信息处理与应用学术会议会刊[C].2014:5.
[10]司雅楠,阮宁.浅析基于HDFS的分布式Namenode模型[J].电脑知识与技术,2016,12(06):239-241.
[11]易俗,殷慧文,王闯,等.多因素自适应心跳检测算法研究[J].计算机工程与应用,2017,53(24):86-93+128
[12]张海,马建红.基于HDFS的小文件存储与读取优化策略[J].计算机系统应用,2014,23(5):167-171.
[13]鄢锋.一种网络节点通信控制的心跳模型[J].信息控制,2008,37(5):524-528.
[2]范素娟,田军锋.基于Hadoop的云计算平台研究与实现[J].计算机技术与发展,2016,26(07):127-132.
[3]黄静,牛鸽.基于心跳机制的农业物联网数据采集系统设计[J].传感器与微系统,2016,35(6):95-97.
[4]何龙,陈晋川.一种面向HDFS的多层索引技术[J].软件学报,2017,28(3):502-513.
[5]王来,翟健宏.基于HDFS的分布式存储策略分析[J].智能计算机与应用,2016,6(01):5-8.
[6]关国栋.基于心跳超时机制的Hadoop实时容错技术[J].计算机应用,2015,35(10):2784-2788.
[7]郑晓薇,项明,张大为,等.基于节点能力的Hadoop集群任务自适应调度方法[J].计算机研究与发展,2014, 51(3):618-626.
[8]吴方超.基于分布式Pi-演算的Name Node单点故障研究与实现[D].西安电子科技大学,2014.)
[9]饶磊.基于分布式Name Node节点的HDFS优化研究[A].第八届全国信号和智能信息处理与应用学术会议会刊[C].2014:5.
[10]司雅楠,阮宁.浅析基于HDFS的分布式Namenode模型[J].电脑知识与技术,2016,12(06):239-241.
[11]易俗,殷慧文,王闯,等.多因素自适应心跳检测算法研究[J].计算机工程与应用,2017,53(24):86-93+128
[12]张海,马建红.基于HDFS的小文件存储与读取优化策略[J].计算机系统应用,2014,23(5):167-171.
[13]鄢锋.一种网络节点通信控制的心跳模型[J].信息控制,2008,37(5):524-528.