基于动态任务代理技术的IP网络数据采集框架研究及实现
|
摘要
随着IP网络规模和复杂性不断增加,各种各样的设备和接口技术层出不穷,对网络进行有效管理,保证网络的运行质量,变得越来越困难。通过有效的采集方法把IP网络管理数据可靠、及时的获取上来,是对IP网络进行管理的基础。因此,实时性高、扩展性强的IP网络管理数据采集机制的研究一直是网络管理领域的一个重要问题。
针对现有数据采集方式存在的问题,本文提出了基于动态任务代理的数据采集方式,并给出了基于动态任务代理的数据采集框架,使之符合采集灵活、采集内容更加广泛并提供多样化的采集策略的要求,并对以下关键问题进行了研究并给出了解决方案:
1)抽象采集策略:将不同类型、不同厂家、不同操作系统平台等的设备性能数据查看命令抽象成采集策略。
2)采集策略适配:将采集任务到采集策略的映射抽象成适配规则,并以配置文件形式进行存储,实现了业务逻辑和过程的分离。
3)传输连接方式:在总体设计思想里,都是以连接终端的方式连接到目标设备,然后执行采集策略。为此,我们找到了两种适应的连接方式:Telnet和SSH连接方式,前一种一般情况下采用明文进行网络数据传输,网络数据的安全性存在隐患,后一种则以加密的方式进行数据传输,解决了网络数据传输的安全问题。
4)采集结果解析:对于采集结果的解析,我们借用了问题2中的规则方式。对于不同厂家、不同类型、操作系统等设备的返回文件必须采用不同的解析规则,解析规则由正则表达式进行描述。
在对上述关键问题研究并给出解决方案的基础上,基于Java技术对本文提出的解决方案进行了实现,并将这种采集方式成功应用到了某大型IP网络综合管理平台项目中,取得了良好的实际效果。
Along with the increasing scale and complicacy of the network, and various of devices and interface technologies, it's harder to manage the network effectively and keep the running quality of the network .It's the base of the IP network management to get the IP network management data reliably and timely through effective collecting methods. So, the research of data collection which is highly timely and strongly expansible is always an important problem in the network management area.
As for the problems in the existing collecting methods, this paper puts forward the data collecting way based on the dynamic task agent, and gives out the structure of this method to make it accord with the demands of flexible collecting, abroad collecting content, and provide varies of collecting strategies, also this paper does some research on some key problems and gives us the solutions to them.
1) Abstract collecting strategy: abstract a serial of commands which are for performance data-acquisition to a collecting strategy.
2) The adapting of the collecting strategy: abstract the corresponding process from a collecting task to a collecting strategy to an adapting regular, and save it in the configuration files, this way divides the operation logic and the process.
3) The way of transformation and link: we use the way of linking terminal to link to the destination device as a whole, and then execute the collecting strategy. We find two linking ways which can fulfill the demands: Telnet and SSH ways, the first way uses the proclaimed in writing way for network data transformation, it has hidden trouble in the security of network data. The second way uses encrypted data transformation, it figures out the security problem of network data transformation.
4) Collecting result parsing: as for the collecting result parsing, we use the way used in problem 2. We define different parsing regulars for different returned files which belong to different device types, different manufactures, and different operation systems, the parsing regulars are described by regular expressions.
On the base of the research and the solution given of the key problems listed above, this paper gives out the Java implement of the dynamic task agent data collecting method, and has successfully applied this implement in a large IP network integrated management platform project, and obtains favorable impacts.
针对现有数据采集方式存在的问题,本文提出了基于动态任务代理的数据采集方式,并给出了基于动态任务代理的数据采集框架,使之符合采集灵活、采集内容更加广泛并提供多样化的采集策略的要求,并对以下关键问题进行了研究并给出了解决方案:
1)抽象采集策略:将不同类型、不同厂家、不同操作系统平台等的设备性能数据查看命令抽象成采集策略。
2)采集策略适配:将采集任务到采集策略的映射抽象成适配规则,并以配置文件形式进行存储,实现了业务逻辑和过程的分离。
3)传输连接方式:在总体设计思想里,都是以连接终端的方式连接到目标设备,然后执行采集策略。为此,我们找到了两种适应的连接方式:Telnet和SSH连接方式,前一种一般情况下采用明文进行网络数据传输,网络数据的安全性存在隐患,后一种则以加密的方式进行数据传输,解决了网络数据传输的安全问题。
4)采集结果解析:对于采集结果的解析,我们借用了问题2中的规则方式。对于不同厂家、不同类型、操作系统等设备的返回文件必须采用不同的解析规则,解析规则由正则表达式进行描述。
在对上述关键问题研究并给出解决方案的基础上,基于Java技术对本文提出的解决方案进行了实现,并将这种采集方式成功应用到了某大型IP网络综合管理平台项目中,取得了良好的实际效果。
Along with the increasing scale and complicacy of the network, and various of devices and interface technologies, it's harder to manage the network effectively and keep the running quality of the network .It's the base of the IP network management to get the IP network management data reliably and timely through effective collecting methods. So, the research of data collection which is highly timely and strongly expansible is always an important problem in the network management area.
As for the problems in the existing collecting methods, this paper puts forward the data collecting way based on the dynamic task agent, and gives out the structure of this method to make it accord with the demands of flexible collecting, abroad collecting content, and provide varies of collecting strategies, also this paper does some research on some key problems and gives us the solutions to them.
1) Abstract collecting strategy: abstract a serial of commands which are for performance data-acquisition to a collecting strategy.
2) The adapting of the collecting strategy: abstract the corresponding process from a collecting task to a collecting strategy to an adapting regular, and save it in the configuration files, this way divides the operation logic and the process.
3) The way of transformation and link: we use the way of linking terminal to link to the destination device as a whole, and then execute the collecting strategy. We find two linking ways which can fulfill the demands: Telnet and SSH ways, the first way uses the proclaimed in writing way for network data transformation, it has hidden trouble in the security of network data. The second way uses encrypted data transformation, it figures out the security problem of network data transformation.
4) Collecting result parsing: as for the collecting result parsing, we use the way used in problem 2. We define different parsing regulars for different returned files which belong to different device types, different manufactures, and different operation systems, the parsing regulars are described by regular expressions.
On the base of the research and the solution given of the key problems listed above, this paper gives out the Java implement of the dynamic task agent data collecting method, and has successfully applied this implement in a large IP network integrated management platform project, and obtains favorable impacts.
引文
[1]王继龙.计算机网络管理系统发展综述.计算机科学 2000Vol27No.9 52-55页
[2]荣国辉.网络管理发展综述.世界电信.2003(1＼2)51-56页
[3]严宣辉、龚家骧.利用SNMP实现网络性能的实时监控.计算机应用与软件2004年第3期67-68页
[4]Sean Harnedy.简单网络管理协议教程(第2版).胡谷雨等译.电子工业出版社 1999年15-21页
[5]任建基、胡延平、陈俊峰.基于WMI技术的局域网计算机设备的监测.计算机工程与应用 2006年25期134-146页
[6]王荣波、周昌乐.移动Agent研究综述.计算机应用研究2001年第6期9-11页
[7]黄鹏宇、杨家玮.基于移动Agent技术的网络管理框架结构.计算机仿真.2005年第22卷第4期139-141页
[8]马旭、杨彬.异构网络网管数据采集问题的研究与应用.网络安全技术与应用 2006年5月44-45页
[9]冯博.移动通信综合网管系统性能数据采集设计与实现.中国优秀博硕士学位论文全文数据库 2007年第33卷第1期15-21页
[10]包铁、刘淑芬.基于规则的网络数据采集处理方法.计算机工程.2007年1月101-103页
[11]丰超.基于Telnet的数据采集及基于文件的处理规则定.中国优秀博硕士学位论文全文数据库 2006年4月17-19页
[12]Postel J、Reynolds J.Telnet Protocol Specification,RFC854,USC Information Sciences Institute[EB/OL].[1983-05-20].ftp://ftp.rfc-editor.org/in-notes /rfc854.txt.
[13]Postel J、Reynolds J.Telnet Protocol Specification,RFC855,USC Information Sciences Institute[EB/OL].[1983-05-25].ftp://ftp.rfc-editor.org/in-notes /rfc855.txt.
[14]http://blog.sina.com.cn/s/blog_4baf2e83010009h6.html
[15]http://www.cnpaf.net/ 中国协议分析网
[16]http://www.ibm.com/developerworks/cn/java/wa-springl/
[17]http://www.chinaitlab.com/www/techspecial/spring/
[18]http://www.ibm.com/developerworks/cn/java/j-quartz/
[19]http://www.opensymphony.com/quartz/
[20]http://www.itpub.net
[2]荣国辉.网络管理发展综述.世界电信.2003(1＼2)51-56页
[3]严宣辉、龚家骧.利用SNMP实现网络性能的实时监控.计算机应用与软件2004年第3期67-68页
[4]Sean Harnedy.简单网络管理协议教程(第2版).胡谷雨等译.电子工业出版社 1999年15-21页
[5]任建基、胡延平、陈俊峰.基于WMI技术的局域网计算机设备的监测.计算机工程与应用 2006年25期134-146页
[6]王荣波、周昌乐.移动Agent研究综述.计算机应用研究2001年第6期9-11页
[7]黄鹏宇、杨家玮.基于移动Agent技术的网络管理框架结构.计算机仿真.2005年第22卷第4期139-141页
[8]马旭、杨彬.异构网络网管数据采集问题的研究与应用.网络安全技术与应用 2006年5月44-45页
[9]冯博.移动通信综合网管系统性能数据采集设计与实现.中国优秀博硕士学位论文全文数据库 2007年第33卷第1期15-21页
[10]包铁、刘淑芬.基于规则的网络数据采集处理方法.计算机工程.2007年1月101-103页
[11]丰超.基于Telnet的数据采集及基于文件的处理规则定.中国优秀博硕士学位论文全文数据库 2006年4月17-19页
[12]Postel J、Reynolds J.Telnet Protocol Specification,RFC854,USC Information Sciences Institute[EB/OL].[1983-05-20].ftp://ftp.rfc-editor.org/in-notes /rfc854.txt.
[13]Postel J、Reynolds J.Telnet Protocol Specification,RFC855,USC Information Sciences Institute[EB/OL].[1983-05-25].ftp://ftp.rfc-editor.org/in-notes /rfc855.txt.
[14]http://blog.sina.com.cn/s/blog_4baf2e83010009h6.html
[15]http://www.cnpaf.net/ 中国协议分析网
[16]http://www.ibm.com/developerworks/cn/java/wa-springl/
[17]http://www.chinaitlab.com/www/techspecial/spring/
[18]http://www.ibm.com/developerworks/cn/java/j-quartz/
[19]http://www.opensymphony.com/quartz/
[20]http://www.itpub.net
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |