基于网络流量日志的分析与安全审计
|
摘要
随着Internet的普及，由于Internet自身的开放性、自由性，在互联网上发布、传播有害信息的问题日渐突出，利用互联网实施的违法犯罪活动也逐渐增多。怎样更好地去监管人们的上网行为，是现代网管面临的一个问题。
GS网关日志系统的应用，使实名制上网成为可能。这也给安全审计功能提供了基础。论文主要基于GS系统的网络流量日志，通过文本日志的查询、日志统计、基于规则的自动报警、基于数据挖掘的用户行为分析，来尽可能实现完善的安全审计，营造一个干净的上网环境。
论文主要分六部分。第一部分概述了相关技术和背景知识，介绍了GS系统、Linux平台、Java语言和Oracle9i数据库：第二部分着重介绍了数据挖掘技术，对多种模式及其算法进行比较分析，及其挖掘软件Weka的介绍，为后面用户行为分析奠定了理论基础：第三部分讲述了整个系统的设计，包括日志查询、日志统计、自动报警、数据挖掘：第四部分详细描述了日志查询和日志统计的实现；第五部分详细描述了自动报警和数据挖掘的实现，是论文重点；第五部分是系统总结，评价本系统的优势及待改进的地方。
With the development of Internet, because of the opening-up and freedom of Internet, more and more problems grow up,including broadcasting harmful information,crimines by Internet. How to control and supervise peoples' behavior in Internet is faced by modern network management.
With the usage of GS (Gateway System), it's possible to using real name in Internet. It's also the fundation of Security Audit. The paper implements Security Audit based on Network Flow log ,by text log query,log statistics,auto alerting by Alert Rules and user's behavior analysis, to create a clean Internet.
The paper is made up with 6 parts. The 1st part introduces the background knowlege and related technology, also including GS, Linux platform,Java language and Oracle 9i database. The 2nd part introduces data mining technology emphatically, analyses several modes,compares their alogrithm and gives a brief of Weka sofware. The 3rd part describes the whole system design,including log query.log statistics,auto alerting,data mining and so on. The 4th describes how to implement log query and log statistics in detail.The 5th part describes how to implement auto alerting and data mining in detail. The last part is the summary of the paper,mainly evaluating the advantage and shortage.
GS网关日志系统的应用，使实名制上网成为可能。这也给安全审计功能提供了基础。论文主要基于GS系统的网络流量日志，通过文本日志的查询、日志统计、基于规则的自动报警、基于数据挖掘的用户行为分析，来尽可能实现完善的安全审计，营造一个干净的上网环境。
论文主要分六部分。第一部分概述了相关技术和背景知识，介绍了GS系统、Linux平台、Java语言和Oracle9i数据库：第二部分着重介绍了数据挖掘技术，对多种模式及其算法进行比较分析，及其挖掘软件Weka的介绍，为后面用户行为分析奠定了理论基础：第三部分讲述了整个系统的设计，包括日志查询、日志统计、自动报警、数据挖掘：第四部分详细描述了日志查询和日志统计的实现；第五部分详细描述了自动报警和数据挖掘的实现，是论文重点；第五部分是系统总结，评价本系统的优势及待改进的地方。
With the development of Internet, because of the opening-up and freedom of Internet, more and more problems grow up,including broadcasting harmful information,crimines by Internet. How to control and supervise peoples' behavior in Internet is faced by modern network management.
With the usage of GS (Gateway System), it's possible to using real name in Internet. It's also the fundation of Security Audit. The paper implements Security Audit based on Network Flow log ,by text log query,log statistics,auto alerting by Alert Rules and user's behavior analysis, to create a clean Internet.
The paper is made up with 6 parts. The 1st part introduces the background knowlege and related technology, also including GS, Linux platform,Java language and Oracle 9i database. The 2nd part introduces data mining technology emphatically, analyses several modes,compares their alogrithm and gives a brief of Weka sofware. The 3rd part describes the whole system design,including log query.log statistics,auto alerting,data mining and so on. The 4th describes how to implement log query and log statistics in detail.The 5th part describes how to implement auto alerting and data mining in detail. The last part is the summary of the paper,mainly evaluating the advantage and shortage.
引文
[1] Kevin Loney Marlene Theriault, Oracle9i DBA 手册，王磊、蒋蕊，2002，机工业出版社
[2] 飞思科技产品研发中心，Oracle数据仓库构建技术，2003，电子工业出版社
[3] Jiawei Han Micheline Kamber,数据挖掘概念与技术，范明、孟小峰，2001机械工业出版社
[4] David Hand Heikki Mannila,数据挖掘原理，张银奎、廖丽、宋俊，2003，机械工业出版社、中信出版社
[5] R. Agrawai, T. Imielinski, and A. Swami, Mining association rules between sets of items in large databases,, Proceedings of the ACM SIGMOD Conference on Management of data, 1993, 207-216
[6] 程继华、施鹏飞，多层次关联规则的有效挖掘算法，软件学报，1998 Vol．9，No．12：937-941
[7] 朱扬勇、周欣、施伯乐，规则型数据采掘工具集AMINE，高技术通讯，2000，Vol．10，No．3：19-22
[8] 陈栋、徐洁，Knight：一个通用知识挖掘工具，计算机研究与发展，1998，Vol．35，No．4：338-343
[9] 周欣、沙朝锋、朱扬勇、施伯乐，兴趣度—关联规则的又一个阈值．计算机研究与发展，2000 Vol．37，No．5：627-633
[10] S. Brin, R. Motwani, J.D.Ullman, and S. Tsur, Dynamic Itemset counting and implication rules for market basket data, In ACM SIGMOD International Conference On the Management of Data, 1997
[11] 萨师煊、王珊，数据库系统概论，第三版，2000，高等教育出版社
[12] 林宇，数据仓库原理与实践，2003，人民邮电出版社
[13] Wendy Boggs Michael Boggs, UML 与Rational Rose 2002从入门到精通，邱仲潘，2002，电子工业出版社
[14] 顾晓刚、谢义军、林锐，高质量Java程序设计，2003，电子工业出版社
[15] Sun Microsystems, Inc. Java2 Platform Enterprise Edition Specification[R], v1.4 , 2002, Public Draft
[16] Anne Tbooms, Patricia Scybold Group. Enterprise JavaBeans Technology, 2000, http://java.sun.com/products/ejb/white paper.html
[17] Mike Jasnowski, Java XML和Web服务宝典，盖江南、王勇，2002，电子工业版社
[18] Feghhi J,怎样用JavaBeans开发WEB，陈强璋，1998，机械工业出版社
[19] Bruce Eckel, Java编程思想，候捷，第二版，2002，机械工业出版社
[20] Borland公司， Borland Jbuilder使用技术手册，2003，电子工业出版社
[21] 网冠科技，JSP时尚编程百例，2001，机械工业出版社
[22] 刘波，JSP程序设计精彩实例，2002，清华大学出版社
[23] R.Agrawal,and J. Shafer,Parallel mining of association rules:Design,Implementation, and Experience,Technical Report FJ10004,IBM Almaden Research Center, San Jose, CA 95120, Jan. 1996.
[24] S. Brin, R. Motwani, and C. Silverstein, Beyond market baskets:generlizing association rules to correlations, Proceedings of the ACM SIGMOD, 1996,255-276
[25] J. Kleinberg, C. Papadimitriou, and P. Raghavan, Segmentation problems., Proceedings of the 30th Annual Symposium on Theory of Computing, ACM. 1998
[26] Rakesh Agrawal, et al. The quest data mining system. In: Proc. of KDD, Portland, Oregon, 1996:244-249
[27] Jiawei Han, et al., DBMiner: a system for mining knowledge in large relational databases, In: Proc. of KDD, Portland, Oregon,1996.8:250-255
[28] Jiawei Han Yongjian Fu Wei Wang Krzysztof Koperski, Osmar Zaiane. "DMQL:A Data Mining Query Language for Relational Database", VLDB'96
[29] Jian Hu, Database Connectivity and Interoperability in the Internet, Roslin Institute Edinburgh, 1997
[2] 飞思科技产品研发中心，Oracle数据仓库构建技术，2003，电子工业出版社
[3] Jiawei Han Micheline Kamber,数据挖掘概念与技术，范明、孟小峰，2001机械工业出版社
[4] David Hand Heikki Mannila,数据挖掘原理，张银奎、廖丽、宋俊，2003，机械工业出版社、中信出版社
[5] R. Agrawai, T. Imielinski, and A. Swami, Mining association rules between sets of items in large databases,, Proceedings of the ACM SIGMOD Conference on Management of data, 1993, 207-216
[6] 程继华、施鹏飞，多层次关联规则的有效挖掘算法，软件学报，1998 Vol．9，No．12：937-941
[7] 朱扬勇、周欣、施伯乐，规则型数据采掘工具集AMINE，高技术通讯，2000，Vol．10，No．3：19-22
[8] 陈栋、徐洁，Knight：一个通用知识挖掘工具，计算机研究与发展，1998，Vol．35，No．4：338-343
[9] 周欣、沙朝锋、朱扬勇、施伯乐，兴趣度—关联规则的又一个阈值．计算机研究与发展，2000 Vol．37，No．5：627-633
[10] S. Brin, R. Motwani, J.D.Ullman, and S. Tsur, Dynamic Itemset counting and implication rules for market basket data, In ACM SIGMOD International Conference On the Management of Data, 1997
[11] 萨师煊、王珊，数据库系统概论，第三版，2000，高等教育出版社
[12] 林宇，数据仓库原理与实践，2003，人民邮电出版社
[13] Wendy Boggs Michael Boggs, UML 与Rational Rose 2002从入门到精通，邱仲潘，2002，电子工业出版社
[14] 顾晓刚、谢义军、林锐，高质量Java程序设计，2003，电子工业出版社
[15] Sun Microsystems, Inc. Java2 Platform Enterprise Edition Specification[R], v1.4 , 2002, Public Draft
[16] Anne Tbooms, Patricia Scybold Group. Enterprise JavaBeans Technology, 2000, http://java.sun.com/products/ejb/white paper.html
[17] Mike Jasnowski, Java XML和Web服务宝典，盖江南、王勇，2002，电子工业版社
[18] Feghhi J,怎样用JavaBeans开发WEB，陈强璋，1998，机械工业出版社
[19] Bruce Eckel, Java编程思想，候捷，第二版，2002，机械工业出版社
[20] Borland公司， Borland Jbuilder使用技术手册，2003，电子工业出版社
[21] 网冠科技，JSP时尚编程百例，2001，机械工业出版社
[22] 刘波，JSP程序设计精彩实例，2002，清华大学出版社
[23] R.Agrawal,and J. Shafer,Parallel mining of association rules:Design,Implementation, and Experience,Technical Report FJ10004,IBM Almaden Research Center, San Jose, CA 95120, Jan. 1996.
[24] S. Brin, R. Motwani, and C. Silverstein, Beyond market baskets:generlizing association rules to correlations, Proceedings of the ACM SIGMOD, 1996,255-276
[25] J. Kleinberg, C. Papadimitriou, and P. Raghavan, Segmentation problems., Proceedings of the 30th Annual Symposium on Theory of Computing, ACM. 1998
[26] Rakesh Agrawal, et al. The quest data mining system. In: Proc. of KDD, Portland, Oregon, 1996:244-249
[27] Jiawei Han, et al., DBMiner: a system for mining knowledge in large relational databases, In: Proc. of KDD, Portland, Oregon,1996.8:250-255
[28] Jiawei Han Yongjian Fu Wei Wang Krzysztof Koperski, Osmar Zaiane. "DMQL:A Data Mining Query Language for Relational Database", VLDB'96
[29] Jian Hu, Database Connectivity and Interoperability in the Internet, Roslin Institute Edinburgh, 1997