基于RBNB和MapReduce的海量结构工程数据处理与分析
|
摘要
目前我国土木工程事故频发,造成了重大的人员伤亡和财产损失。对于长期在役的重要结构,如果能够定期通过结构健康监测对其累积损伤的程度做出正确评估,就可以充分掌握结构的工作状态,确保结构的安全。然而结构健康监测往往需要持续较长的时间,短时间的监测数据就往往达到GB级别,因此数据的规模相当大。采用RBNB对海量结构工程数据进行缓存,同时通过MapReduce模型对这些数据进行并行的处理分析,大大缩短了海量数据的处理时间,提高了数据处理分析的实时性。
At present,there are frequent accidents in civil engineering sector,which results in significant casualties and property losses.Therefore,for those important structures of long-term in-service,if we can make a correct assessment on the degree of their accumulated damages periodically by the structural health surveillance,we may fully grip the working state of the structures to ensure their safety.However,the structure health surveillance usually requires longer period of time,and the monitored data in a short period will reach an amount at GB level,this means the data size is quite big.In this article we use RBNB as the real-time data cache for massive data of structural engineering and meanwhile use MapReduce model to process and analysis these data in parallel mode,which greatly shortens the processing time on massive data and improves the real-time property of data processing and analyses.
At present,there are frequent accidents in civil engineering sector,which results in significant casualties and property losses.Therefore,for those important structures of long-term in-service,if we can make a correct assessment on the degree of their accumulated damages periodically by the structural health surveillance,we may fully grip the working state of the structures to ensure their safety.However,the structure health surveillance usually requires longer period of time,and the monitored data in a short period will reach an amount at GB level,this means the data size is quite big.In this article we use RBNB as the real-time data cache for massive data of structural engineering and meanwhile use MapReduce model to process and analysis these data in parallel mode,which greatly shortens the processing time on massive data and improves the real-time property of data processing and analyses.
引文
[1]朱绍玮,张宇峰,樊可清,等.江阴大桥结构健康监测系统升级改造及初步数据分析[J].公路,2007(4):69-73.
[2]结构工程[EB/OL].2011-05-19.http://zh.wikipedia.org/wiki/结构工程学.
[3]龙浩,梁毅,邸瑞华,等.地震网格数据采集与遥现系统设计[J].计算机工程与设计,2009(2).
[4]Yang H C,Dasdan A,Hsiao R L,et al.Ap-Reducemerge:Simplified re-lational data processing on large clusters[C]//SIGMOD,2007.
[5]Hadoop[OL].2011.http://hadoop.apache.org.
[6]季虎,夏胜平,等.快速傅里叶变化算法概述[J].现代电子技术,2001(8):11-14.
[7]陈国良,安虹,等.并行算法实践[M].北京:高等教育出版社,2004.
[8]陆鑫达.并行和分布计算技术现状及发展策略[N].计算机世界报,1997(38).
[2]结构工程[EB/OL].2011-05-19.http://zh.wikipedia.org/wiki/结构工程学.
[3]龙浩,梁毅,邸瑞华,等.地震网格数据采集与遥现系统设计[J].计算机工程与设计,2009(2).
[4]Yang H C,Dasdan A,Hsiao R L,et al.Ap-Reducemerge:Simplified re-lational data processing on large clusters[C]//SIGMOD,2007.
[5]Hadoop[OL].2011.http://hadoop.apache.org.
[6]季虎,夏胜平,等.快速傅里叶变化算法概述[J].现代电子技术,2001(8):11-14.
[7]陈国良,安虹,等.并行算法实践[M].北京:高等教育出版社,2004.
[8]陆鑫达.并行和分布计算技术现状及发展策略[N].计算机世界报,1997(38).
版权所有：© 2023 中国地质图书馆 中国地质调查局地学文献中心