基于大数据技术的水情云数据中心设计与研究
|
摘要
随着浙江省水情信息化的不断推进,水情数据种类、结构和数据量不断丰富和发展,传统的单结构化数据存储管理方式已不适合当前信息化社会的应用和服务需求。本研究采用先进的分布式数据采集、智能数据过滤、大数据存储等技术,构建基于大数据云平台技术的浙江省水情云数据中心,对各类分散的多源异构数据进行全面整合。实现了水情大数据的质量管控、深度挖掘和高效共享,丰富了水情信息的资源样本,拓展了服务深度和广度,提升了数据挖掘的服务效率和能力,为水利业务和事务的现代化发展奠定了坚实的基础。
With the rapid advancement of hydrologic informatization in Zhejiang province,the type,structure and volume of hydrological data are continuously enriched and developed.Traditional single-structured data storage and management methods cannot meet the requirements of the information-based society,hence not suitable today.This research adopts advanced technologies such as distributed data acquisition,intelligent data filtering and big data storage to build a hydrologic cloud data center in Zhejiang province.Based on the big data cloud platform technology,the cloud data center integrates various types of distributed multi-source heterogeneous hydrological data.It not only supports quality control,deep mining and efficient sharing of hydrological data,but enriches the resource sample of hydrological information and expands the depth and breadth of hydrological services.With the improved data mining and analysis efficiency,the cloud data center lays a solid foundation for modernization of water conservancy.
With the rapid advancement of hydrologic informatization in Zhejiang province,the type,structure and volume of hydrological data are continuously enriched and developed.Traditional single-structured data storage and management methods cannot meet the requirements of the information-based society,hence not suitable today.This research adopts advanced technologies such as distributed data acquisition,intelligent data filtering and big data storage to build a hydrologic cloud data center in Zhejiang province.Based on the big data cloud platform technology,the cloud data center integrates various types of distributed multi-source heterogeneous hydrological data.It not only supports quality control,deep mining and efficient sharing of hydrological data,but enriches the resource sample of hydrological information and expands the depth and breadth of hydrological services.With the improved data mining and analysis efficiency,the cloud data center lays a solid foundation for modernization of water conservancy.
引文
[1]叶勇,王振宇,范波芹.浙江省小流域山洪灾害临界雨量确定方法分析[J].水文,2008,28(1):56-58.DOI:10.3969/j.issn.1000-0852.2008.01.013YE Y,WANG Z Y,FAN B Q.An analysis method for ascertain critical rainfall of mountain flood disaster of small watershed in Zhejiang province[J].Journal of China Hydrology,2008,28(1):56-58.DOI:10.3969/j.issn.1000-0852.2008.01.013
[2]陈军飞,邓梦华,王慧敏.水利大数据研究综述[J].水科学进展,2017,28(4):622-631.DOI:10.14042/j.cnki.32.1309.2017.04.017CHEN J F,DENG M H,WANG H M.A review of water resources big data[J].Advances in Water Science,2017,28(4):622-631.DOI:10.14042/j.cnki.32.1309.2017.04.017
[3]李国杰,程学旗.大数据研究:未来科技及经济社会发展的重大战略领域--大数据的研究现状与科学思考[J].中国科学院院刊,2012,27(6):647-657.DOI:10.3969/j.issn.1000-3045.2012.06.001LI G J,CHENG X Q.Research status and scientific thinking of big data[J].Bulletin of the Chinese Academy of Sciences,2012,27(6):647-657.DOI:10.3969/j.issn.1000-3045.2012.06.001
[4]夏宁宁.广东省水利数据中心数据架构研究[J].科技视界,2014(20):76-77.DOI:10.19694/j.cnki.issn2095-2457.2014.20.054XIA N N.Research for data architecture of Guangdong provincial data center[J].Science&Technology Vision,2014(20):76-77.DOI:10.19694/j.cnki.issn2095-2457.2014.20.054
[5]杨帆,冯鹏,岳兆新,等.广东省水利数据中心应用效果分析[J].水利信息化,2014(2):20-23.DOI:10.19364/j.1674-9405.2014.02.005YANG F,FENG P,YUE Z X,et al.Analysis of application effects of Guangdong water resources data center[J].Water Resources Informatization,2014(2):20-23.DOI:10.19364/j.1674-9405.2014.02.005
[6]赵晶东.吉林省水文数据库系统设计与开发[J].水利信息化,2014(5):21-25.DOI:10.3969/j.issn.1674-9405.2014.05.006ZHAO J D.Design and development of hydrologic database system of Jilin province[J].Water Resources Informatization,2014(5):21-25.DOI:10.3969/j.issn.1674-9405.2014.05.006
[7]高学萍,于志波,赵伟.基于虚拟化技术的山东黄河数据中心改建探析[J].山东化工,2016,45(17):141-143.DOI:10.19319/j.cnki.issn.1008-021x.2016.17.062GAO X P,YU Z B,ZHAO W.Research on the reconstruction of Shandong the Yellow River data center based on virtualization technology[J].Shandong Chemical Industry,2016,45(17):141-143.DOI:10.19319/j.cnki.issn.1008-021x.2016.17.062
[8]马泽生,徐静保.低碳能效型水利数据中心的规划与建设[J].水利信息化,2011(s1):48-51.DOI:10.19364/j.1674-9405.2011.s1.015MA Z S,XU J B.Planning and construction of low carbon energy efficiency water resources data center[J].Water Resources Informatization,2011(s1):48-51.DOI:10.19364/j.1674-9405.2011.s1.015
[9]吴涵宇,马明,余虹剑.基于GIS的水利数据中心建设研究[J].测绘与空间地理信息,2017,40(s1):193-196.DOI:10.3969/j.issn.1672-5867.2017.z1.054WU H Y,MA M,YU H J.Research on the construction of hydraulic data center based on GIS[J].Geomatics&Spatial Information Technology,2017,40(s1):193-196.DOI:10.3969/j.issn.1672-5867.2017.z1.054
[10]王海峰.基于物联网和虚拟化在宁夏水利数据中心应用系统部署的初探[J].中国电子商务,2014,18(24):26-26.WANG H F.Application system deployment based on the Internet of things and virtualization in the Ningxia water data center[J].China Management Informationization,2014,18(24):26-26.
[11]陈德清,吴礼福,冯径,等.基于网格的水利数据中心数据集成研究[J].水文,2010,30(1):76-78.DOI:10.3969/j.issn.1000-0852.2010.01.018CHEN D Q,WU L F,FENG J,et al.Research on data integration of water resources data center based grid technology[J].Hydrology,2010,30(1):76-78.DOI:10.3969/j.issn.1000-0852.2010.01.018
[12]胡金龙,柏屏,朱海波,等.基于数据中心的数据交互及应用服务技术研究[J].水利信息化,2014(2)5-8.DOI:10.19364/j.1674-9405.2014.02.002HU J L,BAI P,ZHU H B,et al.Technology research of data exchange and application service based on data center[J].Water Resources Informatization,2014(2):5-8.DOI:10.19364/j.1674-9405.2014.02.002
[13]龚琪慧,刘伟,李坤,等.基于大数据的水利数据中心建设[C]//中国水利信息化与数字水利技术论坛论文集.北京:北京沃特咨询有限公司,2015:6.GONG Q H,LIU W,LI K,et al.Construction o water data center based on big data[C]//China WRI&Hydrotechnics.Beijing:Beijing Water Consulting Co.Ltd.,2015:6.
[14]杨楚骅.基于SOA的水利地理信息数据管理平台的设计与实现[D].广州:中山大学,2015.YANG C H.Design and Implementation on the Geographic Information Data Based on SOA[D].Guangzhou:Sun Yat-sen University,2015.
[15]SHAFIEE M E,BARKER Z,RASEKH A.Enhancing water system models by integrating big data[J].Sustainable Cities&Society,2018,37:485-491.DOI:10.1016/j.scs.2017.11.042
[16]HU Y,GARCIA-CABREJO O,CAI X,et al.Global sensitivity analysis for large-scale sociohydrological models using Hadoop[J].Environmental Modelling&Software,2015,73:231-243.DOI:10.1016/j.envsoft.2015.08.015
[17]BERG D J,CASSIDY A S,DEBOLE M V,et al.Streaming Programmable Point Mapper and Compute Hardware:US20170155698[P].2017-06-01.
[18]刘陈,李冰.基于阿里云在计算产业的应用研究[J].科学与信息化,2016(34):52-53.LIU C,LI B.Research on the application of Aliyun in computing industry[J].Science and Informatization,2016(34):52-53.
[2]陈军飞,邓梦华,王慧敏.水利大数据研究综述[J].水科学进展,2017,28(4):622-631.DOI:10.14042/j.cnki.32.1309.2017.04.017CHEN J F,DENG M H,WANG H M.A review of water resources big data[J].Advances in Water Science,2017,28(4):622-631.DOI:10.14042/j.cnki.32.1309.2017.04.017
[3]李国杰,程学旗.大数据研究:未来科技及经济社会发展的重大战略领域--大数据的研究现状与科学思考[J].中国科学院院刊,2012,27(6):647-657.DOI:10.3969/j.issn.1000-3045.2012.06.001LI G J,CHENG X Q.Research status and scientific thinking of big data[J].Bulletin of the Chinese Academy of Sciences,2012,27(6):647-657.DOI:10.3969/j.issn.1000-3045.2012.06.001
[4]夏宁宁.广东省水利数据中心数据架构研究[J].科技视界,2014(20):76-77.DOI:10.19694/j.cnki.issn2095-2457.2014.20.054XIA N N.Research for data architecture of Guangdong provincial data center[J].Science&Technology Vision,2014(20):76-77.DOI:10.19694/j.cnki.issn2095-2457.2014.20.054
[5]杨帆,冯鹏,岳兆新,等.广东省水利数据中心应用效果分析[J].水利信息化,2014(2):20-23.DOI:10.19364/j.1674-9405.2014.02.005YANG F,FENG P,YUE Z X,et al.Analysis of application effects of Guangdong water resources data center[J].Water Resources Informatization,2014(2):20-23.DOI:10.19364/j.1674-9405.2014.02.005
[6]赵晶东.吉林省水文数据库系统设计与开发[J].水利信息化,2014(5):21-25.DOI:10.3969/j.issn.1674-9405.2014.05.006ZHAO J D.Design and development of hydrologic database system of Jilin province[J].Water Resources Informatization,2014(5):21-25.DOI:10.3969/j.issn.1674-9405.2014.05.006
[7]高学萍,于志波,赵伟.基于虚拟化技术的山东黄河数据中心改建探析[J].山东化工,2016,45(17):141-143.DOI:10.19319/j.cnki.issn.1008-021x.2016.17.062GAO X P,YU Z B,ZHAO W.Research on the reconstruction of Shandong the Yellow River data center based on virtualization technology[J].Shandong Chemical Industry,2016,45(17):141-143.DOI:10.19319/j.cnki.issn.1008-021x.2016.17.062
[8]马泽生,徐静保.低碳能效型水利数据中心的规划与建设[J].水利信息化,2011(s1):48-51.DOI:10.19364/j.1674-9405.2011.s1.015MA Z S,XU J B.Planning and construction of low carbon energy efficiency water resources data center[J].Water Resources Informatization,2011(s1):48-51.DOI:10.19364/j.1674-9405.2011.s1.015
[9]吴涵宇,马明,余虹剑.基于GIS的水利数据中心建设研究[J].测绘与空间地理信息,2017,40(s1):193-196.DOI:10.3969/j.issn.1672-5867.2017.z1.054WU H Y,MA M,YU H J.Research on the construction of hydraulic data center based on GIS[J].Geomatics&Spatial Information Technology,2017,40(s1):193-196.DOI:10.3969/j.issn.1672-5867.2017.z1.054
[10]王海峰.基于物联网和虚拟化在宁夏水利数据中心应用系统部署的初探[J].中国电子商务,2014,18(24):26-26.WANG H F.Application system deployment based on the Internet of things and virtualization in the Ningxia water data center[J].China Management Informationization,2014,18(24):26-26.
[11]陈德清,吴礼福,冯径,等.基于网格的水利数据中心数据集成研究[J].水文,2010,30(1):76-78.DOI:10.3969/j.issn.1000-0852.2010.01.018CHEN D Q,WU L F,FENG J,et al.Research on data integration of water resources data center based grid technology[J].Hydrology,2010,30(1):76-78.DOI:10.3969/j.issn.1000-0852.2010.01.018
[12]胡金龙,柏屏,朱海波,等.基于数据中心的数据交互及应用服务技术研究[J].水利信息化,2014(2)5-8.DOI:10.19364/j.1674-9405.2014.02.002HU J L,BAI P,ZHU H B,et al.Technology research of data exchange and application service based on data center[J].Water Resources Informatization,2014(2):5-8.DOI:10.19364/j.1674-9405.2014.02.002
[13]龚琪慧,刘伟,李坤,等.基于大数据的水利数据中心建设[C]//中国水利信息化与数字水利技术论坛论文集.北京:北京沃特咨询有限公司,2015:6.GONG Q H,LIU W,LI K,et al.Construction o water data center based on big data[C]//China WRI&Hydrotechnics.Beijing:Beijing Water Consulting Co.Ltd.,2015:6.
[14]杨楚骅.基于SOA的水利地理信息数据管理平台的设计与实现[D].广州:中山大学,2015.YANG C H.Design and Implementation on the Geographic Information Data Based on SOA[D].Guangzhou:Sun Yat-sen University,2015.
[15]SHAFIEE M E,BARKER Z,RASEKH A.Enhancing water system models by integrating big data[J].Sustainable Cities&Society,2018,37:485-491.DOI:10.1016/j.scs.2017.11.042
[16]HU Y,GARCIA-CABREJO O,CAI X,et al.Global sensitivity analysis for large-scale sociohydrological models using Hadoop[J].Environmental Modelling&Software,2015,73:231-243.DOI:10.1016/j.envsoft.2015.08.015
[17]BERG D J,CASSIDY A S,DEBOLE M V,et al.Streaming Programmable Point Mapper and Compute Hardware:US20170155698[P].2017-06-01.
[18]刘陈,李冰.基于阿里云在计算产业的应用研究[J].科学与信息化,2016(34):52-53.LIU C,LI B.Research on the application of Aliyun in computing industry[J].Science and Informatization,2016(34):52-53.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |