基于Floyd算法的海温时间序列分割
|
摘要
时间序列数据的处理及挖掘一直是业界关注的热点,而海表温度也一直是人们观测、研究和预报的重要对象。本文主要考虑对一年的跨度进行切割,使得落在每个切割区间的海表温度数据满足最优的正态分布,以便对遥感数据的异常性作出检验。结合2003-2011年南海和东海海表温度数据集,本文引入Floyd算法,将寻求数据集最优分割问题转化为图论中网络中最短路求解问题,将不超过30天的点之间的距离设定为无穷大,以避免分割点过于密集的情况,并将频率与概率的距离定义的误差转化为线路权重,实现了动态全局最优分割。且正态分布下的3σ异常值检验法,实现了对异常值的识别。
Sea surface temperature(SST) time series data processing and mining has been the focus of the industry, and the SST has also been an important object of the observation, research and forecasting.This paper mainly considers SST time series segmentation based on the span of one year, to make the interval in each of the SST data to meet the optimal normal distribution, and make it convenient to test abnormity of the remote sensing data. Based on the SST data set of China East Sea and South Sea in 2003-2011, this paper introduces the Floyd algorithm, and the optimal segmentation problem is transformed into graph network for solving shortest path problems. In order to avoid the segmentation points is too dense, the distance between two points which not exceed 30 days is set to infinity, and the error between probability and frequency distance is defined as line weights, then achieved dynamic global optimal segmentation. And the 3σ outlier test method under normal distribution achieves the recognition of outliers.
Sea surface temperature(SST) time series data processing and mining has been the focus of the industry, and the SST has also been an important object of the observation, research and forecasting.This paper mainly considers SST time series segmentation based on the span of one year, to make the interval in each of the SST data to meet the optimal normal distribution, and make it convenient to test abnormity of the remote sensing data. Based on the SST data set of China East Sea and South Sea in 2003-2011, this paper introduces the Floyd algorithm, and the optimal segmentation problem is transformed into graph network for solving shortest path problems. In order to avoid the segmentation points is too dense, the distance between two points which not exceed 30 days is set to infinity, and the error between probability and frequency distance is defined as line weights, then achieved dynamic global optimal segmentation. And the 3σ outlier test method under normal distribution achieves the recognition of outliers.
引文
[1] Keogh E, Kasetty S. On the need for time series data mining benchmarks:A survey and empirical demonstration[J]. Data Mining&Knowledge Discovery, 2003, 7(4):349-371.
[2]王达.时间序列数据挖掘研究与应用[D].浙江大学博士学位论文,2004.
[3]黄超,朱扬勇.基于ARMA模型的联机时间序列数据分割算法[J].模式识别与人工智能,2005, 18(2):130-134.
[4]李爱国,覃征.在线分割时间序列数据[J].软件学报,2004, 15(11):1671-1679.
[5]覃征,李爱国.时间序列数据的稳健最优分割方法[J].西安交通大学学报,2003, 37(4):338-342.
[6]廖俊,周中良,寇英信,等.一种基于重要点的时间序列分割方法[J].计算机工程与应用,2011, 47(24):166-170.
[7]江艺羡,张岐山.基于重要点与灰色GM(1,1)模型的时间序列分段算法[J].统计与决策,2016,(24):28-30.
[8]王立柱,刘晓东.基于信息颗粒和模糊聚类的时间序列分割[J].模糊系统与数学,2015, 29(1):175-182.
[9]崔世杰,于重重,苏维均,等.海量实时序列数据指数平滑预测分割算法[J].计算机应用研究,2016, 33(9):2712-2715.
[10] Kehagias A. A hidden Markov model segmentation procedure for hydrological and environmental time series[J]. Stochastic Environmental Research&Risk Assessment, 2004, 18(2):117-130.
[11] Kehagias A, Nidelkou E, Petridis V. A dynamic programming segmentation procedure for hydrological and environmental time series[J]. Stochastic Environmental Research&Risk Assessment, 2006,20(1-2):77-94.
[12] Wang N, Xia J, Yin J, et al. Trend analysis of land surface temperatures using time series segmentation algorithm[J]. Journal of Intelligent&Fuzzy Systems, 2016, 31(2):1121-1131.
[13] Guo H, Liu X, Song L. Dynamic programming approach for segmentation of multivariate time series[J]. Stochastic Environmental Research&Risk Assessment, 2015, 29(1):265-273.
[14] Chung F L, Fu T C, Luk R, et al. Evolutionary Time Series Segmentation for Stock Data Mining[A]. IEEE International Conference on Data Mining[C]. IEEE Computer Society, 2002.
[15] Yin Y, Shang P, Xia J. Compositional segmentation of time series in the financial markets[J]. Applied Mathematics&Computation, 2015, 268(C):399-412.
[16]李勇,孙瑞博,王贵银.厚尾金融时间序列的贝叶斯单位根检验[J].数理统计与管理,2012, 31(1):184-190.
[17]孙慧慧,林金官.基于M估计的纵向数据线性混合模型中方差的齐性检验[J].数理统计与管理,2013,32(4):646-657.
[18]王志坚,王斌会.稳健改进的AO型异常点检测法在金融时序中的应用[J].数理统计与管理,2016, 35(2):369-380.
[19]王志坚.稳健残差控制图的构建及在金融时序中的应用[J].数理统计与管理,2017, 36(5):930-942.
[20]李洪波,王茂波.Floyd最短路径算法的动态优化[J].计算机工程与应用,2006, 42(34):60-63.
[21]王荣,江东,韩惠.基于Floyd方法的最短路径算法优化算法[J].甘肃科学学报,2012, 24(4):110-114.
[22] Ridi L, Torrini J, Vicario E. Developing a scheduler with difference-bound matrices and the FloydWarshall algorithm[J]. IEEE Software, 2012, 29(1):76-83.
[23]张建方,王秀祥.直方图理论与最优直方图制作[J].应用概率统计,2009, 25(2):201-214.
[24] Wang X X, Zhang J F. Histogram-kernel error and its application for bin width selection in histograms[J]. Acta Mathematicae Applicatae Sinica(English Series), 2012, 28(3):607-624.
[2]王达.时间序列数据挖掘研究与应用[D].浙江大学博士学位论文,2004.
[3]黄超,朱扬勇.基于ARMA模型的联机时间序列数据分割算法[J].模式识别与人工智能,2005, 18(2):130-134.
[4]李爱国,覃征.在线分割时间序列数据[J].软件学报,2004, 15(11):1671-1679.
[5]覃征,李爱国.时间序列数据的稳健最优分割方法[J].西安交通大学学报,2003, 37(4):338-342.
[6]廖俊,周中良,寇英信,等.一种基于重要点的时间序列分割方法[J].计算机工程与应用,2011, 47(24):166-170.
[7]江艺羡,张岐山.基于重要点与灰色GM(1,1)模型的时间序列分段算法[J].统计与决策,2016,(24):28-30.
[8]王立柱,刘晓东.基于信息颗粒和模糊聚类的时间序列分割[J].模糊系统与数学,2015, 29(1):175-182.
[9]崔世杰,于重重,苏维均,等.海量实时序列数据指数平滑预测分割算法[J].计算机应用研究,2016, 33(9):2712-2715.
[10] Kehagias A. A hidden Markov model segmentation procedure for hydrological and environmental time series[J]. Stochastic Environmental Research&Risk Assessment, 2004, 18(2):117-130.
[11] Kehagias A, Nidelkou E, Petridis V. A dynamic programming segmentation procedure for hydrological and environmental time series[J]. Stochastic Environmental Research&Risk Assessment, 2006,20(1-2):77-94.
[12] Wang N, Xia J, Yin J, et al. Trend analysis of land surface temperatures using time series segmentation algorithm[J]. Journal of Intelligent&Fuzzy Systems, 2016, 31(2):1121-1131.
[13] Guo H, Liu X, Song L. Dynamic programming approach for segmentation of multivariate time series[J]. Stochastic Environmental Research&Risk Assessment, 2015, 29(1):265-273.
[14] Chung F L, Fu T C, Luk R, et al. Evolutionary Time Series Segmentation for Stock Data Mining[A]. IEEE International Conference on Data Mining[C]. IEEE Computer Society, 2002.
[15] Yin Y, Shang P, Xia J. Compositional segmentation of time series in the financial markets[J]. Applied Mathematics&Computation, 2015, 268(C):399-412.
[16]李勇,孙瑞博,王贵银.厚尾金融时间序列的贝叶斯单位根检验[J].数理统计与管理,2012, 31(1):184-190.
[17]孙慧慧,林金官.基于M估计的纵向数据线性混合模型中方差的齐性检验[J].数理统计与管理,2013,32(4):646-657.
[18]王志坚,王斌会.稳健改进的AO型异常点检测法在金融时序中的应用[J].数理统计与管理,2016, 35(2):369-380.
[19]王志坚.稳健残差控制图的构建及在金融时序中的应用[J].数理统计与管理,2017, 36(5):930-942.
[20]李洪波,王茂波.Floyd最短路径算法的动态优化[J].计算机工程与应用,2006, 42(34):60-63.
[21]王荣,江东,韩惠.基于Floyd方法的最短路径算法优化算法[J].甘肃科学学报,2012, 24(4):110-114.
[22] Ridi L, Torrini J, Vicario E. Developing a scheduler with difference-bound matrices and the FloydWarshall algorithm[J]. IEEE Software, 2012, 29(1):76-83.
[23]张建方,王秀祥.直方图理论与最优直方图制作[J].应用概率统计,2009, 25(2):201-214.
[24] Wang X X, Zhang J F. Histogram-kernel error and its application for bin width selection in histograms[J]. Acta Mathematicae Applicatae Sinica(English Series), 2012, 28(3):607-624.