面向卫星遥测数据流的最小稀有模式挖掘方法
|
摘要
模式挖掘是应用于卫星智能监控服务中的一项重要技术.当前频繁模式挖掘的使用率要远远高于稀有模式挖掘,然而对于卫星遥测数据流来说,频繁模式挖掘在安全监测和故障预防等方面所取得的成效不如稀有模式挖掘.因为频繁模式挖掘无法从卫星的遥测数据中揭示卫星可能存在的潜在故障.卫星遥测是持续不断进行的,所以其数据流存在数据量大、传输速度快和数据重复性高的特点.如果采用一般的稀有模式挖掘方法来挖掘卫星数据流,尽管其速度比频繁模式挖掘快,但总体上仍然较慢,不能满足卫星实时监测的需要.针对上述问题,本文提出一种可快速找出卫星遥测数据流中隐藏信息的最小稀有模式挖掘方法,它具有如下优点:(1)无需卫星领域知识;(2)引用滑动窗口技术并将主观参数(窗口尺寸)客观化,使得算法能够实时地处理数据流;(3)通过仅挖掘最小稀有模式方式来提高算法的挖掘效率;(4)该算法使用双向遍历技术提高算法的运行速度.从某在轨卫星的遥测数据流中选取10个关键特征参数进行算法验证.实验结果表明,本文所提算法能有效地从卫星遥测数据流中挖掘出全部的最小稀有模式,并且其挖掘速度比现有的方法快.
The pattern mining is an important technology applied to satellite intelligent monitoring services.At present,the usage ration of frequent pattern mining is much higher than that of the rare pattern mining.However,as far as the research whose target object is satellite telemetry data stream,the results of the frequent pattern mining is not as effective as those of the rare pattern mining in terms of the security monitoring,fault prevention and so on.This is due to the fact that the frequent pattern mining can not reveal the potential faults and hidden dangers that may exist in the components of satellites from satellite telemetry data stream,and it cannot also provide preventive measures.Since telemetry for satellites is continuously carried out,the data stream on satellite telemetry has the following characteristics:in the first place,it includes a large amount of data;in another,its transmission speed is fast,and what is more,its data repetition rate is high.If the common rare pattern mining methods are used to mine satellite telemetry data stream,although its mining speed is faster than the mining speed of the frequent pattern mining,it is still slow on the whole and cannot meet the needs of real-time monitoring for satellite.What is more,the general rare pattern mining mainly takes static data as the object of the research.In order to find out all the rare patterns,the strategy of generating all candidates and then pruning is usually adopted,but which may be leading to the problem of combinatorial explosion and the speed of algorithm is slower.With the development of the big data and the technologies of cloud platform,traditional storage devices have been unable to keep up with the speed of data transmission.Therefore,if static data is still used as the object of the research,the storage device may easily lose data when storing the data.In order to solve the above problems,this paper proposes a minimal rare pattern mining method that can quickly discover the hidden information in the satellite telemetry data stream.It has the following advantages:First,the algorithm proposed in this paper,that is,the minimal rare pattern mining algorithm,does not require knowledge in the satellite field.Second,this paper references the sliding window technology and objectifies the subjective parameters(that is,window size)so that the algorithm proposed in this paper can process satellite telemetry data stream in real time.Third,the algorithm improves the mining efficiency by only mining the minimal rare pattern,instead of all the rare patterns.Fourth,the algorithm uses bidirectional traversal techniques to improve the speed of the algorithm.Ten key features are selected from the satellite telemetry data stream of some orbiting satellite to verify the proposed method in this paper.The experimental results indicate that the algorithm proposed in this paper can effectively mine all the minimal rare patterns from satellite telemetry data stream,and the speed of the proposed algorithm is faster than that of the existing methods.
The pattern mining is an important technology applied to satellite intelligent monitoring services.At present,the usage ration of frequent pattern mining is much higher than that of the rare pattern mining.However,as far as the research whose target object is satellite telemetry data stream,the results of the frequent pattern mining is not as effective as those of the rare pattern mining in terms of the security monitoring,fault prevention and so on.This is due to the fact that the frequent pattern mining can not reveal the potential faults and hidden dangers that may exist in the components of satellites from satellite telemetry data stream,and it cannot also provide preventive measures.Since telemetry for satellites is continuously carried out,the data stream on satellite telemetry has the following characteristics:in the first place,it includes a large amount of data;in another,its transmission speed is fast,and what is more,its data repetition rate is high.If the common rare pattern mining methods are used to mine satellite telemetry data stream,although its mining speed is faster than the mining speed of the frequent pattern mining,it is still slow on the whole and cannot meet the needs of real-time monitoring for satellite.What is more,the general rare pattern mining mainly takes static data as the object of the research.In order to find out all the rare patterns,the strategy of generating all candidates and then pruning is usually adopted,but which may be leading to the problem of combinatorial explosion and the speed of algorithm is slower.With the development of the big data and the technologies of cloud platform,traditional storage devices have been unable to keep up with the speed of data transmission.Therefore,if static data is still used as the object of the research,the storage device may easily lose data when storing the data.In order to solve the above problems,this paper proposes a minimal rare pattern mining method that can quickly discover the hidden information in the satellite telemetry data stream.It has the following advantages:First,the algorithm proposed in this paper,that is,the minimal rare pattern mining algorithm,does not require knowledge in the satellite field.Second,this paper references the sliding window technology and objectifies the subjective parameters(that is,window size)so that the algorithm proposed in this paper can process satellite telemetry data stream in real time.Third,the algorithm improves the mining efficiency by only mining the minimal rare pattern,instead of all the rare patterns.Fourth,the algorithm uses bidirectional traversal techniques to improve the speed of the algorithm.Ten key features are selected from the satellite telemetry data stream of some orbiting satellite to verify the proposed method in this paper.The experimental results indicate that the algorithm proposed in this paper can effectively mine all the minimal rare patterns from satellite telemetry data stream,and the speed of the proposed algorithm is faster than that of the existing methods.
引文
[1]Wen Jing,Wang Da-Ming,Meng Yue-Yue,et al.Application of Beidou navigation satellite system to geological survey.Journal of Geomechanics,2012,18(3):213-223(in Chinese)(温静,汪大明,孟月玥等.北斗卫星导航系统在地质调查领域应用综述.地质力学学报,2012,18(3):213-223)
[2]Gu Sheng,Wei Jiao-Long,Pi De-Chang.Particle swarm optimization-fuzzy support vector machine based prediction of spacecraft parameters.Journal of Astronautics,2014,35(11):1270-1276(in Chinese)(顾胜,魏蛟龙,皮德常.一种粒子群模糊支持向量机的航天器参量预测方法.宇航学报,2014,35(11):1270-1276)
[3]Dai Cheng-Long,Pi De-Chang,Fang Zhen,et al.A longterm lifetime prediction method for hemispherical resonator gyroscope.Journal of Astronautics,2015,36(1):109-116(in Chinese)(代成龙,皮德常,方针等.半球谐振陀螺仪寿命的一种长周期预测方法.宇航学报,2015,36(1):109-116)
[4]Henzinger M R,Raghavan P,Rajagopalan S.Computing on data streams//James M,Vitter J S,et al,eds.External Memory Algorithms.Boston,USA:American Mathematical Society,1998:107-118
[5]Hu Jun-Zhe,Zhou Li-Zhu,Shi Jing.Research of random I/Oschedule of tertiary device in massive storage management.Computer Engineering&Applications,2002,38(12):70-73(in Chinese)(胡俊哲,周立柱,石晶.三级存储设备随机调度研究.计算机工程与应用,2002,38(12):70-73)
[6]Han Meng,Wang Zhi-Hai,Ding Jian.Efficient decision tree for evolving data streams based on frequent patterns.Chinese Journal of Computers,2016,39(8):1541-1554(in Chinese)(韩萌,王志海,丁剑.一种频繁模式决策树处理可变数据流.计算机学报,2016,39(8):1541-1554)
[7]Pang Jingyue,Liu Datong,Peng Yu,et al.Intelligent pattern analysis and anomaly detection of satellite telemetry series with improved time series representation.Journal of Intelligent&Fuzzy Systems,2018,34(6):3785-3798
[8]Ouyang W.Mining rare sequential patterns in data streams with a sliding Window//Proceedings of the International Conference on Systems and Informatics.Hangzhou,China,2017:1023-1027
[9]Han Yanbo,Liu Chen,Su Shen,et al.A proactive service model facilitating stream data fusion and correlation.International Journal of Web Services Research,2017,14(3):1-16
[10]Han Yanbo,Zhu Meiling,Liu Chen.A service-oriented approach to modeling and reusing event correlations//Proceedings of the2018IEEE 42nd Annual Computer Software and Applications Conference.Tokyo,Japan,2018:498-507
[11]Deng Shuiguang,Huang Longtao,Hu Daning,et al.Mobilityenabled service selection for composite services.IEEETransactions on Services Computing,2016,9(3):394-407
[12]Zomaya A,Deng Shuiguang,Huang Longtao,et al.Toward mobile service computing:Opportunities and challenges.IEEE Cloud Computing,2016,3(4):32-41
[13]Han Meng,Wang Zhi-Hai,Yuan Ji-Dong.Efficient method for mining closed frequent patterns from data streams based on time decay model.Journal of Astronautics,2015,38(7):1473-1483(in Chinese)(韩萌,王志海,原继东.一种基于时间衰减模型的数据流闭合模式挖掘方法.计算机学报,2015,38(7):1473-1483)
[14]Wu You-Xi,Zhou Kun,Liu Jing-Yu,et al.Mining sequential patterns with periodic general gap constraints.Journal of Astronautics,2017,40(6):1338-1352(in Chinese)(武优西,周坤,刘靖宇等.周期性一般间隙约束的序列模式挖掘.计算机学报,2017,40(6):1338-1352)
[15]Kang Xu,Pi Dechang.A data-driven method of health monitoring for spacecraft,Aircraft Engineering and Aerospace Technology,2018,90(2):435-451
[16]Lei Yu,Li Man,Hu Wei-Song,et al.Efficient methods for rare sequential pattern mining.Journal of Frontiers of Computer Science and Technology,2015,9(4):429-437(in Chinese)(雷雨,李曼,胡卫松等.高效的稀有序列模式挖掘方法.计算机科学与探索,2015,9(4):429-437)
[17]Hemalatha C S,Vaidehi V,Lakshmi R.Minimal infrequent pattern based approach for mining outliers in data streams.Expert Systems with Applications,2015,42(4):1998-2012
[18]Borah A,Nath B.Mining rare patterns using hyper-linked data structure//Proceedings of the International Conference on Pattern Recognition and Machine Intelligence.Cham,Switzerland,2017:467-472
[19]Bhatt U Y,Patel P A.An effective approach to mine rare items using maximum constraint///Proceedings of the 2015IEEE 9th International Conference on Intelligent Systems and Control.Coimbatore,India,2015:1-6
[20]Chang J H,Lee W S.A sliding window method for finding recently frequent itemsets over online data stream.Journal of Information Science&Engineering,2004,20(4):753-762
[21]Zhang Lu-Min,Jia Yan,Zhou Bin,et al.Online bursty events detection based on emoticons.Chinese Journal of Computers,2013,36(8):1659-1667(in Chinese)(张鲁民,贾焰,周斌等.一种基于情感符号的在线突发事件检测方法.计算机学报,2013,36(8):1659-1667)
[22]Chen Jing.Similarity Measure of Time Series for Satellite Telemetry Data[M.S.dissertation].Harbin Institute of Technology,Harbin,2015(in Chinese)(陈静.卫星遥测数据的时间序列相似性度量方法研究[硕士学位论文].哈尔滨工业大学,哈尔滨,2015)
[23]Li Wei-Zheng,Meng Qiao.Fault detection for in-orbit satellites using an adaptive prediction model.Chinese Journal of Space Science,2014,34(2):201-207(in Chinese)(李维铮,孟桥.基于遥测数据动态特征的卫星异常检测方法.空间科学学报,2014,34(2):201-207)
[24]Yun S K,Rountree N.Finding sporadic rules using AprioriInverse//Ho T B,Cheung D,Liu H,eds.Advances in Knowledge Discovery and Data Mining.Berlin,Germany:Springer-Verlag,2005:97-106
[25]Adda M,Wu L,Feng Y.Rare itemset mining//Proceedings of the International Conference on Machine Learning and Applications.Oregon,USA,2007:73-80
[26]Tsang S,Yun S K,Dobbie G.RP-Tree:Rare pattern tree mining//Proceedings of the International Conference on Data Warehousing and Knowledge Discovery.Toulouse,France,2011:277-288
[27]Huang D,Yun S K,Dobbie G.Rare pattern mining on data streams//Proceedings of the International Conference on Data Warehousing and Knowledge Discovery.Berlin,Germany,2012:303-314
[28]Liu R,Yang K,Sun Y,et al.Spark-based rare association rule mining for big datasets//Proceedings of the IEEE International Conference on Big Data.Boston,USA,2017:2734-2739
[29]Li Guo-Hui,Chen Hui.Mining the frequent patterns in an arbitrary sliding window over online data streams.Journal of Software,2008,19(10):2585-2596(in Chinese)(李国徽,陈辉.挖掘数据流任意滑动时间窗口内频繁模式.软件学报,2008,19(10):2585-2596)
[30]Kang Xu,Pi De-Chang,Tian Hua-Dong.An anomaly detection method based on angle deviation for satellite subsystem.Journal of Astronautics,2017,38(6):638-646(in Chinese)(康旭,皮德常,田华东.一种基于角度偏离的卫星分系统异常检测方法.宇航学报,2017,38(6):638-646)
[31]Dong Xin,Pi Dechang.An effective method for mining quantitative association rules with clustering partition in satellite telemetry data//Proceedings of the 2014 2nd International Conference on Advanced Cloud and Big Data.Huangshan,China,2014:26-33
[32]Bao Jun-Peng,Yang Ke,Zhou Jing.Node level parallel and optimization of satellite data mining.Journal of Beijing University of Aeronautics and Astronautics,2018.https://doi.org/10.13700/j.bh.1001-5965.2018.0334(in Chinese)(鲍军鹏,杨科,周静.卫星数据挖掘节点级并行与优化方法.北京航空航天大学学报,2018.https://doi.org/10.13700/j.bh.1001-5965.2018.0334)
[2]Gu Sheng,Wei Jiao-Long,Pi De-Chang.Particle swarm optimization-fuzzy support vector machine based prediction of spacecraft parameters.Journal of Astronautics,2014,35(11):1270-1276(in Chinese)(顾胜,魏蛟龙,皮德常.一种粒子群模糊支持向量机的航天器参量预测方法.宇航学报,2014,35(11):1270-1276)
[3]Dai Cheng-Long,Pi De-Chang,Fang Zhen,et al.A longterm lifetime prediction method for hemispherical resonator gyroscope.Journal of Astronautics,2015,36(1):109-116(in Chinese)(代成龙,皮德常,方针等.半球谐振陀螺仪寿命的一种长周期预测方法.宇航学报,2015,36(1):109-116)
[4]Henzinger M R,Raghavan P,Rajagopalan S.Computing on data streams//James M,Vitter J S,et al,eds.External Memory Algorithms.Boston,USA:American Mathematical Society,1998:107-118
[5]Hu Jun-Zhe,Zhou Li-Zhu,Shi Jing.Research of random I/Oschedule of tertiary device in massive storage management.Computer Engineering&Applications,2002,38(12):70-73(in Chinese)(胡俊哲,周立柱,石晶.三级存储设备随机调度研究.计算机工程与应用,2002,38(12):70-73)
[6]Han Meng,Wang Zhi-Hai,Ding Jian.Efficient decision tree for evolving data streams based on frequent patterns.Chinese Journal of Computers,2016,39(8):1541-1554(in Chinese)(韩萌,王志海,丁剑.一种频繁模式决策树处理可变数据流.计算机学报,2016,39(8):1541-1554)
[7]Pang Jingyue,Liu Datong,Peng Yu,et al.Intelligent pattern analysis and anomaly detection of satellite telemetry series with improved time series representation.Journal of Intelligent&Fuzzy Systems,2018,34(6):3785-3798
[8]Ouyang W.Mining rare sequential patterns in data streams with a sliding Window//Proceedings of the International Conference on Systems and Informatics.Hangzhou,China,2017:1023-1027
[9]Han Yanbo,Liu Chen,Su Shen,et al.A proactive service model facilitating stream data fusion and correlation.International Journal of Web Services Research,2017,14(3):1-16
[10]Han Yanbo,Zhu Meiling,Liu Chen.A service-oriented approach to modeling and reusing event correlations//Proceedings of the2018IEEE 42nd Annual Computer Software and Applications Conference.Tokyo,Japan,2018:498-507
[11]Deng Shuiguang,Huang Longtao,Hu Daning,et al.Mobilityenabled service selection for composite services.IEEETransactions on Services Computing,2016,9(3):394-407
[12]Zomaya A,Deng Shuiguang,Huang Longtao,et al.Toward mobile service computing:Opportunities and challenges.IEEE Cloud Computing,2016,3(4):32-41
[13]Han Meng,Wang Zhi-Hai,Yuan Ji-Dong.Efficient method for mining closed frequent patterns from data streams based on time decay model.Journal of Astronautics,2015,38(7):1473-1483(in Chinese)(韩萌,王志海,原继东.一种基于时间衰减模型的数据流闭合模式挖掘方法.计算机学报,2015,38(7):1473-1483)
[14]Wu You-Xi,Zhou Kun,Liu Jing-Yu,et al.Mining sequential patterns with periodic general gap constraints.Journal of Astronautics,2017,40(6):1338-1352(in Chinese)(武优西,周坤,刘靖宇等.周期性一般间隙约束的序列模式挖掘.计算机学报,2017,40(6):1338-1352)
[15]Kang Xu,Pi Dechang.A data-driven method of health monitoring for spacecraft,Aircraft Engineering and Aerospace Technology,2018,90(2):435-451
[16]Lei Yu,Li Man,Hu Wei-Song,et al.Efficient methods for rare sequential pattern mining.Journal of Frontiers of Computer Science and Technology,2015,9(4):429-437(in Chinese)(雷雨,李曼,胡卫松等.高效的稀有序列模式挖掘方法.计算机科学与探索,2015,9(4):429-437)
[17]Hemalatha C S,Vaidehi V,Lakshmi R.Minimal infrequent pattern based approach for mining outliers in data streams.Expert Systems with Applications,2015,42(4):1998-2012
[18]Borah A,Nath B.Mining rare patterns using hyper-linked data structure//Proceedings of the International Conference on Pattern Recognition and Machine Intelligence.Cham,Switzerland,2017:467-472
[19]Bhatt U Y,Patel P A.An effective approach to mine rare items using maximum constraint///Proceedings of the 2015IEEE 9th International Conference on Intelligent Systems and Control.Coimbatore,India,2015:1-6
[20]Chang J H,Lee W S.A sliding window method for finding recently frequent itemsets over online data stream.Journal of Information Science&Engineering,2004,20(4):753-762
[21]Zhang Lu-Min,Jia Yan,Zhou Bin,et al.Online bursty events detection based on emoticons.Chinese Journal of Computers,2013,36(8):1659-1667(in Chinese)(张鲁民,贾焰,周斌等.一种基于情感符号的在线突发事件检测方法.计算机学报,2013,36(8):1659-1667)
[22]Chen Jing.Similarity Measure of Time Series for Satellite Telemetry Data[M.S.dissertation].Harbin Institute of Technology,Harbin,2015(in Chinese)(陈静.卫星遥测数据的时间序列相似性度量方法研究[硕士学位论文].哈尔滨工业大学,哈尔滨,2015)
[23]Li Wei-Zheng,Meng Qiao.Fault detection for in-orbit satellites using an adaptive prediction model.Chinese Journal of Space Science,2014,34(2):201-207(in Chinese)(李维铮,孟桥.基于遥测数据动态特征的卫星异常检测方法.空间科学学报,2014,34(2):201-207)
[24]Yun S K,Rountree N.Finding sporadic rules using AprioriInverse//Ho T B,Cheung D,Liu H,eds.Advances in Knowledge Discovery and Data Mining.Berlin,Germany:Springer-Verlag,2005:97-106
[25]Adda M,Wu L,Feng Y.Rare itemset mining//Proceedings of the International Conference on Machine Learning and Applications.Oregon,USA,2007:73-80
[26]Tsang S,Yun S K,Dobbie G.RP-Tree:Rare pattern tree mining//Proceedings of the International Conference on Data Warehousing and Knowledge Discovery.Toulouse,France,2011:277-288
[27]Huang D,Yun S K,Dobbie G.Rare pattern mining on data streams//Proceedings of the International Conference on Data Warehousing and Knowledge Discovery.Berlin,Germany,2012:303-314
[28]Liu R,Yang K,Sun Y,et al.Spark-based rare association rule mining for big datasets//Proceedings of the IEEE International Conference on Big Data.Boston,USA,2017:2734-2739
[29]Li Guo-Hui,Chen Hui.Mining the frequent patterns in an arbitrary sliding window over online data streams.Journal of Software,2008,19(10):2585-2596(in Chinese)(李国徽,陈辉.挖掘数据流任意滑动时间窗口内频繁模式.软件学报,2008,19(10):2585-2596)
[30]Kang Xu,Pi De-Chang,Tian Hua-Dong.An anomaly detection method based on angle deviation for satellite subsystem.Journal of Astronautics,2017,38(6):638-646(in Chinese)(康旭,皮德常,田华东.一种基于角度偏离的卫星分系统异常检测方法.宇航学报,2017,38(6):638-646)
[31]Dong Xin,Pi Dechang.An effective method for mining quantitative association rules with clustering partition in satellite telemetry data//Proceedings of the 2014 2nd International Conference on Advanced Cloud and Big Data.Huangshan,China,2014:26-33
[32]Bao Jun-Peng,Yang Ke,Zhou Jing.Node level parallel and optimization of satellite data mining.Journal of Beijing University of Aeronautics and Astronautics,2018.https://doi.org/10.13700/j.bh.1001-5965.2018.0334(in Chinese)(鲍军鹏,杨科,周静.卫星数据挖掘节点级并行与优化方法.北京航空航天大学学报,2018.https://doi.org/10.13700/j.bh.1001-5965.2018.0334)
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |