大容量存储测试数据显示预处理方法研究
|
摘要
装甲车辆测试中经常产生大量测试数据用于反映车辆设备或子系统的运行状态,传统车载测试系统在处理大容量数据时存在数据处理耗时较长和不能反映数据整体变化趋势等问题.针对上述问题,在分析传统大容量测试数据处理过程的基础上,提出Max-Min抽样、内存拷贝和多线程并行处理等改进方法.经过实际测试验证,使用改进后的方法在预处理大容量数据时耗时减少达60%以上,程序的运行效率得到极大提高,并且在显示大容量测试数据时能够准确反映数据的整体变化趋势,具有较高的实际应用价值.
In the armored vehicle test, a large number of test data is often produced to reflect the operation state of the vehicle's equipment or system. The traditional vehicle test system has a long time consuming when processing the data and can not reflect the overall trend of data when dealing with large capacity data. On the basis of analyzing the processing of traditional large capacity test data, the improvement method of test data processing method was put forward: Max-Min sampling, the memory copy method, multithreading parallel processing technology and so on. After the actual test and verification, the time of using the improved method to preprocess large capacity data is reduced by 60%, the efficiency of the the program is greatly improved, and it can reflect the overall change trend of the data accurately when display the large capacity test data, it has a high practical application value.
In the armored vehicle test, a large number of test data is often produced to reflect the operation state of the vehicle's equipment or system. The traditional vehicle test system has a long time consuming when processing the data and can not reflect the overall trend of data when dealing with large capacity data. On the basis of analyzing the processing of traditional large capacity test data, the improvement method of test data processing method was put forward: Max-Min sampling, the memory copy method, multithreading parallel processing technology and so on. After the actual test and verification, the time of using the improved method to preprocess large capacity data is reduced by 60%, the efficiency of the the program is greatly improved, and it can reflect the overall change trend of the data accurately when display the large capacity test data, it has a high practical application value.
引文
[1] 孟庆. 基于存储测试的大容量数据处理技术研究[D]. 太原: 中北大学, 2017.
[2] 曹冬杰, 田立言, 肖瑾, 等. 闪电多参量高速大容量实时数据采集、 显示和分析系统[J]. 高原气象, 2011, 30(2): 518-524. Cao Dongjie, Tian Liyan, Xiao Jin, et al. A fast recording, display and waveform analysis system on multi-parameter of lightning flash[J]. Plateau Meteorology, 2011, 30(2): 518-524. (in Chinese)
[3] 马志刚, 刘文怡, 张文栋. 航天测试系统大容量数据处理方法研究[J]. 火力与指挥控制, 2014, 39(10): 74-77. Ma Zhigang, Liu Wenyi, Zhang Wendong. Research on processing method of massive data in space measurement system[J]. Fire Control & Command Control, 2014, 39(10): 74-77. (in Chinese)
[4] 何青, 吴征, 苏淑靖. 基于LabWindows的大容量数据分析处理方法[J]. 计算机测量与控制, 2014, 22(11): 3770-3773. He Qing, Wu Zheng, Su Shujing. Method of bulk data analysising and processing based on LabWindows[J]. Computer Measurement & Control, 2014, 22(11): 3770-3773. (in Chinese)
[5] Stellman A, Greene J. Head First C#[M]. 第3版. 徐阳, 丁小峰, 译. 北京: 中国电力出版社, 2016: 409-414.
[6] Cleary S. C#并发编程经典实例[M]. 第4版. 北京: 人民邮电出版社, 2017: 35-39.
[7] 马志刚, 刘文怡, 张文栋. 航天测试系统通用数据处理软件的设计[J]. 微电子学与计算机, 2014, 31(11): 98-101. Ma Zhigang, Liu Wenyi, Zhang Wendong. Design of universal data processing software in aerospace test system[J]. Microelectronics & Computer, 2014, 31(11): 98-101. (in Chinese)
[8] 姜三义, 代真真, 李阳, 等. 基于内存映射文件的进化算法数据存储引擎[J]. 计算机工程与应用, 2015, 51(1): 49-53. Jiang Sanyi, Dai Zhenzhen, Li Yang, et al. Data storage engine based on memory-mapped file for evolutionary algorithms[J]. Computer Engineering and Applications, 2015, 51(1): 49-53. (in Chinese)
[9] 孙治, 秦小林, 冯勇. 基于多磁盘内存映射的并行湍流相屏生成方法[J]. 四川大学学报(工程科学版), 2014, 46(3): 102-108. Sun Zhi, Qin Xiaolin, Feng Yong. Generation method of atmosphere turbulence phase screen based on parallel computation and multi-disk memory mapping[J]. Journal of Sichuan University (Engineering Science Edition), 2014, 46(3): 102-108. (in Chinese)
[10] 程学旗, 靳小龙, 王元卓, 等. 大数据系统和技术分析[J]. 软件学报, 2014, 25(9): 1889-1908. Cheng Xueqi, Jin Xiaolong, Wang Yuanzhuo, et al. Survey on big data system and analytic technology[J]. Journal of Software, 2014, 25(9): 1889-1908. (in Chinese)
[2] 曹冬杰, 田立言, 肖瑾, 等. 闪电多参量高速大容量实时数据采集、 显示和分析系统[J]. 高原气象, 2011, 30(2): 518-524. Cao Dongjie, Tian Liyan, Xiao Jin, et al. A fast recording, display and waveform analysis system on multi-parameter of lightning flash[J]. Plateau Meteorology, 2011, 30(2): 518-524. (in Chinese)
[3] 马志刚, 刘文怡, 张文栋. 航天测试系统大容量数据处理方法研究[J]. 火力与指挥控制, 2014, 39(10): 74-77. Ma Zhigang, Liu Wenyi, Zhang Wendong. Research on processing method of massive data in space measurement system[J]. Fire Control & Command Control, 2014, 39(10): 74-77. (in Chinese)
[4] 何青, 吴征, 苏淑靖. 基于LabWindows的大容量数据分析处理方法[J]. 计算机测量与控制, 2014, 22(11): 3770-3773. He Qing, Wu Zheng, Su Shujing. Method of bulk data analysising and processing based on LabWindows[J]. Computer Measurement & Control, 2014, 22(11): 3770-3773. (in Chinese)
[5] Stellman A, Greene J. Head First C#[M]. 第3版. 徐阳, 丁小峰, 译. 北京: 中国电力出版社, 2016: 409-414.
[6] Cleary S. C#并发编程经典实例[M]. 第4版. 北京: 人民邮电出版社, 2017: 35-39.
[7] 马志刚, 刘文怡, 张文栋. 航天测试系统通用数据处理软件的设计[J]. 微电子学与计算机, 2014, 31(11): 98-101. Ma Zhigang, Liu Wenyi, Zhang Wendong. Design of universal data processing software in aerospace test system[J]. Microelectronics & Computer, 2014, 31(11): 98-101. (in Chinese)
[8] 姜三义, 代真真, 李阳, 等. 基于内存映射文件的进化算法数据存储引擎[J]. 计算机工程与应用, 2015, 51(1): 49-53. Jiang Sanyi, Dai Zhenzhen, Li Yang, et al. Data storage engine based on memory-mapped file for evolutionary algorithms[J]. Computer Engineering and Applications, 2015, 51(1): 49-53. (in Chinese)
[9] 孙治, 秦小林, 冯勇. 基于多磁盘内存映射的并行湍流相屏生成方法[J]. 四川大学学报(工程科学版), 2014, 46(3): 102-108. Sun Zhi, Qin Xiaolin, Feng Yong. Generation method of atmosphere turbulence phase screen based on parallel computation and multi-disk memory mapping[J]. Journal of Sichuan University (Engineering Science Edition), 2014, 46(3): 102-108. (in Chinese)
[10] 程学旗, 靳小龙, 王元卓, 等. 大数据系统和技术分析[J]. 软件学报, 2014, 25(9): 1889-1908. Cheng Xueqi, Jin Xiaolong, Wang Yuanzhuo, et al. Survey on big data system and analytic technology[J]. Journal of Software, 2014, 25(9): 1889-1908. (in Chinese)