HEVC压缩域的视频摘要关键帧提取方法
|
摘要
为了提高关键帧提取的准确率,改善视频摘要的质量,提出了一种HEVC压缩域的视频摘要关键帧提取方法。首先,对视频序列进行编解码,在解码中统计HEVC帧内编码PU块的亮度预测模式数目。然后,特征提取是利用统计得到的模式数目构建成模式特征向量,并将其作为视频帧的纹理特征用于关键帧的提取。最后,利用融合迭代自组织数据分析算法(ISODATA)的自适应聚类算法对模式特征向量进行聚类,在聚类结果中选取每个类内中间向量对应的帧作为候选关键帧,并通过相似度对候选关键帧进行再次筛选,剔除冗余帧,得到最终的关键帧。实验结果表明,在Open Video Project数据集上进行的大量实验验证,该方法提取关键帧的精度为79.9%、召回率达到93.6%、F-score为86.2%,有效地改善了视频摘要的质量。
In order to improve the accuracy of key frame extraction, and improve the quality of video summaries, a video summarization key frame extraction method for HEVC compressed domain is proposed. Firstly, the video sequence is coded and decoded, and the number of luminance prediction modes of the HEVC intra-coded PU block is counted in the decoding. Secondly, the feature extraction is constructed by using the number of patterns obtained by statistics as a pattern feature vector and used as a texture feature of the video frame for key frame extraction. Finally, the pattern feature vector is clustered by adaptive clustering algorithm, which the fusion iterative self-organizing data analysis algorithm(ISODATA). The frames corresponding to the intermediate vector in each class is selected as the candidate key frames in the clustering result, and the candidate key frames are again filtered by the similarity, which the redundant frames are eliminated to obtain the final key frames. The experimental results show that a large number of experiments on the Open Video Project dataset indicate that the precision of the key frames extraction is 79.9%, the recall rate is 93.6%, and the F-score is 86.2%, which effectively improves the quality of the video summarization.
In order to improve the accuracy of key frame extraction, and improve the quality of video summaries, a video summarization key frame extraction method for HEVC compressed domain is proposed. Firstly, the video sequence is coded and decoded, and the number of luminance prediction modes of the HEVC intra-coded PU block is counted in the decoding. Secondly, the feature extraction is constructed by using the number of patterns obtained by statistics as a pattern feature vector and used as a texture feature of the video frame for key frame extraction. Finally, the pattern feature vector is clustered by adaptive clustering algorithm, which the fusion iterative self-organizing data analysis algorithm(ISODATA). The frames corresponding to the intermediate vector in each class is selected as the candidate key frames in the clustering result, and the candidate key frames are again filtered by the similarity, which the redundant frames are eliminated to obtain the final key frames. The experimental results show that a large number of experiments on the Open Video Project dataset indicate that the precision of the key frames extraction is 79.9%, the recall rate is 93.6%, and the F-score is 86.2%, which effectively improves the quality of the video summarization.
引文
[1] 王娟,蒋兴浩,孙锬锋.视频摘要技术综述[J].中国图象图形学报,2014,19(12):1685-1695.Wang Juan,Jiang Xinghao,Sun Tanfeng.Review of video abstraction[J].Journal of Image and Graphics,2014,19(12):1685-1695.(in Chinese)
[2] Mendi E,Bayrak C.Shot boundary detection and key frame extraction using salient region detection and structural similarity[C]//Proceedings of the 48th Annual Southeast Regional Conference.ACM,2010:66- 68.
[3] 白慧茹,吕进来.基于聚类方法改进的关键帧提取算法[J].计算机工程与设计,2017,38(7):1929-1933.Bai Huiru,Lv Jinlai.Improved algorithm of key frame extraction based on clustering methods[J].Computer Engineering and Design,2017,38(7):1929-1933.(in Chinese)
[4] 王宇,汪荣贵,杨娟.一种新的自适应的视频关键帧提取方法[J].合肥工业大学学报:自然科学版,2016,39(11):1483-1487,1542.Wang Yu,Wang Ronggui,Yang Juan.A novel adaptive video key frame extraction method[J].Journal of Hefei University of Technology:Natural Science,2016,39(11):1483-1487,1542.(in Chinese)
[5] 赵磊,黄华.AVS监控档视频的压缩域摘要研究[J].计算机科学,2016,43(7):46-50.Zhao Lei,Huang Hua.Compressed domain synopsis research in AVS surveillance profile[J].Computer Science,2016,43(7):46-50.(in Chinese)
[6] Li Y,Zuo Y,Yang Z,et al.High quality voice conversion based on ISODATA clustering algorithm[C]//International Conference on Intelligent Systems and Knowledge Engineering.IEEE,2018:1-5.
[7] 韩雪,冯桂,曹海燕.3D-HEVC深度图帧内编码快速算法[J].信号处理,2018,34(6):680- 687.Han Xue,Feng Gui,Cao Haiyan.Efficient fast algorithm for depth map in 3D-HEVC[J].Journal of Signal Processing,2018,34(6):680- 687.(in Chinese)
[8] 杨宇航,蔡灿辉,王张欣.利用纹理结构的HEVC快速帧内模式选择算法[J].信号处理,2015,31(9):1094-1100.Yang Yuhang,Cai Canhui,Wang Zhangxin.Fast Mode Decision for HEVC Intra Coding Using Texture Information[J].Journal of Signal Processing,2015,31(9):1094-1100.(in Chinese)
[9] 朱威,张晗钰,易瑶,等.低复杂度的HEVC帧内编码模式决策算法[J].小型微型计算机系统,2017,38(12):2630-2636.Zhu Wei,Zhang Hanyu,Yi Yao,et al.Low complexity mode decision algorithm for HEVC intra coding[J].Journal of Chinese Computer Systems,2017,38(12):2630-2636.(in Chinese)
[10] Tzortzis G,Likas A,Tzortzis G.The Min Max K-means clustering algorithm[J].Pattern Recognition,2014,47(7):2505-2516.
[11] Avila S,Lopes A,Luz A,et al.VSUMM:A mechanism designed to produce static video summaries and a novel evaluation method[J].Pattern Recognition Letters,2011,32(1):56- 68.
[12] Dementhon D,Kobla V,Doermann D.Video summarization by curve simplification[C]//ACM International Conference on Multimedia,1999:211-218.
[13] Mundur P,Rao Y,Yesha Y.Keyframe-based video summarization using delaunay clustering[J].International Journal on Digital Libraries,2006,6(2):219-232.
[14] Mahmoud K M,Ismail M A,Ghanem N M.VSCAN:an enhanced video summarization using density-based spatial clustering[C]//International Conference on Image Analysis and Processing(ICIAP),2013:733-742.
[15] Li J T,Yao T,Ling Q,et al.Detecting shot boundary with sparse coding for video summarization[J].Neurocomputing,2017,266:66-78.
[16] 冀中,马亚茹,何宇清.最大边界重要和覆盖的视频摘要方法[J].计算机科学与探索,2018,12(8):1286-1294.Ji Zhong,Ma Yaru,He Yuqing.Video Summarization with Maximal Marginal Importance and Coverage[J].Journal of Frontiers of Computer Science and Technology,2018,12(8):1286-1294.(in Chinese)
[2] Mendi E,Bayrak C.Shot boundary detection and key frame extraction using salient region detection and structural similarity[C]//Proceedings of the 48th Annual Southeast Regional Conference.ACM,2010:66- 68.
[3] 白慧茹,吕进来.基于聚类方法改进的关键帧提取算法[J].计算机工程与设计,2017,38(7):1929-1933.Bai Huiru,Lv Jinlai.Improved algorithm of key frame extraction based on clustering methods[J].Computer Engineering and Design,2017,38(7):1929-1933.(in Chinese)
[4] 王宇,汪荣贵,杨娟.一种新的自适应的视频关键帧提取方法[J].合肥工业大学学报:自然科学版,2016,39(11):1483-1487,1542.Wang Yu,Wang Ronggui,Yang Juan.A novel adaptive video key frame extraction method[J].Journal of Hefei University of Technology:Natural Science,2016,39(11):1483-1487,1542.(in Chinese)
[5] 赵磊,黄华.AVS监控档视频的压缩域摘要研究[J].计算机科学,2016,43(7):46-50.Zhao Lei,Huang Hua.Compressed domain synopsis research in AVS surveillance profile[J].Computer Science,2016,43(7):46-50.(in Chinese)
[6] Li Y,Zuo Y,Yang Z,et al.High quality voice conversion based on ISODATA clustering algorithm[C]//International Conference on Intelligent Systems and Knowledge Engineering.IEEE,2018:1-5.
[7] 韩雪,冯桂,曹海燕.3D-HEVC深度图帧内编码快速算法[J].信号处理,2018,34(6):680- 687.Han Xue,Feng Gui,Cao Haiyan.Efficient fast algorithm for depth map in 3D-HEVC[J].Journal of Signal Processing,2018,34(6):680- 687.(in Chinese)
[8] 杨宇航,蔡灿辉,王张欣.利用纹理结构的HEVC快速帧内模式选择算法[J].信号处理,2015,31(9):1094-1100.Yang Yuhang,Cai Canhui,Wang Zhangxin.Fast Mode Decision for HEVC Intra Coding Using Texture Information[J].Journal of Signal Processing,2015,31(9):1094-1100.(in Chinese)
[9] 朱威,张晗钰,易瑶,等.低复杂度的HEVC帧内编码模式决策算法[J].小型微型计算机系统,2017,38(12):2630-2636.Zhu Wei,Zhang Hanyu,Yi Yao,et al.Low complexity mode decision algorithm for HEVC intra coding[J].Journal of Chinese Computer Systems,2017,38(12):2630-2636.(in Chinese)
[10] Tzortzis G,Likas A,Tzortzis G.The Min Max K-means clustering algorithm[J].Pattern Recognition,2014,47(7):2505-2516.
[11] Avila S,Lopes A,Luz A,et al.VSUMM:A mechanism designed to produce static video summaries and a novel evaluation method[J].Pattern Recognition Letters,2011,32(1):56- 68.
[12] Dementhon D,Kobla V,Doermann D.Video summarization by curve simplification[C]//ACM International Conference on Multimedia,1999:211-218.
[13] Mundur P,Rao Y,Yesha Y.Keyframe-based video summarization using delaunay clustering[J].International Journal on Digital Libraries,2006,6(2):219-232.
[14] Mahmoud K M,Ismail M A,Ghanem N M.VSCAN:an enhanced video summarization using density-based spatial clustering[C]//International Conference on Image Analysis and Processing(ICIAP),2013:733-742.
[15] Li J T,Yao T,Ling Q,et al.Detecting shot boundary with sparse coding for video summarization[J].Neurocomputing,2017,266:66-78.
[16] 冀中,马亚茹,何宇清.最大边界重要和覆盖的视频摘要方法[J].计算机科学与探索,2018,12(8):1286-1294.Ji Zhong,Ma Yaru,He Yuqing.Video Summarization with Maximal Marginal Importance and Coverage[J].Journal of Frontiers of Computer Science and Technology,2018,12(8):1286-1294.(in Chinese)
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |