视频点播系统客户端设计及H.264差错隐藏的研究
摘要
随着宽带技术的发展,基于流媒体技术的多媒体业务的应用在网络上也越来越广泛,人们追求高质量多媒体业务的脚步一刻也没有停止。然而,网络带宽的限制依然是目前Internet上高清晰度视频业务发展的瓶颈。如何在有限带宽上提供高质量的视频服务依然是研究的热点。H.264/AVC是ITU-T的视频编码专家组和ISO/IEC的运动图像专家组联合提出新一代的视频编码标准,具有很高的压缩效率和很强的鲁棒性,非常适合于网络传输。基于这样的背景,我们研究开发了基于H.264的视频点播系统。由于IP网络是尽力而为的,视频数据包在经过网络传输过程中,往往会发生数据包丢失的情况。由于H.264的高压缩率,相比以前的视频压缩标准,数据包丢失会对H.264码流造成更加严重的后果。
为此,本文主要研究视频点播系统中客户端设计以及客户端的QoS技术。具体的,本文主要完成以下工作:1、本文设计并开发了视频点播系统的客户端,实现了客户端建连、点播、播放、暂停等基本用户需求,同时提出客户端RTP乱序重排和可调节大小的环形缓冲区算法,有效的缓解了视频传输中延时抖动等问题;2、本文针对H.264视频压缩的特性,提出了一种实时差错隐藏技术,对于帧间编码的图像利用相邻宏块的宏块划分模式有效的预测了丢失宏块的宏块划分模式以及运动向量;对于帧内编码的图像使用运动搜索获得周围宏块的运动信息,根据自适应的门限条件将运动补偿与双线性插值结合使用。实验结果表明本算法能在解码时实时进行差错隐藏,同时主观和客观质量比双线性插值有较大的提高。
Multimedia service, based on streaming media technology, was applied abroad on the Internet. People never stop pursuing multimedia service of high quality for a moment. However, the limitation of bandwidth still is the bottleneck of the development of high quality multimedia service. How to provide high quality mulimedia service on limited bandwidth becomes the focus of research field. H.264/AVC, which has outstanding compress performance and strong robustness, is a new video compress standard, issued by the joint video team (JVT) consisting of the members of MPEG and the ITU's Video Coding Experts Group. It is very suitable for network transmission. Based on the background, we research and develop the Video On Demand(VOD) system for H.264. Because IP network provides best effort service, the video data packet may lost during network transmission. However, due to high compress rate, in contrast with previous video compress standards, the influence of data packet loss will make more serious effect over H.264 bitstream.
Therefore, this thesis mainly discusses the design and implementation of the client of the VOD system. Our major contributions are listed as follow. First, we design and develop the client of the VOD system. We implement basic function for user operation,such as connection of the client, programme order, play,pause. We also propose a sorting algorithm to deal with out-of-order of RTP data packet. And a algorithm for auto-adjustment of ring buffer size is developed as well. This algorithm may reduce the impact of delay jitter during data packet tansmission. Second, we propose a new real-time error concealment algrithm ,based on the new feature of H.264. We predict the lost Macroblock partion model and motion vector by using of the Macroblock partion model of around Macroblock. The experiment result shows that the algrithm meets real-time need, and the object quality is great improved ,compared with Bilinear interpolation algorithm.
为此,本文主要研究视频点播系统中客户端设计以及客户端的QoS技术。具体的,本文主要完成以下工作:1、本文设计并开发了视频点播系统的客户端,实现了客户端建连、点播、播放、暂停等基本用户需求,同时提出客户端RTP乱序重排和可调节大小的环形缓冲区算法,有效的缓解了视频传输中延时抖动等问题;2、本文针对H.264视频压缩的特性,提出了一种实时差错隐藏技术,对于帧间编码的图像利用相邻宏块的宏块划分模式有效的预测了丢失宏块的宏块划分模式以及运动向量;对于帧内编码的图像使用运动搜索获得周围宏块的运动信息,根据自适应的门限条件将运动补偿与双线性插值结合使用。实验结果表明本算法能在解码时实时进行差错隐藏,同时主观和客观质量比双线性插值有较大的提高。
Multimedia service, based on streaming media technology, was applied abroad on the Internet. People never stop pursuing multimedia service of high quality for a moment. However, the limitation of bandwidth still is the bottleneck of the development of high quality multimedia service. How to provide high quality mulimedia service on limited bandwidth becomes the focus of research field. H.264/AVC, which has outstanding compress performance and strong robustness, is a new video compress standard, issued by the joint video team (JVT) consisting of the members of MPEG and the ITU's Video Coding Experts Group. It is very suitable for network transmission. Based on the background, we research and develop the Video On Demand(VOD) system for H.264. Because IP network provides best effort service, the video data packet may lost during network transmission. However, due to high compress rate, in contrast with previous video compress standards, the influence of data packet loss will make more serious effect over H.264 bitstream.
Therefore, this thesis mainly discusses the design and implementation of the client of the VOD system. Our major contributions are listed as follow. First, we design and develop the client of the VOD system. We implement basic function for user operation,such as connection of the client, programme order, play,pause. We also propose a sorting algorithm to deal with out-of-order of RTP data packet. And a algorithm for auto-adjustment of ring buffer size is developed as well. This algorithm may reduce the impact of delay jitter during data packet tansmission. Second, we propose a new real-time error concealment algrithm ,based on the new feature of H.264. We predict the lost Macroblock partion model and motion vector by using of the Macroblock partion model of around Macroblock. The experiment result shows that the algrithm meets real-time need, and the object quality is great improved ,compared with Bilinear interpolation algorithm.
引文
[1]H.264.Advanced video coding for generic audiovisual services.Switzerland:ITU-T,2005.
[2]毕厚杰.《新一代视频压缩编码标准——H.264/AVC》.北京:人民邮电出版社,2005.
[3]rfc1889.RTP:A Transport Protocol for Real-Time Applications.USA:IETF,1996.
[4]rfc2326.Real Time Streaming Protocol.USA:IETF,1998.
[5]殷人昆,陶永雷,谢若阳等.《数据结构(用面向对象方法与C++描述)》.北京:清华大学出版社,1999.
[6]Thomas H.Cormen,Charles E.Leiserson,Ronald L.Rivest.etc.Introduction to Algorithms(Second Edition).USA:MIT Press,2001.
[7]Zhang aidong etc.Netmedia:Streaming Multimedia Presentations in DistributedEnvironments.IEEE Multimedia,2002,Jan-March
[8]李庆宅.《分布式编辑系统客户端设计与实现》.北京:北京邮电大学,2005.
[9]E.Biersack,W.Geyer,Synchronized delivery and playout of distributed stored multimedia streams.Multimedia Sys,1999,Vol.7,70-90.
[10]Y.Wang,Q.-F.Zhu,Error control and concealment for video communication:A review.Proc.IEEE,vol.86,pp.974-997,May 1998.
[11]Haskell P.and Messerschmitt D.,Resynchronization of motion compensated video affected by Atm cell loss,in Proceeding of the 1992 IEEE International Conference on Acoustic,Speech and Signal Processing,vol.3,pp.545-548,1992.
[12]S.Ghanbari,Bober,M.Z,A cluster based method for the recovery of the lost motion vectors in video coding.Mobile and Wireless Communications Network,pp583-586,2002.
[13]W.M.Lain,etc.,Recovery of lost or erroneously received motion vectors,ICASSP'93,Volume 5,pp417-420 vol.5,1993
[14]Bo Yan,etc.,A Novel Motion Vector Recovery Algorithm for Error Concealment in Video Transmission,Consumer Communications and Networking Conference.pp.621-623,2004.
[15]M.J.Chen,L.G.Chen,etc.,Error Concealment of Lost Motion Vectors with OverlappedMotion Compensation,IEEE Trans, on Cir&Sys. for Video Tech Vol. 7,pp.560-563,1997.
[16] S. Teskeridou, etc., Motion field estimation by vector rational interpolation for error concealment purposes, ICASSP '99, Vol. 6, pp.3397-3400 vol.6, 1999.
[17] S. Aign, K. Fazel. Temporal and Spatial Error Concealment Techniques for Hierarchical MPEG-2 Video Codec. In Proc, of Globe Comm'95: 1778-1783,1995.
[18] Kwok W., Sun, H. Multi-directional interpolation for spatial error concealment. IEEE Transactions on Consumer Electronics, Vol. 39, pp.455-460,1993.
[19] Jae-Won Suh; Yo-Sung Ho; Error concealment based on directional interpolation., IEEE Transactions on Consumer Electronics, Vol. 43, pp.295-302,1997.
[20] H. Sun and W.Kwok, Concealment of damaged blocks transform coded images using projections onto convex sets. IEEE Trans. on Image Processing, vol. 4, pp. 470-477, Apr. 1995.
[21] Shirani S., etc. A concealment method for video communications in an error prone environment, IEEE Journal on Selected Areas in Communications, Vol 18, pp.1122-128, 2000.
[22] Wang, Y, Zhu, Q.-F.; Shaw, L.,Maximally smooth image recovery in transform coding, IEEE Transactions on Communications, Vol. 41, pp. 1544-1551, 1993.
[23] Zhu Q. F.etc, Coding and cell-loss recovery in DCT-based packet video, IEEE Transactions on CSVT, Vol.3, pp.248-25,1993.
[24] J. W. Park; etc. , DCT coefficients recovery-based error concealment technique and its application to the MPEG-2 bit streamerror, IEEE Trans, on CSVT, Vol. 7, pp.845-854,1997.
[25] Tsai, I. W. and Huang, C. L., Hybrid cell loss concealment methods for MPEG-II based packet video. Signal Processing: Image Communication, vol. 9, pp. 99-124, 1997.
[26] Zhu, Q. F., Wang, Y, and Shaw, L., Coding and cell-loss recovery in dct-based packet video. IEEE Transactions on Circuits and Systems for Video Technology, vol. 3,no. 3, pp. 248-258, 1993.
[2]毕厚杰.《新一代视频压缩编码标准——H.264/AVC》.北京:人民邮电出版社,2005.
[3]rfc1889.RTP:A Transport Protocol for Real-Time Applications.USA:IETF,1996.
[4]rfc2326.Real Time Streaming Protocol.USA:IETF,1998.
[5]殷人昆,陶永雷,谢若阳等.《数据结构(用面向对象方法与C++描述)》.北京:清华大学出版社,1999.
[6]Thomas H.Cormen,Charles E.Leiserson,Ronald L.Rivest.etc.Introduction to Algorithms(Second Edition).USA:MIT Press,2001.
[7]Zhang aidong etc.Netmedia:Streaming Multimedia Presentations in DistributedEnvironments.IEEE Multimedia,2002,Jan-March
[8]李庆宅.《分布式编辑系统客户端设计与实现》.北京:北京邮电大学,2005.
[9]E.Biersack,W.Geyer,Synchronized delivery and playout of distributed stored multimedia streams.Multimedia Sys,1999,Vol.7,70-90.
[10]Y.Wang,Q.-F.Zhu,Error control and concealment for video communication:A review.Proc.IEEE,vol.86,pp.974-997,May 1998.
[11]Haskell P.and Messerschmitt D.,Resynchronization of motion compensated video affected by Atm cell loss,in Proceeding of the 1992 IEEE International Conference on Acoustic,Speech and Signal Processing,vol.3,pp.545-548,1992.
[12]S.Ghanbari,Bober,M.Z,A cluster based method for the recovery of the lost motion vectors in video coding.Mobile and Wireless Communications Network,pp583-586,2002.
[13]W.M.Lain,etc.,Recovery of lost or erroneously received motion vectors,ICASSP'93,Volume 5,pp417-420 vol.5,1993
[14]Bo Yan,etc.,A Novel Motion Vector Recovery Algorithm for Error Concealment in Video Transmission,Consumer Communications and Networking Conference.pp.621-623,2004.
[15]M.J.Chen,L.G.Chen,etc.,Error Concealment of Lost Motion Vectors with OverlappedMotion Compensation,IEEE Trans, on Cir&Sys. for Video Tech Vol. 7,pp.560-563,1997.
[16] S. Teskeridou, etc., Motion field estimation by vector rational interpolation for error concealment purposes, ICASSP '99, Vol. 6, pp.3397-3400 vol.6, 1999.
[17] S. Aign, K. Fazel. Temporal and Spatial Error Concealment Techniques for Hierarchical MPEG-2 Video Codec. In Proc, of Globe Comm'95: 1778-1783,1995.
[18] Kwok W., Sun, H. Multi-directional interpolation for spatial error concealment. IEEE Transactions on Consumer Electronics, Vol. 39, pp.455-460,1993.
[19] Jae-Won Suh; Yo-Sung Ho; Error concealment based on directional interpolation., IEEE Transactions on Consumer Electronics, Vol. 43, pp.295-302,1997.
[20] H. Sun and W.Kwok, Concealment of damaged blocks transform coded images using projections onto convex sets. IEEE Trans. on Image Processing, vol. 4, pp. 470-477, Apr. 1995.
[21] Shirani S., etc. A concealment method for video communications in an error prone environment, IEEE Journal on Selected Areas in Communications, Vol 18, pp.1122-128, 2000.
[22] Wang, Y, Zhu, Q.-F.; Shaw, L.,Maximally smooth image recovery in transform coding, IEEE Transactions on Communications, Vol. 41, pp. 1544-1551, 1993.
[23] Zhu Q. F.etc, Coding and cell-loss recovery in DCT-based packet video, IEEE Transactions on CSVT, Vol.3, pp.248-25,1993.
[24] J. W. Park; etc. , DCT coefficients recovery-based error concealment technique and its application to the MPEG-2 bit streamerror, IEEE Trans, on CSVT, Vol. 7, pp.845-854,1997.
[25] Tsai, I. W. and Huang, C. L., Hybrid cell loss concealment methods for MPEG-II based packet video. Signal Processing: Image Communication, vol. 9, pp. 99-124, 1997.
[26] Zhu, Q. F., Wang, Y, and Shaw, L., Coding and cell-loss recovery in dct-based packet video. IEEE Transactions on Circuits and Systems for Video Technology, vol. 3,no. 3, pp. 248-258, 1993.