面向对象的MPEG-4视频编码算法的研究及其DSP实现
摘要
在信息社会,人们迫切希望计算机能以人类习惯的方式提供信息服务,因而多媒体技术应运而生。它的出现,使得原本只有文字界面的计算机有了生动的面孔。人们不仅可以通过直接看到的影像和听到的声音来学习和使用对象,并可以参与或者改变信息的演示。接着,多媒体视频、音频通信也随之出现了,这将是通信的发展方向。
本文讨论了面向对象的MPEG-4视频压缩编码算法的整体流程及MPEG-4简单子集(SP)的DSP实现过程,并结合硬件主芯片——ADI公司的Blackfin系列ADSP-BF533的特点,对编码过程中的部分算法模块进行优化。本文最后使用几种标准测试序列对ADSP-BF533实现的编码器进行了测试,并对实验结果分别进行了定性和定量的分析。实验结果验证了编码器的有效性,并且通过硬件仿真说明了基于ADSP-BF533的实时嵌入式视频处理器是可行的。
In the past several years, Telecommunication makes progress constantly. With the development of hardware and the increase of frequency bandwidth, video communication plays a more and more important role. Because of the popularization of mobile communication recently, wireless video becomes the mainstream. Facing to extensive and quantity sharply increasing users, their demands can not be satisfied with the increase of bandwidth. Under the condition of limited bandwidth, video compression can make use of bandwidth more effectively, so more quality and more compress ratio video compress standard becomes the focus of the market.
Nowadays, there are a lot of video compressing encoder standards. In Part 1 of this paper, we introduce some familiar video compress encoder standards, such as MPEG-1/2/4/7/21 and H.261, H.263, H.264. Among these standards, MPEG-4 is a new video standard based on H.263, MPEG-1 and MPEG-2, it commits itself to realize video compress and communication at different bit rate. Because MPEG-4 is support of MPEG-1 and MPEG-2 and introduce new encode algorithm, compression rate improve by a large margin. And MPEG-4 offers new multimedia service. Multimedia communication will become more vivid, convenient and inter-dynamic.
MPEG-4 is a video compress standard based on content, this is its most remarkable character. In this method, a scene is composed of some interesting characteristic video object (VO). We can encode these objects respectively and transmit and decode these objects selectively, or add new object, then create the scene which we are fond of. The video object is an accessible and operable entity in a scene. In an idiographic scene, video object is an interesting region composed of entities. Encoding an object can form an object bit layer. This layer includes shape, position, texture, etc. This “based on content”encode method is very
flexible, so MPEG-4 is suitable for low bit rate and interact situation. Another key function of MPEG-4 is expansibility. It includes definition adjustable and objects adjustable character. The former is that the content and the quality of the image can be expanded at accurate interval. The latter means that it can add or delete object in the encoded scenes. When the network bandwidth becomes narrow, we can transmit the video signal by reducing the quality in order to reduce the demand of the bandwidth and guarantee the continuity of video communication. MPEG4-SP is a special simple MPEG-4 model. SP means simple profile. Because its algorithm is simple correspondingly, it is suitable to realize by hardware real-timely. Different from above-mentioned MPEG-4 standard, MPEG4-SP is not a kind of encoding standard based on objects, and it has not multi-object encoding tools at the same time. It encodes the video signal based on rectangle frames. It inherits from H.263, MPEG-1 and MPEG-2 and it has not information of scenes and objects. At present, most MPEG-4 products in the market are all realized in the way of MPEG4-SP. They can not realize the algorithm of based on objects. The algorithm of based on objects indeed is not mature. So people are discussing and studying extensively in this field. MPEG4-SP model inherit the algorithm from MPEG-1 and MPEG-2, enumerate them as followed: IVOP (visual object plane, it is based on rectangle frame): DCT&IDCT; Quantization & Inverse Quantization; (we can select the method of H.263 or MPEG-2) Rebuild the VOP; VLC (Variable Length Code, that means entropy code); P_VOP: Interpolate; ME (Motion Estimation); Half Pixel Interpolate; MC (Motion Compensation);
Difference of VOP frames; MPEG4-SP model has its own characters (compared with H.263, MPEG-1 and MPEG-2). Such as followed: In support of the quantization and inverse quantization algorithm of H.263 and MPEG-2; Forecast the DC and AC coefficients of intra-macro-block; Advanced forecasting motion vectors; Unlimited motion estimation and motion compensation; Special Zigzag scanning mode Overlap motion compensation; On application of MPEG4-SP model the encoded data includes two kinds of information. One is texture information, namely the video compressed data; other is motion information, we can comeback the PVOP by using motion information. By applying these algorithms on the MPEG4-SP encoder, we can obtain better compressed results. Considering the realization of video compression standard, the appearance of high performance and universal chip make it possible that we realize video encode by hardware except that we realize the encoder by software. And that the realization on DSP is a quite popular solution in recent years. Its advantages are that the cycle of research and development is short, the chip’s speed can meet the demand of the MPEG-4 encoder, and we also can ameliorate the algorithm according to the demand at any time. Certainly it has a lot of shortcomings, for example, the DSP chip is very expensive. We can not produce a large number of products. But after the appearance of mature DSP, there will be better commercial value. In this paper, we use the product of Analog Device-ADSP-BF533 to realize the MPEG-4 encoder. In part 2, we introduce the core architecture and main peripherals of ADSP-BF533. The algorithm of the encoder is MPEG4-SP, it has mature technology and simple operation. We also can optimize the algorithm module according to the character of ADSP-BF533 and special video pixel
本文讨论了面向对象的MPEG-4视频压缩编码算法的整体流程及MPEG-4简单子集(SP)的DSP实现过程,并结合硬件主芯片——ADI公司的Blackfin系列ADSP-BF533的特点,对编码过程中的部分算法模块进行优化。本文最后使用几种标准测试序列对ADSP-BF533实现的编码器进行了测试,并对实验结果分别进行了定性和定量的分析。实验结果验证了编码器的有效性,并且通过硬件仿真说明了基于ADSP-BF533的实时嵌入式视频处理器是可行的。
In the past several years, Telecommunication makes progress constantly. With the development of hardware and the increase of frequency bandwidth, video communication plays a more and more important role. Because of the popularization of mobile communication recently, wireless video becomes the mainstream. Facing to extensive and quantity sharply increasing users, their demands can not be satisfied with the increase of bandwidth. Under the condition of limited bandwidth, video compression can make use of bandwidth more effectively, so more quality and more compress ratio video compress standard becomes the focus of the market.
Nowadays, there are a lot of video compressing encoder standards. In Part 1 of this paper, we introduce some familiar video compress encoder standards, such as MPEG-1/2/4/7/21 and H.261, H.263, H.264. Among these standards, MPEG-4 is a new video standard based on H.263, MPEG-1 and MPEG-2, it commits itself to realize video compress and communication at different bit rate. Because MPEG-4 is support of MPEG-1 and MPEG-2 and introduce new encode algorithm, compression rate improve by a large margin. And MPEG-4 offers new multimedia service. Multimedia communication will become more vivid, convenient and inter-dynamic.
MPEG-4 is a video compress standard based on content, this is its most remarkable character. In this method, a scene is composed of some interesting characteristic video object (VO). We can encode these objects respectively and transmit and decode these objects selectively, or add new object, then create the scene which we are fond of. The video object is an accessible and operable entity in a scene. In an idiographic scene, video object is an interesting region composed of entities. Encoding an object can form an object bit layer. This layer includes shape, position, texture, etc. This “based on content”encode method is very
flexible, so MPEG-4 is suitable for low bit rate and interact situation. Another key function of MPEG-4 is expansibility. It includes definition adjustable and objects adjustable character. The former is that the content and the quality of the image can be expanded at accurate interval. The latter means that it can add or delete object in the encoded scenes. When the network bandwidth becomes narrow, we can transmit the video signal by reducing the quality in order to reduce the demand of the bandwidth and guarantee the continuity of video communication. MPEG4-SP is a special simple MPEG-4 model. SP means simple profile. Because its algorithm is simple correspondingly, it is suitable to realize by hardware real-timely. Different from above-mentioned MPEG-4 standard, MPEG4-SP is not a kind of encoding standard based on objects, and it has not multi-object encoding tools at the same time. It encodes the video signal based on rectangle frames. It inherits from H.263, MPEG-1 and MPEG-2 and it has not information of scenes and objects. At present, most MPEG-4 products in the market are all realized in the way of MPEG4-SP. They can not realize the algorithm of based on objects. The algorithm of based on objects indeed is not mature. So people are discussing and studying extensively in this field. MPEG4-SP model inherit the algorithm from MPEG-1 and MPEG-2, enumerate them as followed: IVOP (visual object plane, it is based on rectangle frame): DCT&IDCT; Quantization & Inverse Quantization; (we can select the method of H.263 or MPEG-2) Rebuild the VOP; VLC (Variable Length Code, that means entropy code); P_VOP: Interpolate; ME (Motion Estimation); Half Pixel Interpolate; MC (Motion Compensation);
Difference of VOP frames; MPEG4-SP model has its own characters (compared with H.263, MPEG-1 and MPEG-2). Such as followed: In support of the quantization and inverse quantization algorithm of H.263 and MPEG-2; Forecast the DC and AC coefficients of intra-macro-block; Advanced forecasting motion vectors; Unlimited motion estimation and motion compensation; Special Zigzag scanning mode Overlap motion compensation; On application of MPEG4-SP model the encoded data includes two kinds of information. One is texture information, namely the video compressed data; other is motion information, we can comeback the PVOP by using motion information. By applying these algorithms on the MPEG4-SP encoder, we can obtain better compressed results. Considering the realization of video compression standard, the appearance of high performance and universal chip make it possible that we realize video encode by hardware except that we realize the encoder by software. And that the realization on DSP is a quite popular solution in recent years. Its advantages are that the cycle of research and development is short, the chip’s speed can meet the demand of the MPEG-4 encoder, and we also can ameliorate the algorithm according to the demand at any time. Certainly it has a lot of shortcomings, for example, the DSP chip is very expensive. We can not produce a large number of products. But after the appearance of mature DSP, there will be better commercial value. In this paper, we use the product of Analog Device-ADSP-BF533 to realize the MPEG-4 encoder. In part 2, we introduce the core architecture and main peripherals of ADSP-BF533. The algorithm of the encoder is MPEG4-SP, it has mature technology and simple operation. We also can optimize the algorithm module according to the character of ADSP-BF533 and special video pixel
引文
[1] 黎洪松,《数字视频技术及其应用》,清华大学出版社,1997年5月
[2] 研究生“多媒体技术(二)”课程教学,西安电子科技大学多媒体研究所,7.4 图像子采样,http://mti.xidian.edu.cn/multimedia/multi/course1-7-4.html。
[3] 张立保,王珂,一种基于小波变换的图像编码算法。软件学报
[4] 司徽东付永生王俊,噪声图像的分形压缩编码的研究,电子技术应用
[5] 虞露,周峰,姚庆栋.脸部序列图像的模型基编码.通信学报,1997(10):1-6.YU Lu, ZHOU Feng, YAO Qing-dong. Model-based coding for facial sequence. Journal of China Institute of Communications, 1997,(10):1-6.
[6] ITU-T Recommendation H.261: “Video codec for audiovisual services at px64 kbit/s,”Geneva, 1990, revised at Helsinki, March 1993.
[7] ISO/IEC JTC1,Generic Coding of Audiovisual Objects-Part2: Visual (MPEG-4 Visual), ISO/IEC 14496-2, Version 1: January 1999; Version 2: January 2000; Version 3: January 2001
[8] Atul Puri, Alexandros Eleftheriadis, “MPEG-4: An objiect-based multimedia coding standard supporting mobile applications”, Mobile Networks and Applications, PP5-32, No.3, 1998
[9] 精英科技编著,视频压缩与音频编码技术,中国电力出版社,北京,2001
[10] 钟玉琢,王琪,贺玉文(Zhong Yu-zhuo, Wang Qi, He Yu-wen)。基于对象的多媒体数据压缩编码国际标准——MPEG-4 及其校验模型(Based on Video Object Multimedia Data Compress Encode International Standard-MPEG-4 and Video Verification Model)。北京:科学出版社(Beijing: Press of Science),2000。
[11] 高梅国、何佩琨、王飞,《数字信号处理器DSPs的发展》,《电子技术应用》2000年第5期。
[12] 孟逢逢、蒋建国,《DSP技术的应用与发展》,《微处理器》,2000年8 月第3期。
[13] 黄峰、熊召新、李胜平,《DSP体系结构发展的新趋势》,《计算机工程》,2002年第4期。
[14] IA-32 Intel Architecture Software Developer's Manual, Intel, 2002
[15] The Trimedia CPU64 VLIW Media Processor,Kees Vissers, PhilipsResearch
[16] IME6400:MPEG4/2/1 Multimedia Encoder, INTiME Corp, 324-4 Yangjae-dong, 2F, Seocho-gu, Seoul 137-130 SOUTH KOREA.
[17] Analog Device Inc, “ADSP-21535 BlackfinTM DSP Hardware Reference”Revision 1.0, November 2002, Part Number, 82-000410-13.
[18] Analog Device Inc, “Blackfin Embedded Processor ADSP-BF531/ADSP-BF532/ADSP-BF533 Data Sheet”.
[19] Analog Device Inc, “ADSP-BF533 EZ-KIT LiteTM Evaluation System Manual”Revision 1.0, April 2003, Part Number 82-000730-01.
[20] Analog Device Inc, “VisualDSP++TM 3.1 C/C++ Compiler and Library Manual for Blackfin Processors”, Revision 2.1, April 2003, Part Number 82-000410-03.
[21] Visual DSP++ 3.1 Component Software Engineering User’s Guide, Revision 4.1, April 2003, Part Number 82-000410-01
[22] Analog Device Inc, “VisualDSP++TM 3.1 Linker and Utilities Manual for Blackfin Processor”Revision 2.1, April 2003, Part Number 82-000410-05.
[23] MPEG-4 Video Verification Model version 18.0, International organization for standardization organization international normalization ISO/IEC JTC1/SC29/WG11 CODING OF MOVING PICTURES AND AUDIO ISO/IEC JTC1/SC29/WG11 N3908 January 2001/Pisa.
[24] Jie Ruan, Mark G.Arnold, Lehigh University, “LNS Arithmetic for MPEG Encoding Using a Fast DCT”,.
[25] JD(FRIO) Software Application Team, IPDC, Bangalore, India. http://www.analog.com/processors/processors/blackfin/technicalLibrary/manuals/codeExamples.html.
[26] 田毅珂,卢朝阳,高西全(Tian Yi-ke, Lu Chao-yang, Gao Xi-quan)。
[2] 研究生“多媒体技术(二)”课程教学,西安电子科技大学多媒体研究所,7.4 图像子采样,http://mti.xidian.edu.cn/multimedia/multi/course1-7-4.html。
[3] 张立保,王珂,一种基于小波变换的图像编码算法。软件学报
[4] 司徽东付永生王俊,噪声图像的分形压缩编码的研究,电子技术应用
[5] 虞露,周峰,姚庆栋.脸部序列图像的模型基编码.通信学报,1997(10):1-6.YU Lu, ZHOU Feng, YAO Qing-dong. Model-based coding for facial sequence. Journal of China Institute of Communications, 1997,(10):1-6.
[6] ITU-T Recommendation H.261: “Video codec for audiovisual services at px64 kbit/s,”Geneva, 1990, revised at Helsinki, March 1993.
[7] ISO/IEC JTC1,Generic Coding of Audiovisual Objects-Part2: Visual (MPEG-4 Visual), ISO/IEC 14496-2, Version 1: January 1999; Version 2: January 2000; Version 3: January 2001
[8] Atul Puri, Alexandros Eleftheriadis, “MPEG-4: An objiect-based multimedia coding standard supporting mobile applications”, Mobile Networks and Applications, PP5-32, No.3, 1998
[9] 精英科技编著,视频压缩与音频编码技术,中国电力出版社,北京,2001
[10] 钟玉琢,王琪,贺玉文(Zhong Yu-zhuo, Wang Qi, He Yu-wen)。基于对象的多媒体数据压缩编码国际标准——MPEG-4 及其校验模型(Based on Video Object Multimedia Data Compress Encode International Standard-MPEG-4 and Video Verification Model)。北京:科学出版社(Beijing: Press of Science),2000。
[11] 高梅国、何佩琨、王飞,《数字信号处理器DSPs的发展》,《电子技术应用》2000年第5期。
[12] 孟逢逢、蒋建国,《DSP技术的应用与发展》,《微处理器》,2000年8 月第3期。
[13] 黄峰、熊召新、李胜平,《DSP体系结构发展的新趋势》,《计算机工程》,2002年第4期。
[14] IA-32 Intel Architecture Software Developer's Manual, Intel, 2002
[15] The Trimedia CPU64 VLIW Media Processor,Kees Vissers, PhilipsResearch
[16] IME6400:MPEG4/2/1 Multimedia Encoder, INTiME Corp, 324-4 Yangjae-dong, 2F, Seocho-gu, Seoul 137-130 SOUTH KOREA.
[17] Analog Device Inc, “ADSP-21535 BlackfinTM DSP Hardware Reference”Revision 1.0, November 2002, Part Number, 82-000410-13.
[18] Analog Device Inc, “Blackfin Embedded Processor ADSP-BF531/ADSP-BF532/ADSP-BF533 Data Sheet”.
[19] Analog Device Inc, “ADSP-BF533 EZ-KIT LiteTM Evaluation System Manual”Revision 1.0, April 2003, Part Number 82-000730-01.
[20] Analog Device Inc, “VisualDSP++TM 3.1 C/C++ Compiler and Library Manual for Blackfin Processors”, Revision 2.1, April 2003, Part Number 82-000410-03.
[21] Visual DSP++ 3.1 Component Software Engineering User’s Guide, Revision 4.1, April 2003, Part Number 82-000410-01
[22] Analog Device Inc, “VisualDSP++TM 3.1 Linker and Utilities Manual for Blackfin Processor”Revision 2.1, April 2003, Part Number 82-000410-05.
[23] MPEG-4 Video Verification Model version 18.0, International organization for standardization organization international normalization ISO/IEC JTC1/SC29/WG11 CODING OF MOVING PICTURES AND AUDIO ISO/IEC JTC1/SC29/WG11 N3908 January 2001/Pisa.
[24] Jie Ruan, Mark G.Arnold, Lehigh University, “LNS Arithmetic for MPEG Encoding Using a Fast DCT”,.
[25] JD(FRIO) Software Application Team, IPDC, Bangalore, India. http://www.analog.com/processors/processors/blackfin/technicalLibrary/manuals/codeExamples.html.
[26] 田毅珂,卢朝阳,高西全(Tian Yi-ke, Lu Chao-yang, Gao Xi-quan)。