iPhone平台移动流媒体播放器的实现与优化
|
摘要
随着移动网络的迅猛发展以及第三代移动通信业务的推陈出新,使用移动流媒体业务的用户也日益增多。与此同时,市场的急剧扩大也加速了移动流媒体技术的进步。
在参照了第三代合作伙伴计划的分组交换流媒体服务技术规范的基础上,介绍移动流媒体网络体系中使用的网络协议,论述H.264视频解码技术和AMR-NB(Adaptive Multi-Rate Speech Codec Narrow Band)及AAC(Advanced Audio Coding)音频解码技术以及承载音视频媒体数据的容器3GPP(3rd Generation Partnership Project)文件结构,针对iPhone平台自身特点,重点研究在iPhone平台上移动流媒体播放器的实现。播放器由四大模块组成:网络模块、缓冲模块和音视频解码播放模块。顺序流式传输采用HTTP(Hypertext Transfer Protocol)协议实现网络模块,从服务器接收3GPP文件流,通过解析索引区数据,分别请求音视频媒体数据后,将其送入缓冲区;实时流式传输通过使用RTSP/RTP/RTCP(Real-time Streaming Protocol/ Real-time Transport Protocol/ Real-time Transport Control Protocol)协议栈进行会话建立、媒体数据传输和网络状况反馈。缓冲模块把从网络模块接收到的音视频数据包进行处理,然后将处理后的数据存储在循环队列中,以此来缓解网络丢包、抖动和乱序引起的问题。音视频解码播放模块从缓冲模块取得AMR-NB/AAC音频帧和H.264视频帧,将其送入相应的解码器解码后,播放声音显示图像,并通过时间戳信息进行音视频同步。
比较了移动流媒体网络体系中的两种流式传输体系的优缺点。在采取码率适配技术、大文件传输优化、内网穿透技术和图像缩放技术优化等措施后,在真实的移动网络环境中测试,实现了移动网络下流媒体的流畅播放,达到了设计要求。
With the rapid development of the mobile network and the 3rd Generation network service, the numbers of the mobile streaming service subscribers have increased dramatically, and the extension of the market has also speed up the improvement of the mobile streaming technology.
On the basis of the Packet Switched Streaming Service technology specifications of the 3rd Generation Partnership Project, the network protocols in the mobile streaming network system are presented, and the H.264 video decoding technology and the AMR-NB/AAC audio decoding technology as well as the configuration of the 3GPP files that acting as a container bearing the audio-video media data are analyzed. According to the iPhone platform’s self-characteristics, enough emphasis has been put on the implementation of the mobile streaming media player in the iPhone platform. The player consists of four modules, which include network module, buffer module, audio-video decoding and playing module. Each part of the player is respectively responsible for accepting media data from the network, audio/video decoding and playback. Progressive streaming uses the HTTP protocol to implement the network module, receive the 3GPP file stream from the servers and acquire the audio-video media data through analyzing data in the index area, then put the media data into the buffer respectively. Real-time streaming makes use of the RTSP/RTP/RTCP protocol stack for session establishment, media data transmission and the feedback. Buffer module stores audio-video packets which received and processed by the network module to the circular buffering queue to solve the problem which caused by packets loss, jitter and packets reordering. Audio-video decoding and playing module fetch AMR-NB/AAC audio data and H.264 video data from buffer module and send to corresponding decoder and finally play back.
The using conditions of the two different kinds of stream transmission in the mobile streaming media network system are analyzed. After taking the measures such as rate adaption, the optimization on the transmission of large files, network address translation technology and the optimization of image scaling, the player is tested in real GPRS/EDGE network and meets the requirements of the design.
在参照了第三代合作伙伴计划的分组交换流媒体服务技术规范的基础上,介绍移动流媒体网络体系中使用的网络协议,论述H.264视频解码技术和AMR-NB(Adaptive Multi-Rate Speech Codec Narrow Band)及AAC(Advanced Audio Coding)音频解码技术以及承载音视频媒体数据的容器3GPP(3rd Generation Partnership Project)文件结构,针对iPhone平台自身特点,重点研究在iPhone平台上移动流媒体播放器的实现。播放器由四大模块组成:网络模块、缓冲模块和音视频解码播放模块。顺序流式传输采用HTTP(Hypertext Transfer Protocol)协议实现网络模块,从服务器接收3GPP文件流,通过解析索引区数据,分别请求音视频媒体数据后,将其送入缓冲区;实时流式传输通过使用RTSP/RTP/RTCP(Real-time Streaming Protocol/ Real-time Transport Protocol/ Real-time Transport Control Protocol)协议栈进行会话建立、媒体数据传输和网络状况反馈。缓冲模块把从网络模块接收到的音视频数据包进行处理,然后将处理后的数据存储在循环队列中,以此来缓解网络丢包、抖动和乱序引起的问题。音视频解码播放模块从缓冲模块取得AMR-NB/AAC音频帧和H.264视频帧,将其送入相应的解码器解码后,播放声音显示图像,并通过时间戳信息进行音视频同步。
比较了移动流媒体网络体系中的两种流式传输体系的优缺点。在采取码率适配技术、大文件传输优化、内网穿透技术和图像缩放技术优化等措施后,在真实的移动网络环境中测试,实现了移动网络下流媒体的流畅播放,达到了设计要求。
With the rapid development of the mobile network and the 3rd Generation network service, the numbers of the mobile streaming service subscribers have increased dramatically, and the extension of the market has also speed up the improvement of the mobile streaming technology.
On the basis of the Packet Switched Streaming Service technology specifications of the 3rd Generation Partnership Project, the network protocols in the mobile streaming network system are presented, and the H.264 video decoding technology and the AMR-NB/AAC audio decoding technology as well as the configuration of the 3GPP files that acting as a container bearing the audio-video media data are analyzed. According to the iPhone platform’s self-characteristics, enough emphasis has been put on the implementation of the mobile streaming media player in the iPhone platform. The player consists of four modules, which include network module, buffer module, audio-video decoding and playing module. Each part of the player is respectively responsible for accepting media data from the network, audio/video decoding and playback. Progressive streaming uses the HTTP protocol to implement the network module, receive the 3GPP file stream from the servers and acquire the audio-video media data through analyzing data in the index area, then put the media data into the buffer respectively. Real-time streaming makes use of the RTSP/RTP/RTCP protocol stack for session establishment, media data transmission and the feedback. Buffer module stores audio-video packets which received and processed by the network module to the circular buffering queue to solve the problem which caused by packets loss, jitter and packets reordering. Audio-video decoding and playing module fetch AMR-NB/AAC audio data and H.264 video data from buffer module and send to corresponding decoder and finally play back.
The using conditions of the two different kinds of stream transmission in the mobile streaming media network system are analyzed. After taking the measures such as rate adaption, the optimization on the transmission of large files, network address translation technology and the optimization of image scaling, the player is tested in real GPRS/EDGE network and meets the requirements of the design.
引文
[1]胡晓峰,吴玲达,老松杨等.多媒体技术教程.北京:人民邮电出版社, 2002. 1~3
[2]杨毅刚. 3G发展趋势展望.电信科学, 2005, (2): 5~7
[3] Schulzrinne H, Casner S, Frederick R et al. RTP: A Transport Protocol for Real-Time Applications. RFC 1889. January 1996
[4] Schulzrinne H, Casner S, Frederick R et al. RTP: A Transport Protocol for Real-Time Applications. RFC 3550. July 2003
[5]周敬利,张超,余胜生等.一种基于RTP的多媒体业务质量控制算法.华中科技大学学报(自然科学版), 2002, 30(9): 18~19
[6] Dr. Jonathan P. Castro. The UMTS Network and Radio Access Technology. 1 edition. New York: John Wiley & Sons, Inc. 2004. 107~125
[7] Wiegand Thomas, Sullivan Gary J, Bj?ntegaard Gisle, et al. Overview of the H. 264/AVC Video Coding Standard. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(7): 560~576
[8] ISO/IEC 14496-10 and ITU-T Rec. H. 264: Advanced video coding, 2003
[9] ISO/IEC 14496. Information technology - Coding of audio-visual objects. 1999
[10] G Patel, S Dennett, TX Richardson. The 3GPP and 3GPP2 Movements Towards an All IP Mobile Network. Personal Communications, Aug 2000, 7(4): 62~64
[11] The 3rd Generation Partnership Project. End-to-end Transparent Streaming Service; Protocols and codecs. 3GPP TS 26.234 V6.2.0. 2004
[12] Dr. Pesari. Packet Switched Streaming Service White Paper. TeliaSonera Finland Medialab, 2003: 2~5
[13] R. Lanphier. RTSP: Real-Time Streaming Protocol. Internet draft, IETF RFC 2326, April. 1998
[14] M. Handley, V. Jacobson. SDP: Session Description Protocol. Internet draft, IETF RFC 2327, April. 1998
[15] R. Fielding, J. Gettys, J. Mogul, et al. HTTP: Hypertext Transfer Protocol. Internet draft, IETF RFC 2616, June 1999
[16] ISO/IEC 14496-12.“Information technology-Coding of audio-visual objects-Part 12: ISO base media file format”, 2004
[17]白建兵. H.264编码技术分析.科技信息, 2007, (19): 226~228
[18]骆峰,何晖. H.264编码技术在3G移动通讯中的应用.景德镇高专学报, 2008, 23(2): 2~3
[19]孙毅,林涛,林争辉.新一代视频编码标准H.264/AVC的特点及应用.广播电视信息, 2006, 12(67): 67~68
[20]王文,陈鹏. H.264与MPEG-2标准的比较及其在广播电视领域的发展.有线电视技术, 2006, (4): 49~51
[21]魏为民.图像缩放的秘密.数码摄影, 2008, (7): 132~133
[22]王森,杨克俭.基于双线性插值的图像缩放算法.电脑编辑技巧与维护, 2008, (10): 53~67
[23]张阿珍,刘政林,邹雪城等.基于双三次插值算法的图像缩放引擎的设计.微电子学与计算机, 2007, 24(1): 49~51
[24] H S Hou, H C Andrews. Cubic splines for image interpolation and digital filtering. IEEE Trans. Acoust., Speech, Signal Processing, 1978, ASSP- 26: 508~517
[25] 3GPP TS 26.090 V5.0.0,“AMR Speech Code about Transcending functions,”June 2002
[26]吴恒. AMR语音编码技术及其在移动通信中的应用.黑龙江科技信息, 2008, (28): 74~74
[27]郭晓强,付光涛,李小雨. AAC音频压缩编码标准的ADTS与LATM格式分析.现代电视技术, 2008, (1): 140~142
[28] James Duncan Davidson. Cocoa入门——使用Objective-C.第一版.彭舰,张磊,李征等.北京:中国电力出版社, 2008. 15～22
[29]王伟.移动流媒体平台设计与实现.信息通信, 2009, (1): 63~64
[30]孙弼阳,李虹,王颖.移动流媒体业务的技术与应用,现代电信科技, 2008, (6): 11~12
[31]黄微萃,李毅,李鹏飞等. UMTS移动流媒体传输码率适配技术.广东通讯技术, 2006, 9:37~39
[32]宋秀娟,陈绍伟.一种符合3GPP标准的动态带宽适配算法.计算机测量与控制, 2008, 16(6): 856~858
[33] G. Tsirtsis. Network Address Translation - Protocol Translation. RFC2766, Feb. 2000
[34]徐娟,陈静. UDP穿透NAT的原理与实现.科技广场, 2007, (1): 69~70
[35]何琼.关于连续多媒体通信同步技术的研究.中国水运(理论版), 2006, 4(8): 86~87
[36]孙任飞,牛建伟.基于Symbian平台音视频同步算法研究与实现.济南大学学报(自然科学版), 2007, 21: 60～61
[37]潘敏,刘征宇,韩江洪.无线传输中3GPP实现基础.计算机技术与应用进展, 2007: 1552~1553
[38] Schularinne H. RTP Profile for Audio and Video Conferences with Minimal Control. RFC 1890. Feb. 1999
[39] Timo Halonen, Javier Romero, Juan Melero. GSM, GPRS and EDGE Performance: Evolution Towards 3G/UMTS.John Wiley and Sons, 2003:14~49
[40] Richard S. Wright Jr. iPhone和iPod Touch上的OpenGL ES技术.程序员, 2008, (10): 85~87
[41] F RBER Nikolaus, D HLA Stefan, ISSING Jochen. Adaptive progressive download based on the MPEG-4 file format, Journal of Zhejiang University, 2006, 7(S1): 106~111
[42]文远保,林建明.嵌入式流媒体客户端缓冲控制策略的研究.华中科技大学学报(自然科学版), 2005, 33(10): 83~85
[2]杨毅刚. 3G发展趋势展望.电信科学, 2005, (2): 5~7
[3] Schulzrinne H, Casner S, Frederick R et al. RTP: A Transport Protocol for Real-Time Applications. RFC 1889. January 1996
[4] Schulzrinne H, Casner S, Frederick R et al. RTP: A Transport Protocol for Real-Time Applications. RFC 3550. July 2003
[5]周敬利,张超,余胜生等.一种基于RTP的多媒体业务质量控制算法.华中科技大学学报(自然科学版), 2002, 30(9): 18~19
[6] Dr. Jonathan P. Castro. The UMTS Network and Radio Access Technology. 1 edition. New York: John Wiley & Sons, Inc. 2004. 107~125
[7] Wiegand Thomas, Sullivan Gary J, Bj?ntegaard Gisle, et al. Overview of the H. 264/AVC Video Coding Standard. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(7): 560~576
[8] ISO/IEC 14496-10 and ITU-T Rec. H. 264: Advanced video coding, 2003
[9] ISO/IEC 14496. Information technology - Coding of audio-visual objects. 1999
[10] G Patel, S Dennett, TX Richardson. The 3GPP and 3GPP2 Movements Towards an All IP Mobile Network. Personal Communications, Aug 2000, 7(4): 62~64
[11] The 3rd Generation Partnership Project. End-to-end Transparent Streaming Service; Protocols and codecs. 3GPP TS 26.234 V6.2.0. 2004
[12] Dr. Pesari. Packet Switched Streaming Service White Paper. TeliaSonera Finland Medialab, 2003: 2~5
[13] R. Lanphier. RTSP: Real-Time Streaming Protocol. Internet draft, IETF RFC 2326, April. 1998
[14] M. Handley, V. Jacobson. SDP: Session Description Protocol. Internet draft, IETF RFC 2327, April. 1998
[15] R. Fielding, J. Gettys, J. Mogul, et al. HTTP: Hypertext Transfer Protocol. Internet draft, IETF RFC 2616, June 1999
[16] ISO/IEC 14496-12.“Information technology-Coding of audio-visual objects-Part 12: ISO base media file format”, 2004
[17]白建兵. H.264编码技术分析.科技信息, 2007, (19): 226~228
[18]骆峰,何晖. H.264编码技术在3G移动通讯中的应用.景德镇高专学报, 2008, 23(2): 2~3
[19]孙毅,林涛,林争辉.新一代视频编码标准H.264/AVC的特点及应用.广播电视信息, 2006, 12(67): 67~68
[20]王文,陈鹏. H.264与MPEG-2标准的比较及其在广播电视领域的发展.有线电视技术, 2006, (4): 49~51
[21]魏为民.图像缩放的秘密.数码摄影, 2008, (7): 132~133
[22]王森,杨克俭.基于双线性插值的图像缩放算法.电脑编辑技巧与维护, 2008, (10): 53~67
[23]张阿珍,刘政林,邹雪城等.基于双三次插值算法的图像缩放引擎的设计.微电子学与计算机, 2007, 24(1): 49~51
[24] H S Hou, H C Andrews. Cubic splines for image interpolation and digital filtering. IEEE Trans. Acoust., Speech, Signal Processing, 1978, ASSP- 26: 508~517
[25] 3GPP TS 26.090 V5.0.0,“AMR Speech Code about Transcending functions,”June 2002
[26]吴恒. AMR语音编码技术及其在移动通信中的应用.黑龙江科技信息, 2008, (28): 74~74
[27]郭晓强,付光涛,李小雨. AAC音频压缩编码标准的ADTS与LATM格式分析.现代电视技术, 2008, (1): 140~142
[28] James Duncan Davidson. Cocoa入门——使用Objective-C.第一版.彭舰,张磊,李征等.北京:中国电力出版社, 2008. 15～22
[29]王伟.移动流媒体平台设计与实现.信息通信, 2009, (1): 63~64
[30]孙弼阳,李虹,王颖.移动流媒体业务的技术与应用,现代电信科技, 2008, (6): 11~12
[31]黄微萃,李毅,李鹏飞等. UMTS移动流媒体传输码率适配技术.广东通讯技术, 2006, 9:37~39
[32]宋秀娟,陈绍伟.一种符合3GPP标准的动态带宽适配算法.计算机测量与控制, 2008, 16(6): 856~858
[33] G. Tsirtsis. Network Address Translation - Protocol Translation. RFC2766, Feb. 2000
[34]徐娟,陈静. UDP穿透NAT的原理与实现.科技广场, 2007, (1): 69~70
[35]何琼.关于连续多媒体通信同步技术的研究.中国水运(理论版), 2006, 4(8): 86~87
[36]孙任飞,牛建伟.基于Symbian平台音视频同步算法研究与实现.济南大学学报(自然科学版), 2007, 21: 60～61
[37]潘敏,刘征宇,韩江洪.无线传输中3GPP实现基础.计算机技术与应用进展, 2007: 1552~1553
[38] Schularinne H. RTP Profile for Audio and Video Conferences with Minimal Control. RFC 1890. Feb. 1999
[39] Timo Halonen, Javier Romero, Juan Melero. GSM, GPRS and EDGE Performance: Evolution Towards 3G/UMTS.John Wiley and Sons, 2003:14~49
[40] Richard S. Wright Jr. iPhone和iPod Touch上的OpenGL ES技术.程序员, 2008, (10): 85~87
[41] F RBER Nikolaus, D HLA Stefan, ISSING Jochen. Adaptive progressive download based on the MPEG-4 file format, Journal of Zhejiang University, 2006, 7(S1): 106~111
[42]文远保,林建明.嵌入式流媒体客户端缓冲控制策略的研究.华中科技大学学报(自然科学版), 2005, 33(10): 83~85
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |