自由立体图像深度图提取算法的理论研究
|
摘要
立体图像可以记录真实的三维世界的信息,使观众产生身临其境的视觉体验,具有广阔的应用前景。目前,立体图像技术已经成功用于科研、军事、教育、工业、医疗等诸多领域,取得了丰硕的成果。然而,对于立体图像的处理需要考虑多个视点的数据,数据量非常庞大,对于当前的压缩算法,硬件处理能力和网络带宽提出了很高的要求。因此,研究一种有效的数据表达方式具有重大的意义和应用价值。
本文详细介绍了人类视觉系统的生理基础,分析了其成像原理,并结合这项研究,提取出了对于立体图像成像效果有重要影响的因素,为今后研究立体图像处理技术奠定理论基础。
欧洲"ATTEST计划”从立体电视的内容产生、编码、传输和显示的整体处理流程出发,应用了"video-plus-depth"的数据表示方法,与传统的数据表达方式相比具有明显的优势。本文深入研究了其中的关键技术即“深度图”的几种获取方法,并重点研究了基于立体像对的“深度图”提取算法。该算法以低复杂度为原则,其理论框架主要分为摄像机校正、视差匹配、视差匹配后处理(包括一致性检测和遮挡检测)、深度图计算、深度图的平滑和量化这五个部分。
最后,本文设计了四个研究立体视觉特性的实验,在HSI空间对立体图像进行处理,得出了色度对于图像立体感知效果影响较小的结论。本文进一步应用该实验结果对深度图提取算法进行了优化,提出了“灰度色度分级联合检测法”,可以在检测到多个对应点的情况下,有效地确定最佳的匹配点,有利于消除某些特殊的图像(比如场景内容周期性出现且纹理复杂的图像)中的误匹配情况。
Stereo image can record the real information of the world and provide more natural perspective effect than the traditional 2-D image, so it has more bright applicable future. At present, the stereo image processing technology has been applied to many fields successfully, such as the scientific research, the military, education, industry, medication and so on. However, as the amount of data is relatively huge compared to the 2-D image, it puts great challenge to the present compression algorithm, the processsing ability of the hardware and the network bandwidth. As a result, an effective data representation has great value in application.
Firstly, this paper introduces the physiological features of the human vision system, analyses the theory of imaging. Based on this research, it abstracts the important factors related with the display effect of the stereo image and serves as the theoretical groundwork for the future research of the stereo image processing.
The ATTEST project studies the whole processing line of 3DTV, which includes the content generation, coding, transmission and display. One of the most important contributions of it is the "video-plus-depth" concept which has great advantage over the traditional data representation. This paper studies several methods to generate the depth map and focuses on the algorithm based on the stereo image pairs. The frame of this algorithm is based on the principle of the low complexity and includes the flowing five steps:camera rectification, disparity matching, post-processing of the disparity matching, depth map calculation, the smoothing and quantization of the depth map.
In the end, this paper designs four experiments to study the features of human stereo vision system. These experiments are based on the HSI model and conclude that the color information contributes little to the stereo effect. Based on this conclusion, a new algorithm which is called "the intensity-color combined scalable matching algorithm" is proposed to optimize the above algorithm. It is especially effective in reducing the mismatching for certain images which may have periodical contents and complicated texture.
本文详细介绍了人类视觉系统的生理基础,分析了其成像原理,并结合这项研究,提取出了对于立体图像成像效果有重要影响的因素,为今后研究立体图像处理技术奠定理论基础。
欧洲"ATTEST计划”从立体电视的内容产生、编码、传输和显示的整体处理流程出发,应用了"video-plus-depth"的数据表示方法,与传统的数据表达方式相比具有明显的优势。本文深入研究了其中的关键技术即“深度图”的几种获取方法,并重点研究了基于立体像对的“深度图”提取算法。该算法以低复杂度为原则,其理论框架主要分为摄像机校正、视差匹配、视差匹配后处理(包括一致性检测和遮挡检测)、深度图计算、深度图的平滑和量化这五个部分。
最后,本文设计了四个研究立体视觉特性的实验,在HSI空间对立体图像进行处理,得出了色度对于图像立体感知效果影响较小的结论。本文进一步应用该实验结果对深度图提取算法进行了优化,提出了“灰度色度分级联合检测法”,可以在检测到多个对应点的情况下,有效地确定最佳的匹配点,有利于消除某些特殊的图像(比如场景内容周期性出现且纹理复杂的图像)中的误匹配情况。
Stereo image can record the real information of the world and provide more natural perspective effect than the traditional 2-D image, so it has more bright applicable future. At present, the stereo image processing technology has been applied to many fields successfully, such as the scientific research, the military, education, industry, medication and so on. However, as the amount of data is relatively huge compared to the 2-D image, it puts great challenge to the present compression algorithm, the processsing ability of the hardware and the network bandwidth. As a result, an effective data representation has great value in application.
Firstly, this paper introduces the physiological features of the human vision system, analyses the theory of imaging. Based on this research, it abstracts the important factors related with the display effect of the stereo image and serves as the theoretical groundwork for the future research of the stereo image processing.
The ATTEST project studies the whole processing line of 3DTV, which includes the content generation, coding, transmission and display. One of the most important contributions of it is the "video-plus-depth" concept which has great advantage over the traditional data representation. This paper studies several methods to generate the depth map and focuses on the algorithm based on the stereo image pairs. The frame of this algorithm is based on the principle of the low complexity and includes the flowing five steps:camera rectification, disparity matching, post-processing of the disparity matching, depth map calculation, the smoothing and quantization of the depth map.
In the end, this paper designs four experiments to study the features of human stereo vision system. These experiments are based on the HSI model and conclude that the color information contributes little to the stereo effect. Based on this conclusion, a new algorithm which is called "the intensity-color combined scalable matching algorithm" is proposed to optimize the above algorithm. It is especially effective in reducing the mismatching for certain images which may have periodical contents and complicated texture.
引文
[1]王元庆,自由立体显示器的应用与现状,现代显示2003(1)：39-41
[2]W.A. IJsselsteijn, P.J.H. Seuntens, L.M.J. Meesters, State of-the-art in human factors and quality issues of stereoscopic broadcast television, Deliverable ATTEST/WP5/01, Aug.2002
[3]P. Kauff, N. Atzpadin, C. Fehn, etc., Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability, Signal Processing:Image Communication,2007(22):217-234
[4]章毓晋,图像工程(下册)——图像理解与计算机视觉,北京：清华大学出版社,2000,58-62
[5]J. Hsu, Z.pizlo, C. F. Babbs, D. M. Chelberg,etc.,Design of Studies to Test the Effectiveness of Stereo Imaging Truth or Dare:Is Stereo Viewing Really Better?,Proceedings of SPIE, Stereoscopic Displays and Virtual Systems,1994(2177):211-220
[6]Alberto Del Bimbo, Leonardo Landi, and Simone Santini. Three-Dimensional Planar-faced Object Classification with Kohonen Maps. Optical Engineering.1993, 32(6):1222-1229
[7]Umesh R. Dhond. Structure from Sereo-A Review. IEEE Transactions on Systems, Man, and Cybernetics.1989,19(6):1489-1510
[8]侯春萍,平面图像立体化技术的研究：[博士学位论文],天津；天津大学,1998
[9]谭军,陆波,余桂丰,立体电视技术的发展概况及基本原理,中国有线电视,2004(12)：25-27
[10]T. Okoshi, Three-Dimensional Imaging Techniques, Academic Press,1976
[11]Sexton I., Surman P., Stereoscopic and autostereoscopic display systems, Signal Processing Magazine, IEEE Volume 16, Issue 3, May 1999:85-99
[12][Online]:http://www.tj3d.com
[13]Dodgson N.A., Autostereoscopic 3D Displays, Computer Volume 38, Issue 8, Aug.2005:31-36
[14]J.-Y. Son, V.V. Saveljev, Y.-J. Choi, J.-E. Bahn, H.-H. Choi, Parameters for designing autostereoscopic imaging systems based on lenticular, parallax barrier and IP plates, Opt. Eng.,2003, vol.42:3326-3333
[15]K. Sakamoto, H. Ueda, H. Takahashi, and E. Shimizu, An approach to 3-D real-time cylindrical solid-state display system, Proc.3D Image Conf.'96, Operating Committee of 3D Image Conf., Tokyo, Japan,1996:96-101
[16]Arakawa Y., Kakeya H., Isogai M., Suzuki K., Yamaguchi F., Space-shared communication based on truly 3D information space, Image Processing,1999. ICIP 99. Proceedings.1999 International Conference on Volume 3,24-28 Oct.1999: 31-35
[17]Ditto T., Compact 3D profilometer with grazing incidence diffraction optics, 3-D Digital Imaging and Modeling,2001. Proceedings. Third International Conference on 28 May-1 June 2001:73-80
[18]V.V.Smirnov, J.-Y. Son, H.-W. Jeon, H.-S. Lee, Y.-J. Choi, A full color reflection type holographic screen, Proc. SPIE,1998, vol.3293,175-182
[19]K. Omura, S. Shiwa, and F. Kishino, Development of lenticular stereoscopic display systems:Multiple images for multiple viewers, in SID 95 Dig.,1995, pp. 761-763
[20]张功釿,视觉原理,北京：北京大学出版社,1987,15-27
[21]郝葆源,张厚粲,陈舒永,实验心理学,北京：北京大学出版社,1983
[22]杨雄里,视觉的神经机制,上海：上海科学技术出版社,1996
[23]王光霁,双眼视觉学,北京：人民卫生出版社,2004,1-19
[24]寿天德,视觉信息处理的脑机制,上海：上海科技教育出版社,1997,3-9
[25]章毓晋。图像工程(下册)——图像理解,北京：清华大学出版社,2007
[26]Aumont J.1994.The Image.Translation:Pajackowska C.British Film Institute
[27]J.-R. Ohm, K. Gru " neberg, E. Hendriks, E. Izquierdo, D. Kalivas, M. Karl, D. Papadimatos, A. Redert, A realtime hardware system for stereoscopic videoconferencing with viewpoint adaptation, Signal Processing:Image Communication (special issue on 3D technology), November,1998,14(1-2): 147-171
[28]C. Fehn, Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV, Proceedings of the SPIE Stereoscopic Displays and Virtual Reality Systems XI, San Jose, CA, USA, January 2004:93-104
[29]A. Smolic, D. McCutchen,3DAV exploration of videobased rendering technology in MPEG, IEEE Trans. Circuits Systems Video Technol. (special issue on immersive telecommunications), March 2004,14 (3):348-356
[30]3D image processing. Three-dimensional real-time HDTV:an overview, White Paper,2005. Available at:(http://www.3d-ip.com)
[31]M. Op de Beeck and A. Redert, Three dimensional video for the home, Proceedings of the International Conference on Augmented, Virtual Environments and Three-Dimensional Imaging (ICAV3D), Mykonos, Greece,2001:188-191
[32]A. Redert, M. Op de Beeck, C. Fehn, etc., ATTEST-advanced three-dimensional television system technologies, Proceedings of the First International Symposium on 3D Data Processing, Visualization and Transmission, Padova, Italy, June 2002:313-319
[33]L. Stelmach, W.J. Tam and D. Meegan, Perceptual basis of stereoscopic video, Proceedings of the SPIE 3639:Stereoscopic Displays and Virtual Reality Systems Ⅵ,1999:260-265
[34]W.A. IJsselsteijn, H. de Ridder and J. Vliegen, Subjective evaluation of stereoscopic images:Effects of camera parameters and display duration, IEEE Transactions on Circuits and Systems for Video Technology 10,2000:225-233
[35]W.A. IJsselsteijn, J. Freeman, D.G. Bouwhuis, etc., Presence as an experiential metric for 3-D display evaluation, the Society for Information Display 2002 International Symposium, Boston, MA, USA, May,2002:19-24
[36]Y Shirai. Three-dimensional Computer Vision,1987
[37]G.J. Iddan and G. Yahav,3D Imaging in the studio (and elsewhere…), SPIE 3D SMPTE Journal, June 1994, vol.4298
[38]P. Wilinski and K. van Overveld, Depth from motion using confidence based block matching, Proceedings of Image and Multidimensional Signal Processing Workshop, Alpbach, Austria,1998:159-162
[39]Kanade T, Okutomi M., A stereo matching algorithm with an adaptive window:theory and experiment, IEEE-PAMI,1994,16(9):920-932
[40]A. Fusiello, E. Trucco, A. Verri, Rectification with unconstrained stereo geometry, in:Proceedings of the British Machine Vision Conference, Essex, UK, September 1997:400-409
[41]C. Fehn, E. Cooke, O. Schreer, etc.,3D analysis and image-based rendering for immersive TV applications, Signal Processing:Image Communication 17 (2) October 2002,17 (2):705-715
[42]C. Fehn, P. Kauff, M. Op de Beeck, etc., An evolutionary and optimised approach on 3D-TV, Proceedings of the International Broadcast Conference, Amsterdam, The Netherlands, September 2002:357-365
[43]J.-X. Chai, X. Tong, S.C. Chan, etc., Plenoptic sampling, Proceedings of the ACM SIGGRAPH, oafes, New Orleans, LA, USA, July 2000:307-318.
[44]85—04—4.1专题：HDTV主观评价图像质量的研究(研究总结)
[45]Sriram Sethuraman, stereoscopic image sequence compression using multiresolution and quadtree decomposition based disparity and motion-adaptive segmentation, doctor of philosophy in electrical engineering, Carnegie Mellon university,1996.
[46]K.Hosaka, A New Picture Quality Evaluation Method, Proc. Int. Picture Coding Symp., Tokyo, Japan, Apr.1986:17-18
[47]Z.Wang, A.C.Bovik, Why is image quality assessment so difficult, Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Proc. Orlando,2002:3313-3316
[48]H.R.Wu, M.Yuen, A generalized block-edge impairment metric for video coding, IEEE Signal Proc. Letters, Nov.1997,4(11):317-320,
[49]A.M.Eskicioglu, Quality Measurement for Monochrome Compressed Images in the past 25 years, Acoustics, Speech, and Signal Processing,2000, ICASSP'00. Proceedings,2000 IEEE International Conference, June 2000,4: 1907-1910
[50]A.M.Eskicioglu, multi-dimensional measure for image quality, Data Compression Conference Proceedings,1995
[2]W.A. IJsselsteijn, P.J.H. Seuntens, L.M.J. Meesters, State of-the-art in human factors and quality issues of stereoscopic broadcast television, Deliverable ATTEST/WP5/01, Aug.2002
[3]P. Kauff, N. Atzpadin, C. Fehn, etc., Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability, Signal Processing:Image Communication,2007(22):217-234
[4]章毓晋,图像工程(下册)——图像理解与计算机视觉,北京：清华大学出版社,2000,58-62
[5]J. Hsu, Z.pizlo, C. F. Babbs, D. M. Chelberg,etc.,Design of Studies to Test the Effectiveness of Stereo Imaging Truth or Dare:Is Stereo Viewing Really Better?,Proceedings of SPIE, Stereoscopic Displays and Virtual Systems,1994(2177):211-220
[6]Alberto Del Bimbo, Leonardo Landi, and Simone Santini. Three-Dimensional Planar-faced Object Classification with Kohonen Maps. Optical Engineering.1993, 32(6):1222-1229
[7]Umesh R. Dhond. Structure from Sereo-A Review. IEEE Transactions on Systems, Man, and Cybernetics.1989,19(6):1489-1510
[8]侯春萍,平面图像立体化技术的研究：[博士学位论文],天津；天津大学,1998
[9]谭军,陆波,余桂丰,立体电视技术的发展概况及基本原理,中国有线电视,2004(12)：25-27
[10]T. Okoshi, Three-Dimensional Imaging Techniques, Academic Press,1976
[11]Sexton I., Surman P., Stereoscopic and autostereoscopic display systems, Signal Processing Magazine, IEEE Volume 16, Issue 3, May 1999:85-99
[12][Online]:http://www.tj3d.com
[13]Dodgson N.A., Autostereoscopic 3D Displays, Computer Volume 38, Issue 8, Aug.2005:31-36
[14]J.-Y. Son, V.V. Saveljev, Y.-J. Choi, J.-E. Bahn, H.-H. Choi, Parameters for designing autostereoscopic imaging systems based on lenticular, parallax barrier and IP plates, Opt. Eng.,2003, vol.42:3326-3333
[15]K. Sakamoto, H. Ueda, H. Takahashi, and E. Shimizu, An approach to 3-D real-time cylindrical solid-state display system, Proc.3D Image Conf.'96, Operating Committee of 3D Image Conf., Tokyo, Japan,1996:96-101
[16]Arakawa Y., Kakeya H., Isogai M., Suzuki K., Yamaguchi F., Space-shared communication based on truly 3D information space, Image Processing,1999. ICIP 99. Proceedings.1999 International Conference on Volume 3,24-28 Oct.1999: 31-35
[17]Ditto T., Compact 3D profilometer with grazing incidence diffraction optics, 3-D Digital Imaging and Modeling,2001. Proceedings. Third International Conference on 28 May-1 June 2001:73-80
[18]V.V.Smirnov, J.-Y. Son, H.-W. Jeon, H.-S. Lee, Y.-J. Choi, A full color reflection type holographic screen, Proc. SPIE,1998, vol.3293,175-182
[19]K. Omura, S. Shiwa, and F. Kishino, Development of lenticular stereoscopic display systems:Multiple images for multiple viewers, in SID 95 Dig.,1995, pp. 761-763
[20]张功釿,视觉原理,北京：北京大学出版社,1987,15-27
[21]郝葆源,张厚粲,陈舒永,实验心理学,北京：北京大学出版社,1983
[22]杨雄里,视觉的神经机制,上海：上海科学技术出版社,1996
[23]王光霁,双眼视觉学,北京：人民卫生出版社,2004,1-19
[24]寿天德,视觉信息处理的脑机制,上海：上海科技教育出版社,1997,3-9
[25]章毓晋。图像工程(下册)——图像理解,北京：清华大学出版社,2007
[26]Aumont J.1994.The Image.Translation:Pajackowska C.British Film Institute
[27]J.-R. Ohm, K. Gru " neberg, E. Hendriks, E. Izquierdo, D. Kalivas, M. Karl, D. Papadimatos, A. Redert, A realtime hardware system for stereoscopic videoconferencing with viewpoint adaptation, Signal Processing:Image Communication (special issue on 3D technology), November,1998,14(1-2): 147-171
[28]C. Fehn, Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV, Proceedings of the SPIE Stereoscopic Displays and Virtual Reality Systems XI, San Jose, CA, USA, January 2004:93-104
[29]A. Smolic, D. McCutchen,3DAV exploration of videobased rendering technology in MPEG, IEEE Trans. Circuits Systems Video Technol. (special issue on immersive telecommunications), March 2004,14 (3):348-356
[30]3D image processing. Three-dimensional real-time HDTV:an overview, White Paper,2005. Available at:(http://www.3d-ip.com)
[31]M. Op de Beeck and A. Redert, Three dimensional video for the home, Proceedings of the International Conference on Augmented, Virtual Environments and Three-Dimensional Imaging (ICAV3D), Mykonos, Greece,2001:188-191
[32]A. Redert, M. Op de Beeck, C. Fehn, etc., ATTEST-advanced three-dimensional television system technologies, Proceedings of the First International Symposium on 3D Data Processing, Visualization and Transmission, Padova, Italy, June 2002:313-319
[33]L. Stelmach, W.J. Tam and D. Meegan, Perceptual basis of stereoscopic video, Proceedings of the SPIE 3639:Stereoscopic Displays and Virtual Reality Systems Ⅵ,1999:260-265
[34]W.A. IJsselsteijn, H. de Ridder and J. Vliegen, Subjective evaluation of stereoscopic images:Effects of camera parameters and display duration, IEEE Transactions on Circuits and Systems for Video Technology 10,2000:225-233
[35]W.A. IJsselsteijn, J. Freeman, D.G. Bouwhuis, etc., Presence as an experiential metric for 3-D display evaluation, the Society for Information Display 2002 International Symposium, Boston, MA, USA, May,2002:19-24
[36]Y Shirai. Three-dimensional Computer Vision,1987
[37]G.J. Iddan and G. Yahav,3D Imaging in the studio (and elsewhere…), SPIE 3D SMPTE Journal, June 1994, vol.4298
[38]P. Wilinski and K. van Overveld, Depth from motion using confidence based block matching, Proceedings of Image and Multidimensional Signal Processing Workshop, Alpbach, Austria,1998:159-162
[39]Kanade T, Okutomi M., A stereo matching algorithm with an adaptive window:theory and experiment, IEEE-PAMI,1994,16(9):920-932
[40]A. Fusiello, E. Trucco, A. Verri, Rectification with unconstrained stereo geometry, in:Proceedings of the British Machine Vision Conference, Essex, UK, September 1997:400-409
[41]C. Fehn, E. Cooke, O. Schreer, etc.,3D analysis and image-based rendering for immersive TV applications, Signal Processing:Image Communication 17 (2) October 2002,17 (2):705-715
[42]C. Fehn, P. Kauff, M. Op de Beeck, etc., An evolutionary and optimised approach on 3D-TV, Proceedings of the International Broadcast Conference, Amsterdam, The Netherlands, September 2002:357-365
[43]J.-X. Chai, X. Tong, S.C. Chan, etc., Plenoptic sampling, Proceedings of the ACM SIGGRAPH, oafes, New Orleans, LA, USA, July 2000:307-318.
[44]85—04—4.1专题：HDTV主观评价图像质量的研究(研究总结)
[45]Sriram Sethuraman, stereoscopic image sequence compression using multiresolution and quadtree decomposition based disparity and motion-adaptive segmentation, doctor of philosophy in electrical engineering, Carnegie Mellon university,1996.
[46]K.Hosaka, A New Picture Quality Evaluation Method, Proc. Int. Picture Coding Symp., Tokyo, Japan, Apr.1986:17-18
[47]Z.Wang, A.C.Bovik, Why is image quality assessment so difficult, Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Proc. Orlando,2002:3313-3316
[48]H.R.Wu, M.Yuen, A generalized block-edge impairment metric for video coding, IEEE Signal Proc. Letters, Nov.1997,4(11):317-320,
[49]A.M.Eskicioglu, Quality Measurement for Monochrome Compressed Images in the past 25 years, Acoustics, Speech, and Signal Processing,2000, ICASSP'00. Proceedings,2000 IEEE International Conference, June 2000,4: 1907-1910
[50]A.M.Eskicioglu, multi-dimensional measure for image quality, Data Compression Conference Proceedings,1995
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |