面向CIS图像处理SoC的算法、架构及复杂多媒体SoC中通信网络研究
|
摘要
CMOS图像传感器随着CMOS制造工艺的成熟在半导体成像领域迅速发展,但工艺尺寸的缩小使得成像质量下降,因此,图像处理片上系统(System-on-Chip, SoC)随着CMOS图像传感器的成熟和发展成为产业界和学术界的研究热点,其中图像处理流水线设计、关键图像处理算法设计以及图像处理SoC架构设计是图像处理SoC设计中的重点。系统规模的增大使得基于共享总线的SoC芯片设计中存在通信、延迟以及设计效率等问题,片上网络(Network-on-Chip, NoC)为未来大规模的复杂多媒体SoC提供了有效的片上通信解决方案。由于目前单个FPGA芯片无法提供足够的资源对基于NoC的复杂多媒体SoC系统进行硬件验证,需要研究基于多FPGA平台的NoC通信网络的硬件评测平台。
本文的主要研究内容包括面向CMOS图像传感器的基于共享总线架构的图像处理SoC系统设计以及基于多FPGA平台的复杂多媒体SoC中的NoC通信网络的硬件评测平台设计。论文的主要研究成果如下:
1.针对传统自动曝光过程缓慢的缺点,提出了一种应用于CMOS图像传感器的快速自动曝光控制方法。此方法基于CMOS器件的感光特性,建立数学模型实现算法,能够克服传统固定步长自动曝光方法曝光过程缓慢的缺点。通过算法验证,所提出的算法在各种光照情况下与传统的自动曝光算法相比,速度有很大的提升,同时能够保证精度受较小影响。
2.为了保证插补后图像的PSNR性能并同时降低算法的复杂度,本文提出了一种基于自适应边缘敏感和模糊分配的改进型CFA插补算法。所提出的CFA插补算法增加了邻近像素的边缘判断作为方向依据并将边缘分为强边缘和弱边缘分别进行插补,对G通道的插补基于自适应模糊分配算法并根据本文的强边缘和弱边缘进行了算法修正,根据边缘判断的结果分别赋予不同的模糊分配系数。所提出的CFA插补算法提高了G通道插补时的PSNR性能和图像质量,同时同复杂的插补算法相比计算量减小,更易于在实际中应用。
3.由于单个FPGA芯片的资源不足以满足未来大规模的图像、视频等复杂多媒体SoC的验证工作,针对此问题本文基于多FPGA平台设计了一个面向复杂多媒体SoC系统的可扩展NoC通信网络的硬件评测系统,该系统基于二维网格拓扑结构的HERMES NoC,仅比HERMES NoC消耗少量额外的资源,能够
有效地完成多FPGA平台中任意路由单元之间的通信,并且支持多媒体处理的单播传输模式和组播传输模式。实验结果证明,所设计的可扩展NoC硬件评测系统能够作为复杂多媒体SoC系统的硬件评测和验证系统。
The mature CMOS process technology leads to the development and growth of CMOS image sensor, however, the image quality gets worse as the semiconductor technology scales down. Thus, the image processing system-on-chip (SoC) for CMOS image sensor has become more and more important. Design of image processing pipeline, the key image processing algorithms and the architecture of image processing SoC are the most critical aspects. However, with the augmentation scale of hardware system, the traditional shared bus based SoC has problems in delay, communication performance bottleneck and design efficiency as, Network-on-chip offers an efficent solution for connecting modules of an ultra-large scale hardware application. NoC emulation using FPGA is widely used for evaluating the performances of the communication, but this experimental approach suffers from scalability issue, mainly due to the resource limitation of FPGA. Multi-FPGA based NoC hardware emulation systems are proposed in this thesis to solve this problem.
In this thesis, FPGA-based hardware architecture design for traditional bus shared image processing SoC with CMOS image sensor, as well as multi-FPGA based NoC hardware emulation platform design for the complex multimidea SoC application is investigated. The main works of this thesis are follows:
Firstly, we proposed a fast auto exposure control method against the slow process of the conventional fixed-step auto exposure control method for CMOS image sensor. The ratio of the output signal after exposure to the product of gain value and exposure time is a constant under the same illumination. According to this characteristic, we obtained the luminous intensity using the current exposure time, gain and brightness of the object. By using the light look-up table, we got the optimal exposure time and realized the fast auto exposure control method. The proposed method can be applied to CMOS image sensor well when verified on FPGA board. As it turned out, this algorithm can ensure exposure correctly and adjust the exposure time and gain value fast.
Moreover, we proposed a CFA demosaicing algorithm which improves on adaptive edge sensitive algorithm and fuzzy assignment algorithm. In order to estimate the direction of edges more accurately, it adds two adjacent pixels of the current pixel as the judgment condition and classifies the edges into strong edges and weak edges. The proposed algorithm adopts fuzzy assignment algorithm to estimate the missing green value at red/blue pixels and assigns different weighting factors according to the results of edge detection. The proposed algorithm can get similar PSNRs without affecting the quality of the image and at the same time reduce the computational cost.
Finally, we proposed a multi-FPGA based scalable NoC emulation system in the context of the single FPGA’s resource limitation for verification of large scale complex multimidea SoC. The proposed scalable NoC on multi-FPGA based platform from existing 2D mesh NoC using the packet-switching technique. We propose a routing algorithm considering inter-FPGA and intra-FPGA communications based on existing 2D routing algorithms. The effectivenesses of the synthesizable hardware NoC emulation system are that it can be used for any multi-FPGA based NoC with small amount extra resource consumption and can support the unicast and multicast transmission mode.
本文的主要研究内容包括面向CMOS图像传感器的基于共享总线架构的图像处理SoC系统设计以及基于多FPGA平台的复杂多媒体SoC中的NoC通信网络的硬件评测平台设计。论文的主要研究成果如下:
1.针对传统自动曝光过程缓慢的缺点,提出了一种应用于CMOS图像传感器的快速自动曝光控制方法。此方法基于CMOS器件的感光特性,建立数学模型实现算法,能够克服传统固定步长自动曝光方法曝光过程缓慢的缺点。通过算法验证,所提出的算法在各种光照情况下与传统的自动曝光算法相比,速度有很大的提升,同时能够保证精度受较小影响。
2.为了保证插补后图像的PSNR性能并同时降低算法的复杂度,本文提出了一种基于自适应边缘敏感和模糊分配的改进型CFA插补算法。所提出的CFA插补算法增加了邻近像素的边缘判断作为方向依据并将边缘分为强边缘和弱边缘分别进行插补,对G通道的插补基于自适应模糊分配算法并根据本文的强边缘和弱边缘进行了算法修正,根据边缘判断的结果分别赋予不同的模糊分配系数。所提出的CFA插补算法提高了G通道插补时的PSNR性能和图像质量,同时同复杂的插补算法相比计算量减小,更易于在实际中应用。
3.由于单个FPGA芯片的资源不足以满足未来大规模的图像、视频等复杂多媒体SoC的验证工作,针对此问题本文基于多FPGA平台设计了一个面向复杂多媒体SoC系统的可扩展NoC通信网络的硬件评测系统,该系统基于二维网格拓扑结构的HERMES NoC,仅比HERMES NoC消耗少量额外的资源,能够
有效地完成多FPGA平台中任意路由单元之间的通信,并且支持多媒体处理的单播传输模式和组播传输模式。实验结果证明,所设计的可扩展NoC硬件评测系统能够作为复杂多媒体SoC系统的硬件评测和验证系统。
The mature CMOS process technology leads to the development and growth of CMOS image sensor, however, the image quality gets worse as the semiconductor technology scales down. Thus, the image processing system-on-chip (SoC) for CMOS image sensor has become more and more important. Design of image processing pipeline, the key image processing algorithms and the architecture of image processing SoC are the most critical aspects. However, with the augmentation scale of hardware system, the traditional shared bus based SoC has problems in delay, communication performance bottleneck and design efficiency as, Network-on-chip offers an efficent solution for connecting modules of an ultra-large scale hardware application. NoC emulation using FPGA is widely used for evaluating the performances of the communication, but this experimental approach suffers from scalability issue, mainly due to the resource limitation of FPGA. Multi-FPGA based NoC hardware emulation systems are proposed in this thesis to solve this problem.
In this thesis, FPGA-based hardware architecture design for traditional bus shared image processing SoC with CMOS image sensor, as well as multi-FPGA based NoC hardware emulation platform design for the complex multimidea SoC application is investigated. The main works of this thesis are follows:
Firstly, we proposed a fast auto exposure control method against the slow process of the conventional fixed-step auto exposure control method for CMOS image sensor. The ratio of the output signal after exposure to the product of gain value and exposure time is a constant under the same illumination. According to this characteristic, we obtained the luminous intensity using the current exposure time, gain and brightness of the object. By using the light look-up table, we got the optimal exposure time and realized the fast auto exposure control method. The proposed method can be applied to CMOS image sensor well when verified on FPGA board. As it turned out, this algorithm can ensure exposure correctly and adjust the exposure time and gain value fast.
Moreover, we proposed a CFA demosaicing algorithm which improves on adaptive edge sensitive algorithm and fuzzy assignment algorithm. In order to estimate the direction of edges more accurately, it adds two adjacent pixels of the current pixel as the judgment condition and classifies the edges into strong edges and weak edges. The proposed algorithm adopts fuzzy assignment algorithm to estimate the missing green value at red/blue pixels and assigns different weighting factors according to the results of edge detection. The proposed algorithm can get similar PSNRs without affecting the quality of the image and at the same time reduce the computational cost.
Finally, we proposed a multi-FPGA based scalable NoC emulation system in the context of the single FPGA’s resource limitation for verification of large scale complex multimidea SoC. The proposed scalable NoC on multi-FPGA based platform from existing 2D mesh NoC using the packet-switching technique. We propose a routing algorithm considering inter-FPGA and intra-FPGA communications based on existing 2D routing algorithms. The effectivenesses of the synthesizable hardware NoC emulation system are that it can be used for any multi-FPGA based NoC with small amount extra resource consumption and can support the unicast and multicast transmission mode.
引文
[1] A.Theuwissen, CMOS image sensors: State-of-the-art. Solid-State Electronics, 2008. 52(9): p. 1401-1406.
[2] El Gamal A, E.H., CMOS image sensors. IEEE Circuits and Devices Magazine, 2005. 21(3): p. 6-20.
[3] http://www.imagesensors.org/.
[4] Bigas M., C.E., Forest J., et al., Review of CMOS image sensors. Microelectronics Journal, 2006. 37(5): p. 433-451.
[5] Vanhorebeek, G.,低成本CMOS图像传感器推动医学技术向前发展.电子产品世界, 2007. C00: p. 132-135.
[6]陈慧敏,栗苹, CMOS图像传感器的研究新进展.半导体光电, 2006. 27(6): p. 664-667.
[7]程开富, CMOS图像传感器技术在军事中的应用.电子元器件应用, 2006. 8(7): p. 98-100.
[8]尤政,李涛, CMOS图像传感器在空间技术中的应用.光学技术, 2002. 28(1): p. 31-35.
[9] K. Kimo, P.I.-C., Combined image signal processing for CMOS image sensors. IEEE International Symposium on Circuits and Systems, 2006: p. 3185-3188.
[10] Wei Miao, Q.L., Wancheng Zhang et al., A Programmable SIMD Vision Chip for Real-Time Vision Applications. IEEE Journal of Solid-State Circuits, 2008. 43: p. 1470-1479.
[11] Lukac, R., Single-sensor imaging in consumer digital cameras: a survey of recent advances and future directions. Journal of real-time image processing, 2005. 1(1): p. 45-52.
[12] Y. Kwangho, K.C., L. Bumha et al., Single-chip CMOS image sensor for mobile applications. IEEE Journal of Solid-State Circuits, 2002. 37: p. 1839-1845.
[13] J. Yun Ho, K.J.S., H. Bong Soo, et al., Design of real-time image enhancement preprocessor for CMOS image sensor. IEEE Transactions on Consumer Electronics, 2000. 46: p. 68-75.
[14] Weerasinghe, C., L. Wanqing, et al., Novel color processing architecture for digital cameras with CMOS image sensors. IEEE Transactions on Consumer Electronics, 2005. 51(4): p. 1092-1098.
[15] R. Ramanath, W.E.S., Y. Yoo et al., Color image processing pipeline. IEEE Signal Processing Magazine, 2005. 22: p. 34-43.
[16] A. Gentile, S.V., L. Verdoscia et al., Image processing chain for digital still cameras based on the SIMPil architecture. International Conference on Parallel Processing, 2005: p. 215-222.
[17] K. Wen-Chung, W.S.-H., C. Lien-Yang et al., Design considerations of color image processing pipeline for digital cameras. IEEE Transactions on Consumer Electronics, 2006. 52: p. 1144-1152.
[18] Glotzbach, J.Z.a.J., Image Pipeline Tuning for Digital Cameras. IEEE International Symposium on Consumer Electronics, 2007: p. 1-4.
[19] Etienne-Cummings, O.Y.-P.a.R., CMOS Imagers: From Phototransduction to Image Processing. New York: Kluwer Academic Publishers, 2004: p. 102-113.
[20] Illgner K. Gruber, H.-G.G., P. Jie Liang et al., Programmable DSP platform for digital still cameras. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1999: p. 2235-2238.
[21] Bayer, B.E., Color imaging array. U.S. Patent, 3971065. July 1976.
[22] Chen, J.C.a.C.S.-Y., CRISP: Coarse-Grained Reconfigurable Image Stream Processor for Digital Still Cameras and Camcorders. IEEE Transactions on Circuits and Systems for Video Technology, 2008. 18(9): p. 1223-1236.
[23] M.H. Sunwoo, O.S., A. Byungdug et al., Design and implementation of a parallel image processor chip for a SIMD array processor. Proceedings of International Conference on Application Specific Array Processors, 1995: p. 66-75.
[24] Smith, M.J.S., Application-Specific Integrated Circuits. USA: Addison-Wesley Professional, 1997: p. 1-30.
[25] N. Nakano, R.N., H. Sai, et al., Digital still camera system for megapixel CCD. IEEE Transactions on Consumer Electronics, 1998. 44(3): p. 581-586.
[26] H. Zen, T.K., H. Yamamoto et al., A new digital signal processor for progressive scan CCD. IEEE Transactions on Consumer Electronics, 1998. 4(2): p. 289-296.
[27] R. Zhou, X.C., F. Liu, et al., System-on-chip for mega-pixel digital camera processor with auto control functions. Proceedings on 5th International Conference on ASIC, 2003. 2: p. 894-897.
[28] P.S. Mandolesi, P.J., and A.G. Andreou, A simplicial CNN architecture for on-chip image processing. Proceedings of the 2004 International Symposium on Circuits and Systems, 2004. 3: p. 29-32.
[29] Etienne-Cummings, R.K., Z.K. Donghui Cai, A programmable focal-plane mimd image processor chip. IEEE Journal of Solid-State Circuits, 2001. 36(1): p. 64-73.
[30] Henry Chang, L.C., Merrill Hunt, et al., Surviving the SoC Revolution: a Guide to Platform-Based Design. New York: Kluwer Academic Publishers, 1999: p. 30-100.
[31]郭炜, SoC设计方法与实现.北京:电子工业出版社, 2005: p. 50-75.
[32]武畅,片上网络体系结构和关键通信技术研究:[博士学位论文],成都;电子科技大学,2008.
[33] Singh H., L.M.-H., et al., MorphoSys: an integrated reconfigurable system for data-parallel and computation-intensive applications. IEEE Transactions on Computers, 2000. 49(5): p. 465-481.
[34] Masselos, N.S.V.a.K., System level design of reconfigurable systems-on-chip. USA: Springer, 2005: p. 15-26.
[35] Stamatis Vassiliadis, D.S., Yale Patt, et al., Fine- and Coarse-Grain Reconfigurable Computing. USA: Springer, 2007: p. 3-27.
[36] L. Benini, G.D.M., Networks on Chips: A New SoC Paradigm. IEEE Computer, 2002. 35(1): p. 70-78.
[37] International Technology Roadmap for Semiconductors, International Technology Roadmap for Semiconductors 2008 Edition: Interconnect. 2008, http://www.itrs.net/Links/ 2008ITRS/Interconnect2008.pdf
[38] W.Wolf, The future of multiprocessor systems-on-chips. Proceedings of the 41st Design Automation Conference, 2004: p. 681-686.
[39] Dally W.J., T.B., Route packets, not wires: on-chip interconnection networks. Design Automation Conference, 2001: p. 684-689.
[40] H., Z., OSI Reference Model-The ISO Model of Architecture for Open Systems Interconnection. IEEE Transactions on Communications, 1980. 28(4): p. 425-432.
[41] F. Moraes, N.C., A. Mello et al., HERMES: an infrastructure for low area overhead packet-switching networks on chip. INTEGRATION, the VLSI Journal, 2004. 38(1): p. 69 - 93.
[42]马立伟,专用片片上网络设计方法:通信建模、拓扑构造与自动生成:[博士学位论文],北京;清华大学,2006.
[43] Dillon P.L.P., L.D.M., Kaspar F.G., Color imaging system using a single CCD area array. IEEE Transactions on Electron Devices, 1978. 25(2): p. 102-107.
[44] Nabeyama H., S.N., et al., All Solid State Color Camera With Single-Chip Mos Imager. IEEE Transactions on Consumer Electronics, 1981. 27(1): p. 40-46.
[45] S., K., On VLSI array architectures for digital image processing. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1986: p. 3113-3126.
[46] J., T.H., DSP solutions run the gamut for color systems. IEEE Signal Processing Magazine, 1993. 10(2): p. 8-23.
[47] Trussell, S.G.a.H.J., Digital color imaging. IEEE Transactions on Image Processing, 1997. 6(7): p. 901-932.
[48] Adams J., K.P., et al., Color processing in digital cameras. IEEE Micro, 1998.18(6): p. 20-30.
[49] Doswald D., J.H., et al., A 30-frames/s megapixel real-time CMOS image processor. IEEE Journal of Solid-State Circuits, 2000. 35(11): p. 1732-1743.
[50] Battiato, S., A. Castorina, et al., A global enhancement pipeline for low-cost imaging devices. IEEE Transactions on Consumer Electronics, 2003. 49(3): p. 670-675.
[51] http://www.canon.com/technology/interview/digic4.
[52] http://www.sony.com.sg/microsite/dslr/technologies/bionz.html.
[53] http://www.cypress.com/.
[54] http://www.aptina.com/products/image_processors_soc/.
[55] http://www.ovt.com/.
[56] Hyung Gyu Lee, U.Y.O., Radu Marculescu, et al., Design space exploration and prototyping for on-chip multimedia applications. 43rd ACM/IEEE Design Automation Conference, 2006: p. 137-142.
[57] Soares, A.B., Carro, L. and Susin, A.A., Reconfigurable communications for image processing applications. 20th International Parallel and Distributed Processing Symposium, 2006: p. 1-4.
[58] Joshi, J., Karandikar, K., Bade, S, et al., Multi-core Image processing system using Network on Chip interconnect. 50th Midwest Symposium on Circuits and Systems, 2007: p. 1257-1260.
[59] A. Beyranvand Nejad, K.G.W.G., J. Walters, et al., Mapping KPN Models of Streaming Applications on A Network-on-Chip Platform. Program for Research on Integrated Systems and Circuits, 2009: p. 441-444.
[60] Thuan Le and Khalid, M., NoC prototyping on FPGAs: A case study using an image processing benchmark. IEEE International Conference on Electro/Information Technology, 2009: p. 441-445.
[61] Yunjie Wu, H.D.a.H.S., A Programming Model and a NoC-Based Architecture for Streaming Applications. 13th Euromicro Conference on Digital System Design: Architectures, Methods and Tools, 2010: p. 393-397.
[62] http://www.vimicro.com.cn/.
[63]周荣政,数码相机处理器设计及系统集成研究:[博士学位论文],上海;复旦大学,2004.
[64] http://www.ime.cas.cn/jgsz/kybm/txydmt/yjsjj/.
[65]汪宿梁,面向多媒体图像处理的高效可重构协处理器设计:[硕士学位论文],上海;上海交通大学,2008.
[66]姚于斌,面向图像处理的可重构协处理器结构设计研究:[硕士学位论文],上海;上海交通大学,2008.
[67]卢明,基于DSP和FPGA的图像采集与处理系统设计:[硕士学位论文],西安;西安电子科技大学,2006.
[68]王强,一种实时图像处理硬件平台的设计与实现:[硕士学位论文],北京;北京交通大学,2009.
[69]庞业勇,基于FPGA的图像处理系统设计方法研究:[硕士学位论文],哈尔滨;哈尔滨工业大学,2010.
[70]戈志伟,基于片上总线的图像处理芯片设计,[硕士学位论文],天津;天津大学,2009.
[71]张钰,监控图像SoC中数字图像处理器关键技术研究:[博士学位论文],天津;天津大学,2009.
[72]钟健,图像处理片上系统:[博士学位论文],天津;天津大学,2010.
[73] Yo-Hwan Noh, G.-d., Method And Apparatus For Compensating Black Level By Dark Current Of Image Sensor. U.S. Patent, US20080192130. Aug. 14, 2008.
[74] Lam, E.Y., Combining gray world and retinex theory for automatic white balance in digital photography. Proceedings of the Ninth International Symposium on Consumer Electronics, 2005: p. 134-139.
[75] M. Nilsson, C.W., Design and implementation of a CMOS sensor based video camera incorporating a combined AWB/AEC module. The 2003 IEEE International Conference on Acoustics, Speech and Signal Processing, 2003. II: p. 477-480.
[76] K. Wen-Chung, H.C.-C., Adaptive exposure control and real-time image fusion for surveillance systems. International Symposium on Circuits and Systems, 2006: p. 935-938.
[77] Lei Zhang, X.W., David Zhang, Color Reproduction From Noisy CFA Data of Single Sensor Digital Cameras. IEEE Transactions on Image Processing, 2007. 16(9): p. 2184-2197.
[78] Lei Zhang, W.D., Xiaolin Wu etc., Spatial-Temporal Color Video Reconstruction From Noisy CFA Sequence. IEEE Transactions on Circuits and Systems for Video Technology, 2010. 20(6): p. 838-847.
[79] Sharma, G., Digital Color Imaging Handbook. Boca Raton: CRC Press LLC, 2003: p. 743-750.
[80] Seok-Han Lee, T.-Y.K., and Jong-Soo Choi, A Color Correction System using a Color Compensation Chart. International Conference on Hybrid Information Technology, 2006. 1: p. 409 - 416.
[81] Rafael C. Gonzalez, R.E.W., Digital Image Processing. Addison-Wesley Pub Co., USA, 2002: p. 89-303.
[82]邱立诚, Algorithm Design and Application for Digital Still Camera:[博士学位论文],台湾;台湾大学, 2009, p] 15-16.
[83] Ge Zhiwei, Y.S., Xu Jiangtao, Field-programmable gate array-based hardware architecture for image processing with complementary metal-oxide-semiconductor image sensor. Journal of Electronic Imaging, 2010. 19(3): p. 033014-1 - 033014-11.
[84] T. Kuno, H.S., and N. Matoba, A new automatic exposure system for digital still cameras. IEEE Transactions on Consumer Electronics, 1998. 44(4): p. 192-199.
[85] Nitin Sampat, S.V., System implications of implementing auto-exposure on consumer digital cameras. SPIE Conference on Sensors, Cameras, and Applications for Digital Photography, 1999: p. 93-99.
[86] Myung Hee Cho, S.G.L., and Byung Deok Nam, The fast auto exposure algorithm based on the numerical analysis. SPIE Conference on Sensors, Cameras, and Applications for Digital Photography, 1999: p. 120-128.
[87]戈志伟,姚素英,徐江涛,等,一种应用于CMOS图像传感器的快速自动曝光控制方法.天津大学学报, 2010. 43(10): p. 854 - 859.
[88] Yang, D.X.D., Digital Pixel CMOS Image Sensors:[Ph.D Dissertation]. California: Deparment of Electrical Engineering, Stanford University, 1999.
[89] Hong, C.S., On-Chip Spatial Image Processing with CMOS Active Pixel Sensors[Ph.D Dissertation]. Ontario, CA: University of Waterloo, 2001.
[90] JiaYi Liang, Y.Q., ZhiLiang Hong, An auto-exposure algorithm for detecting high contrast lighting conditions. Proceedings of IEEE International Conference on ASIC, 2007: p. 1114-1117.
[91] K. Wen-Chung, C.S.-H., An integrated software architecture for real-time video and audio recording systems. IEEE Transactions on Consumer Electronics, 2005. 51: p. 879-884.
[92] Robert, H., Method of Detecting Flicker, and Video Camera Using The Method. U.S. Patent, 6900834. May 31, 2005.
[93]贡青,李锦萍,闵子建,等,一种基于CMOS图像传感器的flicker自动检测方法.首都师范大学学报, 2007. 28(2): p. 32-35.
[94] James E. Adams, J., Interactions between color plane interpolation and other image processing functions in electronic photography. Proceedings of the SPIE Electronic Imaging Conference, 1995. 2416: p. 144 - 151.
[95] Tam, S.C.P.a.I.-K., Effective color interpolation in CCD color filter array using signal correlation. IEEE Transactions on Circuits and Systems for Video Technology, 2003. 13(6): p. 503-513.
[96] Adams, J.F.H.J.a.J.E., Adaptive color plane interpolation in single color electronic camera. U.S. Patent, 5629734. May 1997.
[97] P. S. Tsai, T.A., Ajay K.Ray, Adaptive Fuzzy Color Interpolation. Journal of Electronic Imaging, 2002. 11: p. 1-24.
[98] Wenmiao Lu, Y.-P.T., Color filter array demosaicing: New method andperformance measures. IEEE Transactions on Image Processing, 2003. 12(10): p. 1194 - 1210.
[99] Li, X., Demosaicing by Successive Approximation. IEEE Transactions on Image Processing, 2005. 14(3): p. 370 - 379.
[100] Alleysson D., S.S., Herault J., Linear demosaicing inspired by the human visual system. IEEE Transactions on Image Processing, 2005. 14(4): p. 439 - 449.
[101] King-Hong Chung, Y.-H.C., Color Demosaicing Using Variance of Color Differences. IEEE Transactions on Image Processing, 2006. 15(10): p. 2944 - 2955.
[102] Lei Zhang, X.W., Color demosaicking via directional linear minimum mean square-error estimation. IEEE Transactions on Image Processing, 2005. 14(12): p. 2167 - 2178.
[103] Ferradans S., B.M., Caselles V., Geometry-Based Demosaicking. IEEE Transactions on Image Processing, 2009. 18(3): p. 665 - 670.
[104] Kuo-Liang Chung, W.-J.Y., Wen-Ming Yan, Chung-Chou Wang, Demosaicing of Color Filter Array Captured Images Using Gradient Edge Detection Masks and Adaptive Heterogeneity-Projection. IEEE Transactions on Image Processing, 2008. 17(12): p. 2356 - 2367.
[105] Ge Zhiwei, Y.S., Xu Jiangtao, A demosaicing algorithm based on adaptive edge sensitive and fuzzy assignment in CMOS image sensor. The 2nd International Conference on Signal Processing Systems, 2010. 1: p. V1-420-V1-424.
[106] Sugiura., T.K.a.H., New interpolation method using discriminated color correlation for digital still cameras. IEEE Transactions on Consumer Electronics, 1999. 45(1): p. 259 - 267.
[107]单宝堂,沈庭芝,闫雪梅,等,自适应比例合成反马赛克算法的提出与应用.北京理工大学学报, 2007. 27(11): p. 1013 - 1016.
[108] IBM, 128-Bit Processor Local Bus Architecture Specifications. https://www-01.ibm.com/ chips/techlib/techlib.nsf/techdocs/3BBB27E5BCC165BA 87256A2B0064FFB4/$file/PlbBus_as_01_pub.pdf, 2007.
[109] ARM, AMBA? Specification. http://polimage.polito.it/~lavagno/esd/ IHI0011A_AMBA_ SPEC.pdf, 1999.
[110] Corporation, S., WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. http://www.cerc.utexas.edu/ ~jypark/2011_spring_VLSI_I/LAB3_Website /lab3b/wbspec_b1.pdf, 2001.
[111] Ge Zhiwei, Y.S., Xu Jiangtao, Design of on-chip image processing based on APB bus with CMOS image sensor. IEEE 8th International Conference on ASIC, 2009: p. 963-966.
[112]董德兴,高性能CIS控制系统及关键ISP技术研究,[硕士学位论文],天津;天津大学,2009.
[113] Takashi Miyamori, K.O., REMARC : Reconfigurable Multimedia Array Coprocessor. IEICE transactions on information and systems, 1999. E82-D(2): p. 389-397.
[114] Rossi D., C., F, Spolzino, S, etc., A Heterogeneous Digital Signal Processor for Dynamically Reconfigurable Computing. IEEE Journal of Solid-State Circuits, 2010. 45(8): p. 1615-1626.
[115] Claudia Rusu, L.A., Dimiter Avresky, RILM: Reconfigurable inter-layer routing mechanism for 3D multi-layer networks-on-chip. IEEE 16th International On-Line Testing Symposium, 2010: p. 121-126.
[116] Zhiwei GE, J.T., Virginie FRESSE, etc., A Scalable and Effective Routing Algorithm for Multi-FPGA Based Large Scale NoC. Le Groupement de Recherche System-On-Chip et System-In-Package, 2011. P4: p. 1 - 2.
[117] Leonel Tedesco, A.M., Diego Garibotti, etc., Traffic Generation and Performance Evaluation for Mesh-based NoCs. 18th Symposium on Integrated Circuits and Systems Design, 2005: p. 184 - 189.
[2] El Gamal A, E.H., CMOS image sensors. IEEE Circuits and Devices Magazine, 2005. 21(3): p. 6-20.
[3] http://www.imagesensors.org/.
[4] Bigas M., C.E., Forest J., et al., Review of CMOS image sensors. Microelectronics Journal, 2006. 37(5): p. 433-451.
[5] Vanhorebeek, G.,低成本CMOS图像传感器推动医学技术向前发展.电子产品世界, 2007. C00: p. 132-135.
[6]陈慧敏,栗苹, CMOS图像传感器的研究新进展.半导体光电, 2006. 27(6): p. 664-667.
[7]程开富, CMOS图像传感器技术在军事中的应用.电子元器件应用, 2006. 8(7): p. 98-100.
[8]尤政,李涛, CMOS图像传感器在空间技术中的应用.光学技术, 2002. 28(1): p. 31-35.
[9] K. Kimo, P.I.-C., Combined image signal processing for CMOS image sensors. IEEE International Symposium on Circuits and Systems, 2006: p. 3185-3188.
[10] Wei Miao, Q.L., Wancheng Zhang et al., A Programmable SIMD Vision Chip for Real-Time Vision Applications. IEEE Journal of Solid-State Circuits, 2008. 43: p. 1470-1479.
[11] Lukac, R., Single-sensor imaging in consumer digital cameras: a survey of recent advances and future directions. Journal of real-time image processing, 2005. 1(1): p. 45-52.
[12] Y. Kwangho, K.C., L. Bumha et al., Single-chip CMOS image sensor for mobile applications. IEEE Journal of Solid-State Circuits, 2002. 37: p. 1839-1845.
[13] J. Yun Ho, K.J.S., H. Bong Soo, et al., Design of real-time image enhancement preprocessor for CMOS image sensor. IEEE Transactions on Consumer Electronics, 2000. 46: p. 68-75.
[14] Weerasinghe, C., L. Wanqing, et al., Novel color processing architecture for digital cameras with CMOS image sensors. IEEE Transactions on Consumer Electronics, 2005. 51(4): p. 1092-1098.
[15] R. Ramanath, W.E.S., Y. Yoo et al., Color image processing pipeline. IEEE Signal Processing Magazine, 2005. 22: p. 34-43.
[16] A. Gentile, S.V., L. Verdoscia et al., Image processing chain for digital still cameras based on the SIMPil architecture. International Conference on Parallel Processing, 2005: p. 215-222.
[17] K. Wen-Chung, W.S.-H., C. Lien-Yang et al., Design considerations of color image processing pipeline for digital cameras. IEEE Transactions on Consumer Electronics, 2006. 52: p. 1144-1152.
[18] Glotzbach, J.Z.a.J., Image Pipeline Tuning for Digital Cameras. IEEE International Symposium on Consumer Electronics, 2007: p. 1-4.
[19] Etienne-Cummings, O.Y.-P.a.R., CMOS Imagers: From Phototransduction to Image Processing. New York: Kluwer Academic Publishers, 2004: p. 102-113.
[20] Illgner K. Gruber, H.-G.G., P. Jie Liang et al., Programmable DSP platform for digital still cameras. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1999: p. 2235-2238.
[21] Bayer, B.E., Color imaging array. U.S. Patent, 3971065. July 1976.
[22] Chen, J.C.a.C.S.-Y., CRISP: Coarse-Grained Reconfigurable Image Stream Processor for Digital Still Cameras and Camcorders. IEEE Transactions on Circuits and Systems for Video Technology, 2008. 18(9): p. 1223-1236.
[23] M.H. Sunwoo, O.S., A. Byungdug et al., Design and implementation of a parallel image processor chip for a SIMD array processor. Proceedings of International Conference on Application Specific Array Processors, 1995: p. 66-75.
[24] Smith, M.J.S., Application-Specific Integrated Circuits. USA: Addison-Wesley Professional, 1997: p. 1-30.
[25] N. Nakano, R.N., H. Sai, et al., Digital still camera system for megapixel CCD. IEEE Transactions on Consumer Electronics, 1998. 44(3): p. 581-586.
[26] H. Zen, T.K., H. Yamamoto et al., A new digital signal processor for progressive scan CCD. IEEE Transactions on Consumer Electronics, 1998. 4(2): p. 289-296.
[27] R. Zhou, X.C., F. Liu, et al., System-on-chip for mega-pixel digital camera processor with auto control functions. Proceedings on 5th International Conference on ASIC, 2003. 2: p. 894-897.
[28] P.S. Mandolesi, P.J., and A.G. Andreou, A simplicial CNN architecture for on-chip image processing. Proceedings of the 2004 International Symposium on Circuits and Systems, 2004. 3: p. 29-32.
[29] Etienne-Cummings, R.K., Z.K. Donghui Cai, A programmable focal-plane mimd image processor chip. IEEE Journal of Solid-State Circuits, 2001. 36(1): p. 64-73.
[30] Henry Chang, L.C., Merrill Hunt, et al., Surviving the SoC Revolution: a Guide to Platform-Based Design. New York: Kluwer Academic Publishers, 1999: p. 30-100.
[31]郭炜, SoC设计方法与实现.北京:电子工业出版社, 2005: p. 50-75.
[32]武畅,片上网络体系结构和关键通信技术研究:[博士学位论文],成都;电子科技大学,2008.
[33] Singh H., L.M.-H., et al., MorphoSys: an integrated reconfigurable system for data-parallel and computation-intensive applications. IEEE Transactions on Computers, 2000. 49(5): p. 465-481.
[34] Masselos, N.S.V.a.K., System level design of reconfigurable systems-on-chip. USA: Springer, 2005: p. 15-26.
[35] Stamatis Vassiliadis, D.S., Yale Patt, et al., Fine- and Coarse-Grain Reconfigurable Computing. USA: Springer, 2007: p. 3-27.
[36] L. Benini, G.D.M., Networks on Chips: A New SoC Paradigm. IEEE Computer, 2002. 35(1): p. 70-78.
[37] International Technology Roadmap for Semiconductors, International Technology Roadmap for Semiconductors 2008 Edition: Interconnect. 2008, http://www.itrs.net/Links/ 2008ITRS/Interconnect2008.pdf
[38] W.Wolf, The future of multiprocessor systems-on-chips. Proceedings of the 41st Design Automation Conference, 2004: p. 681-686.
[39] Dally W.J., T.B., Route packets, not wires: on-chip interconnection networks. Design Automation Conference, 2001: p. 684-689.
[40] H., Z., OSI Reference Model-The ISO Model of Architecture for Open Systems Interconnection. IEEE Transactions on Communications, 1980. 28(4): p. 425-432.
[41] F. Moraes, N.C., A. Mello et al., HERMES: an infrastructure for low area overhead packet-switching networks on chip. INTEGRATION, the VLSI Journal, 2004. 38(1): p. 69 - 93.
[42]马立伟,专用片片上网络设计方法:通信建模、拓扑构造与自动生成:[博士学位论文],北京;清华大学,2006.
[43] Dillon P.L.P., L.D.M., Kaspar F.G., Color imaging system using a single CCD area array. IEEE Transactions on Electron Devices, 1978. 25(2): p. 102-107.
[44] Nabeyama H., S.N., et al., All Solid State Color Camera With Single-Chip Mos Imager. IEEE Transactions on Consumer Electronics, 1981. 27(1): p. 40-46.
[45] S., K., On VLSI array architectures for digital image processing. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1986: p. 3113-3126.
[46] J., T.H., DSP solutions run the gamut for color systems. IEEE Signal Processing Magazine, 1993. 10(2): p. 8-23.
[47] Trussell, S.G.a.H.J., Digital color imaging. IEEE Transactions on Image Processing, 1997. 6(7): p. 901-932.
[48] Adams J., K.P., et al., Color processing in digital cameras. IEEE Micro, 1998.18(6): p. 20-30.
[49] Doswald D., J.H., et al., A 30-frames/s megapixel real-time CMOS image processor. IEEE Journal of Solid-State Circuits, 2000. 35(11): p. 1732-1743.
[50] Battiato, S., A. Castorina, et al., A global enhancement pipeline for low-cost imaging devices. IEEE Transactions on Consumer Electronics, 2003. 49(3): p. 670-675.
[51] http://www.canon.com/technology/interview/digic4.
[52] http://www.sony.com.sg/microsite/dslr/technologies/bionz.html.
[53] http://www.cypress.com/.
[54] http://www.aptina.com/products/image_processors_soc/.
[55] http://www.ovt.com/.
[56] Hyung Gyu Lee, U.Y.O., Radu Marculescu, et al., Design space exploration and prototyping for on-chip multimedia applications. 43rd ACM/IEEE Design Automation Conference, 2006: p. 137-142.
[57] Soares, A.B., Carro, L. and Susin, A.A., Reconfigurable communications for image processing applications. 20th International Parallel and Distributed Processing Symposium, 2006: p. 1-4.
[58] Joshi, J., Karandikar, K., Bade, S, et al., Multi-core Image processing system using Network on Chip interconnect. 50th Midwest Symposium on Circuits and Systems, 2007: p. 1257-1260.
[59] A. Beyranvand Nejad, K.G.W.G., J. Walters, et al., Mapping KPN Models of Streaming Applications on A Network-on-Chip Platform. Program for Research on Integrated Systems and Circuits, 2009: p. 441-444.
[60] Thuan Le and Khalid, M., NoC prototyping on FPGAs: A case study using an image processing benchmark. IEEE International Conference on Electro/Information Technology, 2009: p. 441-445.
[61] Yunjie Wu, H.D.a.H.S., A Programming Model and a NoC-Based Architecture for Streaming Applications. 13th Euromicro Conference on Digital System Design: Architectures, Methods and Tools, 2010: p. 393-397.
[62] http://www.vimicro.com.cn/.
[63]周荣政,数码相机处理器设计及系统集成研究:[博士学位论文],上海;复旦大学,2004.
[64] http://www.ime.cas.cn/jgsz/kybm/txydmt/yjsjj/.
[65]汪宿梁,面向多媒体图像处理的高效可重构协处理器设计:[硕士学位论文],上海;上海交通大学,2008.
[66]姚于斌,面向图像处理的可重构协处理器结构设计研究:[硕士学位论文],上海;上海交通大学,2008.
[67]卢明,基于DSP和FPGA的图像采集与处理系统设计:[硕士学位论文],西安;西安电子科技大学,2006.
[68]王强,一种实时图像处理硬件平台的设计与实现:[硕士学位论文],北京;北京交通大学,2009.
[69]庞业勇,基于FPGA的图像处理系统设计方法研究:[硕士学位论文],哈尔滨;哈尔滨工业大学,2010.
[70]戈志伟,基于片上总线的图像处理芯片设计,[硕士学位论文],天津;天津大学,2009.
[71]张钰,监控图像SoC中数字图像处理器关键技术研究:[博士学位论文],天津;天津大学,2009.
[72]钟健,图像处理片上系统:[博士学位论文],天津;天津大学,2010.
[73] Yo-Hwan Noh, G.-d., Method And Apparatus For Compensating Black Level By Dark Current Of Image Sensor. U.S. Patent, US20080192130. Aug. 14, 2008.
[74] Lam, E.Y., Combining gray world and retinex theory for automatic white balance in digital photography. Proceedings of the Ninth International Symposium on Consumer Electronics, 2005: p. 134-139.
[75] M. Nilsson, C.W., Design and implementation of a CMOS sensor based video camera incorporating a combined AWB/AEC module. The 2003 IEEE International Conference on Acoustics, Speech and Signal Processing, 2003. II: p. 477-480.
[76] K. Wen-Chung, H.C.-C., Adaptive exposure control and real-time image fusion for surveillance systems. International Symposium on Circuits and Systems, 2006: p. 935-938.
[77] Lei Zhang, X.W., David Zhang, Color Reproduction From Noisy CFA Data of Single Sensor Digital Cameras. IEEE Transactions on Image Processing, 2007. 16(9): p. 2184-2197.
[78] Lei Zhang, W.D., Xiaolin Wu etc., Spatial-Temporal Color Video Reconstruction From Noisy CFA Sequence. IEEE Transactions on Circuits and Systems for Video Technology, 2010. 20(6): p. 838-847.
[79] Sharma, G., Digital Color Imaging Handbook. Boca Raton: CRC Press LLC, 2003: p. 743-750.
[80] Seok-Han Lee, T.-Y.K., and Jong-Soo Choi, A Color Correction System using a Color Compensation Chart. International Conference on Hybrid Information Technology, 2006. 1: p. 409 - 416.
[81] Rafael C. Gonzalez, R.E.W., Digital Image Processing. Addison-Wesley Pub Co., USA, 2002: p. 89-303.
[82]邱立诚, Algorithm Design and Application for Digital Still Camera:[博士学位论文],台湾;台湾大学, 2009, p] 15-16.
[83] Ge Zhiwei, Y.S., Xu Jiangtao, Field-programmable gate array-based hardware architecture for image processing with complementary metal-oxide-semiconductor image sensor. Journal of Electronic Imaging, 2010. 19(3): p. 033014-1 - 033014-11.
[84] T. Kuno, H.S., and N. Matoba, A new automatic exposure system for digital still cameras. IEEE Transactions on Consumer Electronics, 1998. 44(4): p. 192-199.
[85] Nitin Sampat, S.V., System implications of implementing auto-exposure on consumer digital cameras. SPIE Conference on Sensors, Cameras, and Applications for Digital Photography, 1999: p. 93-99.
[86] Myung Hee Cho, S.G.L., and Byung Deok Nam, The fast auto exposure algorithm based on the numerical analysis. SPIE Conference on Sensors, Cameras, and Applications for Digital Photography, 1999: p. 120-128.
[87]戈志伟,姚素英,徐江涛,等,一种应用于CMOS图像传感器的快速自动曝光控制方法.天津大学学报, 2010. 43(10): p. 854 - 859.
[88] Yang, D.X.D., Digital Pixel CMOS Image Sensors:[Ph.D Dissertation]. California: Deparment of Electrical Engineering, Stanford University, 1999.
[89] Hong, C.S., On-Chip Spatial Image Processing with CMOS Active Pixel Sensors[Ph.D Dissertation]. Ontario, CA: University of Waterloo, 2001.
[90] JiaYi Liang, Y.Q., ZhiLiang Hong, An auto-exposure algorithm for detecting high contrast lighting conditions. Proceedings of IEEE International Conference on ASIC, 2007: p. 1114-1117.
[91] K. Wen-Chung, C.S.-H., An integrated software architecture for real-time video and audio recording systems. IEEE Transactions on Consumer Electronics, 2005. 51: p. 879-884.
[92] Robert, H., Method of Detecting Flicker, and Video Camera Using The Method. U.S. Patent, 6900834. May 31, 2005.
[93]贡青,李锦萍,闵子建,等,一种基于CMOS图像传感器的flicker自动检测方法.首都师范大学学报, 2007. 28(2): p. 32-35.
[94] James E. Adams, J., Interactions between color plane interpolation and other image processing functions in electronic photography. Proceedings of the SPIE Electronic Imaging Conference, 1995. 2416: p. 144 - 151.
[95] Tam, S.C.P.a.I.-K., Effective color interpolation in CCD color filter array using signal correlation. IEEE Transactions on Circuits and Systems for Video Technology, 2003. 13(6): p. 503-513.
[96] Adams, J.F.H.J.a.J.E., Adaptive color plane interpolation in single color electronic camera. U.S. Patent, 5629734. May 1997.
[97] P. S. Tsai, T.A., Ajay K.Ray, Adaptive Fuzzy Color Interpolation. Journal of Electronic Imaging, 2002. 11: p. 1-24.
[98] Wenmiao Lu, Y.-P.T., Color filter array demosaicing: New method andperformance measures. IEEE Transactions on Image Processing, 2003. 12(10): p. 1194 - 1210.
[99] Li, X., Demosaicing by Successive Approximation. IEEE Transactions on Image Processing, 2005. 14(3): p. 370 - 379.
[100] Alleysson D., S.S., Herault J., Linear demosaicing inspired by the human visual system. IEEE Transactions on Image Processing, 2005. 14(4): p. 439 - 449.
[101] King-Hong Chung, Y.-H.C., Color Demosaicing Using Variance of Color Differences. IEEE Transactions on Image Processing, 2006. 15(10): p. 2944 - 2955.
[102] Lei Zhang, X.W., Color demosaicking via directional linear minimum mean square-error estimation. IEEE Transactions on Image Processing, 2005. 14(12): p. 2167 - 2178.
[103] Ferradans S., B.M., Caselles V., Geometry-Based Demosaicking. IEEE Transactions on Image Processing, 2009. 18(3): p. 665 - 670.
[104] Kuo-Liang Chung, W.-J.Y., Wen-Ming Yan, Chung-Chou Wang, Demosaicing of Color Filter Array Captured Images Using Gradient Edge Detection Masks and Adaptive Heterogeneity-Projection. IEEE Transactions on Image Processing, 2008. 17(12): p. 2356 - 2367.
[105] Ge Zhiwei, Y.S., Xu Jiangtao, A demosaicing algorithm based on adaptive edge sensitive and fuzzy assignment in CMOS image sensor. The 2nd International Conference on Signal Processing Systems, 2010. 1: p. V1-420-V1-424.
[106] Sugiura., T.K.a.H., New interpolation method using discriminated color correlation for digital still cameras. IEEE Transactions on Consumer Electronics, 1999. 45(1): p. 259 - 267.
[107]单宝堂,沈庭芝,闫雪梅,等,自适应比例合成反马赛克算法的提出与应用.北京理工大学学报, 2007. 27(11): p. 1013 - 1016.
[108] IBM, 128-Bit Processor Local Bus Architecture Specifications. https://www-01.ibm.com/ chips/techlib/techlib.nsf/techdocs/3BBB27E5BCC165BA 87256A2B0064FFB4/$file/PlbBus_as_01_pub.pdf, 2007.
[109] ARM, AMBA? Specification. http://polimage.polito.it/~lavagno/esd/ IHI0011A_AMBA_ SPEC.pdf, 1999.
[110] Corporation, S., WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. http://www.cerc.utexas.edu/ ~jypark/2011_spring_VLSI_I/LAB3_Website /lab3b/wbspec_b1.pdf, 2001.
[111] Ge Zhiwei, Y.S., Xu Jiangtao, Design of on-chip image processing based on APB bus with CMOS image sensor. IEEE 8th International Conference on ASIC, 2009: p. 963-966.
[112]董德兴,高性能CIS控制系统及关键ISP技术研究,[硕士学位论文],天津;天津大学,2009.
[113] Takashi Miyamori, K.O., REMARC : Reconfigurable Multimedia Array Coprocessor. IEICE transactions on information and systems, 1999. E82-D(2): p. 389-397.
[114] Rossi D., C., F, Spolzino, S, etc., A Heterogeneous Digital Signal Processor for Dynamically Reconfigurable Computing. IEEE Journal of Solid-State Circuits, 2010. 45(8): p. 1615-1626.
[115] Claudia Rusu, L.A., Dimiter Avresky, RILM: Reconfigurable inter-layer routing mechanism for 3D multi-layer networks-on-chip. IEEE 16th International On-Line Testing Symposium, 2010: p. 121-126.
[116] Zhiwei GE, J.T., Virginie FRESSE, etc., A Scalable and Effective Routing Algorithm for Multi-FPGA Based Large Scale NoC. Le Groupement de Recherche System-On-Chip et System-In-Package, 2011. P4: p. 1 - 2.
[117] Leonel Tedesco, A.M., Diego Garibotti, etc., Traffic Generation and Performance Evaluation for Mesh-based NoCs. 18th Symposium on Integrated Circuits and Systems Design, 2005: p. 184 - 189.