视觉特征深度融合的图像质量评价
|
摘要
针对当前视觉感知特性研究和图像特征评价算法的不足,通过构建视觉多通道神经网络融合预测模型,提出一种视觉特征深度融合的图像质量评价方法.首先,结合人类视觉系统特性设计直方图统计和奇异值分解2个互补视觉评价算法,进一步对图像各视觉通道的稀疏化梯度信息进行深度处理.其次,构建BP神经网络融合模型,对各层视觉特征的多通道评价融合分别进行预测.最后,对3层视觉特征评价从内层到外层逐层地进行深度自适应融合.实验结果表明,所构建的融合模型有效提高了各种评价算法的指标水平,所提方法优于已有方法.
The neural network pooling prediction model of visual multi-channel was constructed and an image quality assessment method based on deep pooling of visual feature was proposed, aiming at the shortcoming of current research on visual perception characteristic and image feature assessment algorithms. Firstly, two complementary visual assessment algorithms based on histogram statistics and singular value decomposition were designed with human visual system characteristics. Further, the sparse gradient information of every visual channel for one image was deeply processed. Secondly, the multi-channel assessment pooling of every visual feature layer was predicted, respectively, by constructing a pooling model of the BP neural network. Finally, the visual feature assessment of three layers was deep adaptively pooled from the inner layer to the outer layer. The experiment results show that the constructed pooling model effectively improves the index level of every assessment algorithm, and the proposed method achieves great advantage compared to the existing methods.
The neural network pooling prediction model of visual multi-channel was constructed and an image quality assessment method based on deep pooling of visual feature was proposed, aiming at the shortcoming of current research on visual perception characteristic and image feature assessment algorithms. Firstly, two complementary visual assessment algorithms based on histogram statistics and singular value decomposition were designed with human visual system characteristics. Further, the sparse gradient information of every visual channel for one image was deeply processed. Secondly, the multi-channel assessment pooling of every visual feature layer was predicted, respectively, by constructing a pooling model of the BP neural network. Finally, the visual feature assessment of three layers was deep adaptively pooled from the inner layer to the outer layer. The experiment results show that the constructed pooling model effectively improves the index level of every assessment algorithm, and the proposed method achieves great advantage compared to the existing methods.
引文
[1]GUO Y C,HAO Y T,YU M.Image retargeting quality assessment based on content deformation measurement[J].Signal Processing:Image Communication,2018,67(6):171-181.
[2]CHEN Z B,LIN J X,LIAO N,et al.Full reference quality assessment for image retargeting based on natural scene statistics modeling and bi-directional saliency similarity[J].IEEE Transactions on Image Processing,2017,26(11):5138-5148.
[3]ZHANG Y CH,KING N N,MA L,et al.Objective quality assessment of image retargeting by incorporating fidelity measures and inconsistency detection[J].IEEE Transactions on Image Processing,2017,26(11):5980-5993.
[4]ZHANG Y B,LIN W S,LI Q H,et al.Multiple-level feature-based measure for retargeted image quality[J].IEEE Transactions on Image Processing,2018,27(1):451-463.
[5]丰明坤,王中鹏,叶绿.视觉稀疏化多通道多特征自适应的图像评价[J].仪器仪表学报,2016,37(3):667-674.FENG Ming-kun,WANG ZHong-peng,YE Lv.Image quality assessment based on adaptive sparse visual multi-channel and multi-feature pooling[J].Chinese Journal of Scientific Instrument,2016,37(3):667-674.
[6]BAMPIS C G,GUPTA P,SOUNDARARAJAN R,et al.Sp EED-QA:patial efficient entropic differencing for image and video quality[J].IEEE Signal Processing Letters,2017,24(9):1333-1337.
[7]NI ZH K,MA L,ZENG H Q,et al.Gradient direction for screen content image quality assessment[J].IEEE Signal Processing Letters,2016,23(10):1394-1398.
[8]KIM J,ZENG H,GHADIYARAM D,et al.Deep convolutional neural models for picture-quality prediction:challenges and solutions to data-driven image quality assessment[J].IEEE Signal Processing Magazine,2017,34(6):130-141.
[9]GHADIYARAM D,BOVIK A.C Massive online crowdsourced study of subjective and objective picture quality[J].IEEE Transactions on Image Process,2016,25(1):372-387.
[10]LARSON E C,CHANDLER D M.Most apparent distortion:full-reference image quality assessment and the role of strategy[J].Journal of Electronic Imaging,2010,19(1):011006-1-011006-21.
[11]ZHANG L,ZHANG L,MOU X Q,et al.FSIM:a feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing,2011,20(8):2378-2386.
[12]ZHANG L,SHEN Y LI H Y.VSI:a visual saliencyinduced index for perceptual image quality assessment[J].IEEE Transactions on Image Processing,2014,23(10):4270-4281.
[13]DING Y,WANG SH Z,ZHANG D.Full-reference image quality assessment using statistical local correlation[J].Electronics Letters,2014,50(2):79-81.
[14]WU J J,LIN W S,SHI G M.Perceptual quality metric with internal generative mechanism[J].IEEETransactions on Image Processing,2013,22(1):43-54.
[15]丰明坤,赵生妹,邢超.基于视觉显著失真度的图像质量自适应评价方法[J].电子与信息学报,2015,37(9):2062-2068.FENG Ming-kun,ZHAO SHeng-mei,XING CHao.Image quality self-adaptive assessment based on visual salience distortion[J].Journal of Electronics&Information Technology,2015,37(9):2062-2068.
[16]丰明坤,赵生妹,施祥.视觉多通道梯度与低阶矩自适应图像评价[J].仪器仪表学报,2015,36(11):2531-2537.FENG Ming-kun,ZHAO SHeng-mei,SHI Xiang.Adaptive image quality assessment based on visual multi-channel gradient and low order moment[J].Chinese Journal of Scientific Instrument,2015,36(11):2531-2537.
[17]WANG ZH,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[18]WANG ZH,LU L G,BOVIK A C.Foveation scalable video coding with automatic fixation selection[J].IEEE Transactions on Image Processing,2003,12(2):243-254.
[19]LIU A M,LIN W S,NARWARIA M.Image quality assessment based on gradient similarity[J].IEEETransaction on Image Processing,2012,21(4):1500-1512.
[20]GAO X B,LU W,TAO D CH,et al.Image quality assessment based on multiscale geometric analysis[J].IEEE Transactions on Image Processing,2009,18(7):1409-1423.
[21]林志洁,丰明坤.深度视觉特征与策略互补融合的图像质量评价[J].模式识别与人工智能,2017,30(8):682-691.LIN ZHi-jie,FENG Ming-kun.Image quality assessment based on complementary pooling of deeply visual feature and strategy[J].Pattern Recognition and Artificial Intelligence,2017,30(8):682-691.
[22]SHNAYDERMAN A,GUSEV A,ESKICIOGLU AM.An SVD-based grayscale image quality measure for local and global assessment[J].IEEETransactions on Image Processing,2006,15(2):422-429.
[23]HU A ZH,ZHANG R YIN D,et al.Image quality assessment using an SVD-based structural projection[J].Signal Processing:Image Communication,2014,29(3):293-302.
[24]WANG ZH,SIMONCELLI E P,BOVIK A C.Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th IEEEAsilomar Conference on Signals,Systems and Computers.Pacific Grove:IEEE,2002(2):1398-1402.
[25]SHEIKH H R,BOVIK A C,VECIANA G D.An information fidelity criterion for image quality assessment using natural scene statistics[J].IEEETransactions on Image Processing,2005,14(12):2117-2128.
[26]SHEIKH H R,BOVIK A C.Image information and visual quality[J].IEEE Transactions on Image Processing,2006,15(2):430-444.
[2]CHEN Z B,LIN J X,LIAO N,et al.Full reference quality assessment for image retargeting based on natural scene statistics modeling and bi-directional saliency similarity[J].IEEE Transactions on Image Processing,2017,26(11):5138-5148.
[3]ZHANG Y CH,KING N N,MA L,et al.Objective quality assessment of image retargeting by incorporating fidelity measures and inconsistency detection[J].IEEE Transactions on Image Processing,2017,26(11):5980-5993.
[4]ZHANG Y B,LIN W S,LI Q H,et al.Multiple-level feature-based measure for retargeted image quality[J].IEEE Transactions on Image Processing,2018,27(1):451-463.
[5]丰明坤,王中鹏,叶绿.视觉稀疏化多通道多特征自适应的图像评价[J].仪器仪表学报,2016,37(3):667-674.FENG Ming-kun,WANG ZHong-peng,YE Lv.Image quality assessment based on adaptive sparse visual multi-channel and multi-feature pooling[J].Chinese Journal of Scientific Instrument,2016,37(3):667-674.
[6]BAMPIS C G,GUPTA P,SOUNDARARAJAN R,et al.Sp EED-QA:patial efficient entropic differencing for image and video quality[J].IEEE Signal Processing Letters,2017,24(9):1333-1337.
[7]NI ZH K,MA L,ZENG H Q,et al.Gradient direction for screen content image quality assessment[J].IEEE Signal Processing Letters,2016,23(10):1394-1398.
[8]KIM J,ZENG H,GHADIYARAM D,et al.Deep convolutional neural models for picture-quality prediction:challenges and solutions to data-driven image quality assessment[J].IEEE Signal Processing Magazine,2017,34(6):130-141.
[9]GHADIYARAM D,BOVIK A.C Massive online crowdsourced study of subjective and objective picture quality[J].IEEE Transactions on Image Process,2016,25(1):372-387.
[10]LARSON E C,CHANDLER D M.Most apparent distortion:full-reference image quality assessment and the role of strategy[J].Journal of Electronic Imaging,2010,19(1):011006-1-011006-21.
[11]ZHANG L,ZHANG L,MOU X Q,et al.FSIM:a feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing,2011,20(8):2378-2386.
[12]ZHANG L,SHEN Y LI H Y.VSI:a visual saliencyinduced index for perceptual image quality assessment[J].IEEE Transactions on Image Processing,2014,23(10):4270-4281.
[13]DING Y,WANG SH Z,ZHANG D.Full-reference image quality assessment using statistical local correlation[J].Electronics Letters,2014,50(2):79-81.
[14]WU J J,LIN W S,SHI G M.Perceptual quality metric with internal generative mechanism[J].IEEETransactions on Image Processing,2013,22(1):43-54.
[15]丰明坤,赵生妹,邢超.基于视觉显著失真度的图像质量自适应评价方法[J].电子与信息学报,2015,37(9):2062-2068.FENG Ming-kun,ZHAO SHeng-mei,XING CHao.Image quality self-adaptive assessment based on visual salience distortion[J].Journal of Electronics&Information Technology,2015,37(9):2062-2068.
[16]丰明坤,赵生妹,施祥.视觉多通道梯度与低阶矩自适应图像评价[J].仪器仪表学报,2015,36(11):2531-2537.FENG Ming-kun,ZHAO SHeng-mei,SHI Xiang.Adaptive image quality assessment based on visual multi-channel gradient and low order moment[J].Chinese Journal of Scientific Instrument,2015,36(11):2531-2537.
[17]WANG ZH,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[18]WANG ZH,LU L G,BOVIK A C.Foveation scalable video coding with automatic fixation selection[J].IEEE Transactions on Image Processing,2003,12(2):243-254.
[19]LIU A M,LIN W S,NARWARIA M.Image quality assessment based on gradient similarity[J].IEEETransaction on Image Processing,2012,21(4):1500-1512.
[20]GAO X B,LU W,TAO D CH,et al.Image quality assessment based on multiscale geometric analysis[J].IEEE Transactions on Image Processing,2009,18(7):1409-1423.
[21]林志洁,丰明坤.深度视觉特征与策略互补融合的图像质量评价[J].模式识别与人工智能,2017,30(8):682-691.LIN ZHi-jie,FENG Ming-kun.Image quality assessment based on complementary pooling of deeply visual feature and strategy[J].Pattern Recognition and Artificial Intelligence,2017,30(8):682-691.
[22]SHNAYDERMAN A,GUSEV A,ESKICIOGLU AM.An SVD-based grayscale image quality measure for local and global assessment[J].IEEETransactions on Image Processing,2006,15(2):422-429.
[23]HU A ZH,ZHANG R YIN D,et al.Image quality assessment using an SVD-based structural projection[J].Signal Processing:Image Communication,2014,29(3):293-302.
[24]WANG ZH,SIMONCELLI E P,BOVIK A C.Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th IEEEAsilomar Conference on Signals,Systems and Computers.Pacific Grove:IEEE,2002(2):1398-1402.
[25]SHEIKH H R,BOVIK A C,VECIANA G D.An information fidelity criterion for image quality assessment using natural scene statistics[J].IEEETransactions on Image Processing,2005,14(12):2117-2128.
[26]SHEIKH H R,BOVIK A C.Image information and visual quality[J].IEEE Transactions on Image Processing,2006,15(2):430-444.