图像超分辨率卷积神经网络加速算法
|
摘要
为了实现模型的实时和嵌入式运行,提出了一种轻量级的卷积神经网络结构。通过采用较小的滤波器尺寸和引入深度可分离卷积,可大量减少模型参数,提高模型非线性表达能力;在网络末端引入子像素卷积层,直接从原始低分辨率图像学习到高分辨率图像的映射,计算成本为原来的1/k2(k为放大因子)。在Set5数据集上的实验表明,所提模型的速度较经典的图像超分辨率重建算法速度提高了25. 8倍,能够在通用GPU上实时运行,峰值信噪比平均提高了0. 17 dB,并且参数只有它的35%。
In order to realize the real-time and embedded operation of the model,a lightweight convolution neural network structure was proposed.By using a smaller filter size and introducing depthwise separable convolution,a large number of model parameters can be subtracted and the nonlinear capability can be improved.The sub-pixel convolution was introduced at the end of the network,then the mapping was learned directly from the original low-resolution image(without interpolation) to the high-resolution one,the cost is 1/k2 as much as before(k is the magnification factor).Experimental results on Set5 show that the proposed model is more than 25.8 times faster than the classical super resolution reconstruction algorithm and can run in real-time on a common GPU;and the proposed method with only 35% parameters of SRCNN(super resolution convolution neural network),improves the PSNR(peak signal to noise ratio) with value of 0.17 dB.
In order to realize the real-time and embedded operation of the model,a lightweight convolution neural network structure was proposed.By using a smaller filter size and introducing depthwise separable convolution,a large number of model parameters can be subtracted and the nonlinear capability can be improved.The sub-pixel convolution was introduced at the end of the network,then the mapping was learned directly from the original low-resolution image(without interpolation) to the high-resolution one,the cost is 1/k2 as much as before(k is the magnification factor).Experimental results on Set5 show that the proposed model is more than 25.8 times faster than the classical super resolution reconstruction algorithm and can run in real-time on a common GPU;and the proposed method with only 35% parameters of SRCNN(super resolution convolution neural network),improves the PSNR(peak signal to noise ratio) with value of 0.17 dB.
引文
[1]李素梅,雷国庆,范如.基于卷积神经网络的深度图超分辨率重建[J].光学学报,2017,37(12):1210002.LI Sumei,LEI Guoqing,FAN Ru.Depth map superresolution based on deep convolutional neural networks[J].Acta Optica Sinica,2017,37(12):1210002.(in Chinese)
[2]Dong C,Loy C C,He K,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[3]Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition,2016:1637-1645.
[4]Shi W Z,Caballero J,Huszár F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition,2016:1874-1883.
[5]Ledig C,Theis L,Huszar F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C].ar Xiv:1609.04802v2,2016.
[6]Dong C,Loy C C,Tang X O.Accelerating the superresolution convolutional neural network[C].ar Xiv:1608.00367v1,2016.
[7]肖进胜,刘恩雨,朱力,等.改进的基于卷积神经网络的图像超分辨率算法[J].光学学报,2017,37(3):0318011-1-0318011-9.XIAO Jinsheng,LIU Enyu,ZHU Li,et al.Improved image super-resolution based on convolutional neural network[J].Acta Optica Sinica,2017,37(3):0318011-1-0318011-9.(in Chinese)
[8]Chollet F.Xception:deep learning with depthwise separable convolutions[C].ar Xiv:1610.02357v1[cs],2016.
[9]Ioffe S,Szegedy C.Batch normalization:accelerating deep network training by reducing internal covariate shift[J].ar Xiv:1502.03167v3,2015.
[10]Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,2015.
[11]Yang J C,Wright J,Huang T S,et al.Image superresolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[12]Bevilacqua M,Roumy A,Guillemot C,et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the 23rd British Machine Vision Conference(BMVC),2012.
[13]Zeyde R,Elad M,Protter M.On single image scale-up using sparse-representations[C]//Proceedings of International Conference on Curves and Surfaces,2010:711-730.
[14]Kingma D P,Ba J.Adam:a method for stochastic optimization[C]//Proceedings of 3rd International Conference for Learning Representations,2015.
[15]Glorot X,Bengio Y.Understanding the difculty of training deep feedforward neural networks[J].Journal of Machine Learning Research,2010,9:249-256.
[16]Chollet F.Deep learning library for python[M].Shelter Island,Manning Publications Co.,2017.
[17]Timofte R,de Smet V,van Gool L.A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Proceedings of Asian Conference on Computer Vision,2014:111-126.
[2]Dong C,Loy C C,He K,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[3]Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition,2016:1637-1645.
[4]Shi W Z,Caballero J,Huszár F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition,2016:1874-1883.
[5]Ledig C,Theis L,Huszar F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C].ar Xiv:1609.04802v2,2016.
[6]Dong C,Loy C C,Tang X O.Accelerating the superresolution convolutional neural network[C].ar Xiv:1608.00367v1,2016.
[7]肖进胜,刘恩雨,朱力,等.改进的基于卷积神经网络的图像超分辨率算法[J].光学学报,2017,37(3):0318011-1-0318011-9.XIAO Jinsheng,LIU Enyu,ZHU Li,et al.Improved image super-resolution based on convolutional neural network[J].Acta Optica Sinica,2017,37(3):0318011-1-0318011-9.(in Chinese)
[8]Chollet F.Xception:deep learning with depthwise separable convolutions[C].ar Xiv:1610.02357v1[cs],2016.
[9]Ioffe S,Szegedy C.Batch normalization:accelerating deep network training by reducing internal covariate shift[J].ar Xiv:1502.03167v3,2015.
[10]Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,2015.
[11]Yang J C,Wright J,Huang T S,et al.Image superresolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[12]Bevilacqua M,Roumy A,Guillemot C,et al.Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the 23rd British Machine Vision Conference(BMVC),2012.
[13]Zeyde R,Elad M,Protter M.On single image scale-up using sparse-representations[C]//Proceedings of International Conference on Curves and Surfaces,2010:711-730.
[14]Kingma D P,Ba J.Adam:a method for stochastic optimization[C]//Proceedings of 3rd International Conference for Learning Representations,2015.
[15]Glorot X,Bengio Y.Understanding the difculty of training deep feedforward neural networks[J].Journal of Machine Learning Research,2010,9:249-256.
[16]Chollet F.Deep learning library for python[M].Shelter Island,Manning Publications Co.,2017.
[17]Timofte R,de Smet V,van Gool L.A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Proceedings of Asian Conference on Computer Vision,2014:111-126.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |