基于隐藏主题概率模型的图像结构感知SISR重建方法
|
摘要
在基于示例学习的单幅图像超分辨率(SISR)重建中,假设从低分辨率(LR)到高分辨率(HR)图像块的映射关系是一对一的,但同一LR块会与多个HR块对应,导致了LR与HR块的匹配误差.为解决HR复原块的失配问题,文中首先导出了LR块主题模式的概率模型,引入信号的隐藏主题这一种新的观察信息.然后提出了一种基于块主题差异和上下文最大概率的结构感知复原机制,通过主题模式与邻域块内容的关联,形成LR块的流形描述;在重构中通过自适应主题决策树选择和节点回归矩阵映射,从相似的LR流形信号中准确区分和复原HR信号.主题模型优化实验结果表明,文中基于主题约束信息的算法比未引入隐藏主题的决策树SISR方法的峰值信噪比(PSNR)值提升了0.25 dB;在5种算法的对比实验中,相对于稀疏字典SISR方法,文中方法的PSNR值平均提升了0.92 dB,表明引入隐藏的主题信息和主题流形结构辨识是可行的.
In the process of single image super-resolution reconstruction(SISR)based on learning from examples,the mapping relation was assumed one-to-one from a low-resolution(LR)input to a high-resolution(HR)image patch.But in fact,one LR patch may relate to many HR patches,and thus leads to matching errors.To solve the mismatch problem of restored patch,the probability model of LR patch topic pattern was derived to express new observation information for hidden topics in LR signals.Then a structure-aware recovery mechanism with topic differences and context maximum probability was proposed,and LR manifold description was formed by relating topic modes to LR neighbor contents.The HR signal was accurately distinguished and reconstructed from similar LR manifold signals via an adaptive selection of topic decision trees and regression matrix of nodes.The topic mo-del optimization experiment demonstrates that the peak signal-to-noise ratio(PSNR)of our topic constraint SISR method is improved by 0.25 dB compared to that of the decision tree based SISR algorithm without introducing hidden topics.In the comparative experiment of five algorithms,the average PSNR value of our SISR approach is improved by 0.92 dB compared to that of the sparse dictionary based SISR method.So the introduced hidden topic information and topic-manifold structure identification are feasible.
In the process of single image super-resolution reconstruction(SISR)based on learning from examples,the mapping relation was assumed one-to-one from a low-resolution(LR)input to a high-resolution(HR)image patch.But in fact,one LR patch may relate to many HR patches,and thus leads to matching errors.To solve the mismatch problem of restored patch,the probability model of LR patch topic pattern was derived to express new observation information for hidden topics in LR signals.Then a structure-aware recovery mechanism with topic differences and context maximum probability was proposed,and LR manifold description was formed by relating topic modes to LR neighbor contents.The HR signal was accurately distinguished and reconstructed from similar LR manifold signals via an adaptive selection of topic decision trees and regression matrix of nodes.The topic mo-del optimization experiment demonstrates that the peak signal-to-noise ratio(PSNR)of our topic constraint SISR method is improved by 0.25 dB compared to that of the decision tree based SISR algorithm without introducing hidden topics.In the comparative experiment of five algorithms,the average PSNR value of our SISR approach is improved by 0.92 dB compared to that of the sparse dictionary based SISR method.So the introduced hidden topic information and topic-manifold structure identification are feasible.
引文
[1] ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations [C]//Proceedings of IEEE Conference on Curves and Surfaces.Berlin:IEEE,2010:711- 730.
[2] CHEN X,QI C.Low-rank neighbor embedding for single image super-resolution [J].Journal of IEEE Signal Process Letters,2014,21(1):79- 82.
[3] DONG W,ZHANG L,SHI G,et al.Nonlocal centralized sparse representation for image restoration [J].Journal of IEEE Transactions on Image Processing,2013,22(4):1620- 1630.
[4] LI Y,DONG W,XIE X,et al.Image super-resolution with parametric sparse model learning [J].Journal of IEEE Transactions on Image Processing,2018,27(9):4638- 4650.
[5] 马丽红,黄茵,黎剑晖.基于灵活LBP纹理字典构造及多特征描述的改进SCSR算法 [J].华南理工大学学报(自然科学版),2015,43(3):57- 65.MA Li-hong,HUANG Yin,LI Jian-hui.Improved SCSR algorithm on the basis of flexible LBP texture dictionary and multi-feature description [J].Journal of South China University of Technology(Natural Science Edition),2015,43(3):57- 65.
[6] CHANG H,YEUNG D,XIONG Y.Super-resolution through neighbor embedding [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.Washington:IEEE,2004:275- 282.
[7] TIMOFT R,SMET D.Anchored neighborhood regression for fast example-based super-resolution [C]//Procee-dings of IEEE Conference on Computer Vision.Sydney:IEEE,2013:1920- 1927.
[8] TIMOFT R,SMET D.A+:adjusted anchored neighborhood regression for fast super-resolution [C]//Procee-dings of Asian Conference on Computer Vision.Singapore:IEEE,2014:111- 126.
[9] SCHUITR S,LEISTNER C,BISCHOR H.Fast and accurate image upscaling with super-resolution forests [C]//Proceedings of IEEE Computer Vision and Pattern Recognition.Boston:IEEE,2016:3791- 3799.
[10] HUANG J,LIU T.SRHRF+:self-example enhanced single image super-resolution using hierarchical random forests [C]//Proceedings of IEEE Computer Vision and Pattern Recognition.Honolulu:IEEE,2017:1067- 1075.
[11] JIAN S,ZHU J,TAPPEN M.Context-constrained hallucination for image super-resolution [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE,2010:231- 238.
[12] JIAN L,SUN Y.Context-aware single image super-resolution using sparse representation and cross-scale similarity [J].Journal of Signal Process Image Communication,2015,32(4):40- 53.
[13] WANG X,YU K.Recovering realistic texture in image super-resolution by deep spatial feature transform [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:391- 407.
[14] DONG C,LOY C,HE K.Image super-resolution using deep convolutional networks [J].Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295- 307.
[15] HARIS M,UKITA N.Deep back-projection networks for super-resolution [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:485- 493.
[16] LAI W,HUANG J,AHUJA N.Fast and accurate image super-resolution with deep Laplacian pyramid networks [J].Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,35(1):79- 82.
[2] CHEN X,QI C.Low-rank neighbor embedding for single image super-resolution [J].Journal of IEEE Signal Process Letters,2014,21(1):79- 82.
[3] DONG W,ZHANG L,SHI G,et al.Nonlocal centralized sparse representation for image restoration [J].Journal of IEEE Transactions on Image Processing,2013,22(4):1620- 1630.
[4] LI Y,DONG W,XIE X,et al.Image super-resolution with parametric sparse model learning [J].Journal of IEEE Transactions on Image Processing,2018,27(9):4638- 4650.
[5] 马丽红,黄茵,黎剑晖.基于灵活LBP纹理字典构造及多特征描述的改进SCSR算法 [J].华南理工大学学报(自然科学版),2015,43(3):57- 65.MA Li-hong,HUANG Yin,LI Jian-hui.Improved SCSR algorithm on the basis of flexible LBP texture dictionary and multi-feature description [J].Journal of South China University of Technology(Natural Science Edition),2015,43(3):57- 65.
[6] CHANG H,YEUNG D,XIONG Y.Super-resolution through neighbor embedding [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.Washington:IEEE,2004:275- 282.
[7] TIMOFT R,SMET D.Anchored neighborhood regression for fast example-based super-resolution [C]//Procee-dings of IEEE Conference on Computer Vision.Sydney:IEEE,2013:1920- 1927.
[8] TIMOFT R,SMET D.A+:adjusted anchored neighborhood regression for fast super-resolution [C]//Procee-dings of Asian Conference on Computer Vision.Singapore:IEEE,2014:111- 126.
[9] SCHUITR S,LEISTNER C,BISCHOR H.Fast and accurate image upscaling with super-resolution forests [C]//Proceedings of IEEE Computer Vision and Pattern Recognition.Boston:IEEE,2016:3791- 3799.
[10] HUANG J,LIU T.SRHRF+:self-example enhanced single image super-resolution using hierarchical random forests [C]//Proceedings of IEEE Computer Vision and Pattern Recognition.Honolulu:IEEE,2017:1067- 1075.
[11] JIAN S,ZHU J,TAPPEN M.Context-constrained hallucination for image super-resolution [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE,2010:231- 238.
[12] JIAN L,SUN Y.Context-aware single image super-resolution using sparse representation and cross-scale similarity [J].Journal of Signal Process Image Communication,2015,32(4):40- 53.
[13] WANG X,YU K.Recovering realistic texture in image super-resolution by deep spatial feature transform [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:391- 407.
[14] DONG C,LOY C,HE K.Image super-resolution using deep convolutional networks [J].Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295- 307.
[15] HARIS M,UKITA N.Deep back-projection networks for super-resolution [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:485- 493.
[16] LAI W,HUANG J,AHUJA N.Fast and accurate image super-resolution with deep Laplacian pyramid networks [J].Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,35(1):79- 82.