基于人脸标准化的纹理和光照保持3D人脸重构
|
摘要
现有人脸纹理重建方法对于人脸的皱纹、胡须、瞳孔颜色等重建效果往往不够细致.为了解决此问题,文中提出基于人脸标准化的纹理和光照保持3D人脸重构.首先对2D人脸图像标准化,使用光照信息和对称纹理重构人脸自遮挡区域的纹理.然后依据2D-3D点对应关系从标准化的2D人脸图像获取相应的3D人脸纹理,结合人脸形状重构和纹理信息,得到最终的3D人脸重构结果.实验表明文中方法有效保留原始2D图像的纹理和光照信息,重构的人脸更自然,具有更丰富的人脸细节.
The results of the existing texture recovery methods are not detailed enough for the face features such as wrinkles, beards and pupil colors. Therefore, the texture and illumination preserving 3 D face reconstruction based on face normalization is proposed. Firstly, the 2 D facial image is normalized to reconstruct the self-occlusion area using illumination information and symmetric texture. Then, the corresponding 3 D face texture is obtained by the normalized 2 D image according to the 2 D-3 D point pairs. Finally, combining the face shape reconstruction and texture information, the final 3 D face reconstruction results are generated. The experimental results show that the proposed method preserves the texture and the illumination information of the original 2 D image effectively, and the reconstructed faces are more natural with more facial details.
The results of the existing texture recovery methods are not detailed enough for the face features such as wrinkles, beards and pupil colors. Therefore, the texture and illumination preserving 3 D face reconstruction based on face normalization is proposed. Firstly, the 2 D facial image is normalized to reconstruct the self-occlusion area using illumination information and symmetric texture. Then, the corresponding 3 D face texture is obtained by the normalized 2 D image according to the 2 D-3 D point pairs. Finally, combining the face shape reconstruction and texture information, the final 3 D face reconstruction results are generated. The experimental results show that the proposed method preserves the texture and the illumination information of the original 2 D image effectively, and the reconstructed faces are more natural with more facial details.
引文
[1] BLANZ V,VETTER T.A Morphable Model for the Synthesis of 3D Faces // Proc of the 26th Annual Conference on Computer Graphics and Interactive Techniques.New York,USA:ACM,1999:187-194.
[2] 梁荣华,陈纯,张慧.一个三维人脸真实感模型重建算法.模式识别与人工智能,2003,16(1):116-121.(LIANG R H,CHEN C,ZHANG H.An Algorithm for 3D Realistic-Looking Facial Model Reconstruction.Pattern Recognition and Artificial Intelligence,2003,16(1):116-121.)
[3] CHU B,ROMDHANI S,CHEN L M.3D-Aided Face Recognition Robust to Expression and Pose Variations // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2014:1907-1914.
[4] AMBERG B,KNOTHE R,VETTER T.Expression Invariant 3D Face Recognition with a Morphable Model // Proc of the 8th IEEE International Conference on Automatic Face and Gesture Recognition.Washington,USA:IEEE,2008.DOI:10.1109/AFGR.2008.4813376.
[5] CAO C,WENG Y L,ZHOU S,et al.Facewarehouse:A 3D Facial Expression Database for Visual Computing.IEEE Transactions on Visualization and Computer Graphics,2014,20(3):413-425.
[6] BLANZ V,MEHL A,VETTER T,et al.A Statistical Method for Robust 3D Surface Reconstruction from Sparse Data // Proc of the 2nd International Symposium on 3D Data Processing,Visualization and Transmission.Washington,USA:IEEE,2004:293-300.
[7] ROMDHANI S,VETTER T.Estimating 3D Shape and Texture Using Pixel Intensity,Edges,Specular Highlights,Texture Constraints and a Prior // Proc of the IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2005,II:986-993.
[8] ZHU X Y,YAN J J,YI D,et al.Discriminative 3D Morphable Model Fitting // Proc of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.Washington,USA:IEEE,2015.DOI:10.1109/FG.2015.7163096.
[9] LIU F,ZHAO Q J,LIU X M,et al.Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition[J/OL].[2019-01-13].http://cvlab.cse.msu.edu/pdfs/Liu_Zeng_Zhao_Liu_ECCV2016.pdf.
[10] HUBER P,FENG Z H,CHRISTMAS W,et al.Fitting 3D Morphable Face Models Using Local Features // Proc of the IEEE International Conference on Image Processing.Washington,USA:IEEE,2015:1195-1199.
[11] JOURABLOO A,LIU X M.Pose-Invariant 3D Face Alignment[C/OL].[2019-01-13].https://arxiv.org/pdf/1506.03799.pdf.
[12] ZHU X Y,LEI Z,LIU X M,et al.Face Alignment across Large Poses:A 3D Solution[C/OL].[2019-01-13].https://arxiv.org/pdf/1511.07212.pdf.
[13] DOU P F,SHAH S K,KAKADIARIS I A.End-to-End 3D Face Reconstruction with Deep Neural Networks[C/OL].[2019-01-13].https://arxiv.org/pdf/1704.05020.pdf.
[14] JOURABLOO A,LIU X M.Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:4188-4196.
[15] FENG Y,WU F,SHAO X H,et al.Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network[C/OL].[2019-01-13].https://arxiv.org/pdf/1803.07835.pdf.
[16] BLANZ V,SCHERBAUM K,VETTER T,et al.Exchanging Faces in Images.Computer Graphics Forum,2004,23(3):669-676.
[17] HU G S,YAN F,CHAN C H,et al.Face Recognition Using a Unified 3D Morphable Model // Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2016:73-89.
[18] MA M Y,HU X Y,XU Y Q,et al.A Lighting Robust Fitting Approach of 3D Morphable Model Using Spherical Harmonic Illumination // Proc of the 22nd International Conference on Pattern Recognition.Washington,USA:IEEE,2014:2101-2106.
[19] ZHU X Y,LEI Z,YAN J J,et al.High-Fidelity Pose and Expre-ssion Normalization for Face Recognition in the Wild // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2015:787-796.
[20] PéREZ P,GANGNET M,BLAKE A.Poisson Image Editing.ACM Transactions on Graphics,2003,22(3):313-318.
[21] ZHANG L,SAMARAS D.Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics.IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(3):351-363.
[22] ZHU X X,RAMANAN D.Face Detection,Pose Estimation,and Landmark Localization in the Wild // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2012:2879-2886.
[23] BELHUMEUR P N,JACOBS D W,KRIEGMAN D J,et al.Localizing Parts of Faces Using a Consensus of Exemplars.IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(12):2930-2940.
[2] 梁荣华,陈纯,张慧.一个三维人脸真实感模型重建算法.模式识别与人工智能,2003,16(1):116-121.(LIANG R H,CHEN C,ZHANG H.An Algorithm for 3D Realistic-Looking Facial Model Reconstruction.Pattern Recognition and Artificial Intelligence,2003,16(1):116-121.)
[3] CHU B,ROMDHANI S,CHEN L M.3D-Aided Face Recognition Robust to Expression and Pose Variations // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2014:1907-1914.
[4] AMBERG B,KNOTHE R,VETTER T.Expression Invariant 3D Face Recognition with a Morphable Model // Proc of the 8th IEEE International Conference on Automatic Face and Gesture Recognition.Washington,USA:IEEE,2008.DOI:10.1109/AFGR.2008.4813376.
[5] CAO C,WENG Y L,ZHOU S,et al.Facewarehouse:A 3D Facial Expression Database for Visual Computing.IEEE Transactions on Visualization and Computer Graphics,2014,20(3):413-425.
[6] BLANZ V,MEHL A,VETTER T,et al.A Statistical Method for Robust 3D Surface Reconstruction from Sparse Data // Proc of the 2nd International Symposium on 3D Data Processing,Visualization and Transmission.Washington,USA:IEEE,2004:293-300.
[7] ROMDHANI S,VETTER T.Estimating 3D Shape and Texture Using Pixel Intensity,Edges,Specular Highlights,Texture Constraints and a Prior // Proc of the IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2005,II:986-993.
[8] ZHU X Y,YAN J J,YI D,et al.Discriminative 3D Morphable Model Fitting // Proc of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.Washington,USA:IEEE,2015.DOI:10.1109/FG.2015.7163096.
[9] LIU F,ZHAO Q J,LIU X M,et al.Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition[J/OL].[2019-01-13].http://cvlab.cse.msu.edu/pdfs/Liu_Zeng_Zhao_Liu_ECCV2016.pdf.
[10] HUBER P,FENG Z H,CHRISTMAS W,et al.Fitting 3D Morphable Face Models Using Local Features // Proc of the IEEE International Conference on Image Processing.Washington,USA:IEEE,2015:1195-1199.
[11] JOURABLOO A,LIU X M.Pose-Invariant 3D Face Alignment[C/OL].[2019-01-13].https://arxiv.org/pdf/1506.03799.pdf.
[12] ZHU X Y,LEI Z,LIU X M,et al.Face Alignment across Large Poses:A 3D Solution[C/OL].[2019-01-13].https://arxiv.org/pdf/1511.07212.pdf.
[13] DOU P F,SHAH S K,KAKADIARIS I A.End-to-End 3D Face Reconstruction with Deep Neural Networks[C/OL].[2019-01-13].https://arxiv.org/pdf/1704.05020.pdf.
[14] JOURABLOO A,LIU X M.Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2016:4188-4196.
[15] FENG Y,WU F,SHAO X H,et al.Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network[C/OL].[2019-01-13].https://arxiv.org/pdf/1803.07835.pdf.
[16] BLANZ V,SCHERBAUM K,VETTER T,et al.Exchanging Faces in Images.Computer Graphics Forum,2004,23(3):669-676.
[17] HU G S,YAN F,CHAN C H,et al.Face Recognition Using a Unified 3D Morphable Model // Proc of the European Conference on Computer Vision.Berlin,Germany:Springer,2016:73-89.
[18] MA M Y,HU X Y,XU Y Q,et al.A Lighting Robust Fitting Approach of 3D Morphable Model Using Spherical Harmonic Illumination // Proc of the 22nd International Conference on Pattern Recognition.Washington,USA:IEEE,2014:2101-2106.
[19] ZHU X Y,LEI Z,YAN J J,et al.High-Fidelity Pose and Expre-ssion Normalization for Face Recognition in the Wild // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2015:787-796.
[20] PéREZ P,GANGNET M,BLAKE A.Poisson Image Editing.ACM Transactions on Graphics,2003,22(3):313-318.
[21] ZHANG L,SAMARAS D.Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics.IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(3):351-363.
[22] ZHU X X,RAMANAN D.Face Detection,Pose Estimation,and Landmark Localization in the Wild // Proc of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,USA:IEEE,2012:2879-2886.
[23] BELHUMEUR P N,JACOBS D W,KRIEGMAN D J,et al.Localizing Parts of Faces Using a Consensus of Exemplars.IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(12):2930-2940.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |