基于视频的人脸检测和跟踪算法研究
摘要
人脸检测与人脸跟踪技术是各种人脸图像处理算法的关键技术。人脸图像处理领域包含有人脸识别、姿态估计、表情识别、视频监控等多个研究方向,而几乎所有这些方向都涉及到人脸的检测与跟踪问题。本文在收集和分析了近年来国内外关于人脸检测与跟踪的学术论文及研究报告的基础上,对人脸检测与跟踪算法进行了深入研究,并根据国际、国内人脸检测与人脸跟踪技术的研究成果,设计并实现了基于Adaboost和CamShift算法的人脸检测和跟踪方法。该方法借鉴了前人的一些经验,并针对实验条件,作了一些关键性的改进。本文主要研究工作如下:
1、针对传统Adaboost检测算法对于侧面人脸检测率不高的情况,对Adaboost人脸检测算法进行相关改进。首先从一个较大的特征集中选择少量关键Haar-Like特征完成高效强分类器的构建;其次将每个强分类器级联成为更加复杂的级联分类器,在此基础上,分别训练正面人脸分类器和侧面人脸分类器,将正面、侧面人脸检测结果进行融合得出人脸区域,并对其进行肤色验证,提高算法的鲁棒性、降低了虚警率。实验证明该算法检测速度快,实时性好,在CMU人脸测试库上进行了实验,检测率达到82.7%。
2、分析对比基于肤色和基于Adaboost的两种人脸检测算法。针对Adaboost算法对人脸偏转和遮挡时检测效果差的问题,为了解决Adaboost人脸检测的这一缺点,本文将其与人脸跟踪算法相融合实现人脸的多角度检测。
3、在软件开发平台Visual C++环境中完成对Adaboost算法与Camshift算法的融合。利用检测过程中人脸区域初始化跟踪窗口,建立肤色的色调信息模型对后续帧进行跟踪。通过实验得知该算法计算量小、跟踪速度快,并且不受人脸不规则运动和随意转动的影响,对场景中出现其他人脸不会出现误跟踪,对人脸被部分遮挡也不会出现跟踪丢失现象,大大提高了跟踪的匹配度和精度,证明了该算法的可行性和有效性。
The face detection and tracking are one of the key techniques in various facial processing algorithms. The research fields of facial image processing include face recognition, gesture estimation, facial expression recognition, video monitoring and so on. And nearly all of these are involved in the area of face detection and face tracking. Based on the collection and analysis of numerous the domestic and international academic thesis and research reports on face detection and tracking in recent years, face detection and tracking algorithms is lucubrated. According to the previous research achievements on face detection and tracking, a face detection algorithm based on Adaboost and CamShift has been realized. Aiming at the experiment conditions, crucial improvements have been achieved by learning from some traditional and classical algorithms of face detection and tracking. The study of this paper focused on the following aspects:
1. Aiming at the poor rate of side face detection by traditional Adaboost algorihtm, some related improvements to Adaboost algorihtm were proposed. Firstly, few key Haar-Like features from a large feature set were selected to generate an effective strong classifier. Secondly, cascades these single strong classifier into a more complex cascade classifier. On that basis, the frontal face classifier and side face classifier were trained individually and the detection results of frontal face with side face were inosculated to obtain face region. Afterwards, skin color was further validated on the face region to enhance the robustness of the algorithm and reduce the false alarm rate. The results from experiments demonstrated the high speed of test and good real-time. With the CMU face test library on this algorithm, the detection rate can reach 82.7%.
2. In this paper, the face detection algorithm on skin color was comparative analyzed with the algorithm based on Adaboost. Aiming at the issue that the Adaboost algorithm is not effective when faces rotated or being blocked out. In order to overcome this shortcoming, a method to combine the Adaboost algorithm with face detection algorithm was presented in this paper.
3. The Adaboost algorithm and CamShift algorithm were combined well in software development tool such as Visual C++. Firstly, the tracking window based on face region was initialized in detecting process. Then, color hue information model was established to track the follow-up frame. Presented the experiment results, it is showed that the algorithm needs less calculating amounts with fast tracking speed, and without influenced by irregular movement of face. And the phenomenon of face tracking error does not occur when other people appeared on the scene. Furthermore, track loosing does not happen when the face kept out partly. Therefore, the matching and accuracy of the tracking algorithm has been improved greatly. The feasibility and effectiveness of this improved algorithm was verified.
1、针对传统Adaboost检测算法对于侧面人脸检测率不高的情况,对Adaboost人脸检测算法进行相关改进。首先从一个较大的特征集中选择少量关键Haar-Like特征完成高效强分类器的构建;其次将每个强分类器级联成为更加复杂的级联分类器,在此基础上,分别训练正面人脸分类器和侧面人脸分类器,将正面、侧面人脸检测结果进行融合得出人脸区域,并对其进行肤色验证,提高算法的鲁棒性、降低了虚警率。实验证明该算法检测速度快,实时性好,在CMU人脸测试库上进行了实验,检测率达到82.7%。
2、分析对比基于肤色和基于Adaboost的两种人脸检测算法。针对Adaboost算法对人脸偏转和遮挡时检测效果差的问题,为了解决Adaboost人脸检测的这一缺点,本文将其与人脸跟踪算法相融合实现人脸的多角度检测。
3、在软件开发平台Visual C++环境中完成对Adaboost算法与Camshift算法的融合。利用检测过程中人脸区域初始化跟踪窗口,建立肤色的色调信息模型对后续帧进行跟踪。通过实验得知该算法计算量小、跟踪速度快,并且不受人脸不规则运动和随意转动的影响,对场景中出现其他人脸不会出现误跟踪,对人脸被部分遮挡也不会出现跟踪丢失现象,大大提高了跟踪的匹配度和精度,证明了该算法的可行性和有效性。
The face detection and tracking are one of the key techniques in various facial processing algorithms. The research fields of facial image processing include face recognition, gesture estimation, facial expression recognition, video monitoring and so on. And nearly all of these are involved in the area of face detection and face tracking. Based on the collection and analysis of numerous the domestic and international academic thesis and research reports on face detection and tracking in recent years, face detection and tracking algorithms is lucubrated. According to the previous research achievements on face detection and tracking, a face detection algorithm based on Adaboost and CamShift has been realized. Aiming at the experiment conditions, crucial improvements have been achieved by learning from some traditional and classical algorithms of face detection and tracking. The study of this paper focused on the following aspects:
1. Aiming at the poor rate of side face detection by traditional Adaboost algorihtm, some related improvements to Adaboost algorihtm were proposed. Firstly, few key Haar-Like features from a large feature set were selected to generate an effective strong classifier. Secondly, cascades these single strong classifier into a more complex cascade classifier. On that basis, the frontal face classifier and side face classifier were trained individually and the detection results of frontal face with side face were inosculated to obtain face region. Afterwards, skin color was further validated on the face region to enhance the robustness of the algorithm and reduce the false alarm rate. The results from experiments demonstrated the high speed of test and good real-time. With the CMU face test library on this algorithm, the detection rate can reach 82.7%.
2. In this paper, the face detection algorithm on skin color was comparative analyzed with the algorithm based on Adaboost. Aiming at the issue that the Adaboost algorithm is not effective when faces rotated or being blocked out. In order to overcome this shortcoming, a method to combine the Adaboost algorithm with face detection algorithm was presented in this paper.
3. The Adaboost algorithm and CamShift algorithm were combined well in software development tool such as Visual C++. Firstly, the tracking window based on face region was initialized in detecting process. Then, color hue information model was established to track the follow-up frame. Presented the experiment results, it is showed that the algorithm needs less calculating amounts with fast tracking speed, and without influenced by irregular movement of face. And the phenomenon of face tracking error does not occur when other people appeared on the scene. Furthermore, track loosing does not happen when the face kept out partly. Therefore, the matching and accuracy of the tracking algorithm has been improved greatly. The feasibility and effectiveness of this improved algorithm was verified.
引文
[1]田捷,杨鑫编著.生物特征识别技术理论与应用[M].北京:电子工业出版社, 2005.9.
[2]张翠平,苏光人.人脸识别技术综述[J].中国图像图形学报, 2000, Vol. 5(A版) (11): 885-89.
[3]梁路宏,艾海舟等.人脸检测研究综述[J].计算机学报, 2002, 25(5): 449-458.
[4] A. Azarbayejani, B. Horowitz, A. Pentland. Recursive estimation of structure and motion using the relative orientation constraint[C]. In Proceedings of the Computer Vision and Pattern Recognition Conference, 1993: 294-299.
[5] G. R. Bradski. Computer vision face tracking for use in a perceptual user interface[J]. Intel Technology Journal, Q2, 1998: 1-15.
[6] T. Maurer, C. Malsburg. Tracking and learning graphs and pose on image sequences of faces[J]. IEEE Proc. Of ICFQ1995: 176-181.
[7] A. Colmenarez, R. Lopez, T. San Huang. 3D model-based heard tracking[J]. Visual Communication Jose, CA, 1997: 426-434.
[8] M. J. Black, Y. Yacoob. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion[J]. International workshop on Automatic Face and Gesture-Recognition, Zurich, July 1995: 20-23. B.K.P.
[9] S. Basu, L. Essa, A. Pentland. Motion regularization for model-based head tracking[J]. In Proceedings of the International Conference on Pattern Recognition, 1996: 611-616.
[10] M. H. Yang, D. Kriegman, N. Ahuja. Detecting Faces in Images: A Survey[J], IEEE Trans. Pattern Analysis and Machine Intelligence, 2002, Vol. 24(1), 34-58.
[11] G. Yang, T. S. Huang, Human Face Detection in Complex Background, Pattern Recognition, 1994, Vol. 27(1), 53-63.
[12] C. Kotropoulos, I. Pitas. Rule-Based Face Detection in Frontal Views, Proc. Int’l Conf. Acoustics, Speech and Signal Processing,1997, Vol. 4,2537-2540.
[13] S. A. Sirohey. Human Face Segmentation and Identification[J]. Technical Report CS-TR-3176, Univ. of Maryland, 1993.
[14] Han C-C, Liao H-Y, Yu G-J, Chen L-H.Fast face detection via morphology-based pre-processing[J]. Pattern Recognition, 2000, Vol. 33(10): 1701-1712.
[15]梁路宏,艾海舟,何克忠.基于多模板匹配的单人脸检测[J].中国图象图形学报, 1999, 4A(10): 823-830.
[16] Turk M, Pentland A P. Eigen faces for Recognition[J]. Journal of Cognitive Neuroscience, 1991,Vol. 3, 72-86.
[17] S. J. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, H. Wechsler. Tracking Groups of People[J]. Computer Vision and Image Understanding, October 2000, 80(1): 42-56.
[18] Hariharakrishnan K, Schonfeld D, Raffy P, et al. video tracking using block matching[C]. In: IEEE International Conference on Image Processing, 2003, 945-948.
[19] Kjeldsen R, Kender J. Finding skin in color image[A]. In: Proceedings of International Conference on Automatic Face and Gesture Recognition[C]. Killington, Vt. 1996, 312-317.
[20] Eleftheriadis A, Jacquin A. Automatic face location detection and tracking for model-assisted coding of video teleconferencing sequences at low bit-rates[J]. Signal Processing: Image Communication. 1995, 7: 231-248.
[21] Santana M C, ejera M H, Gamez J C. Encara: real-time detection of frontal faces. In: IEEE International Conference on Image Processing, 2003, (3): 881-884.
[22]向桂山,王宣银,梁冬泰.基于人脸肤色和特征的实时检测跟踪算法[J].光电工程, 2007. 04.
[23] M. Kass, A. Witkin, D. Terzopoulos. Snake: Active contour model[J]. Computer Vision, 1987, 1: 321-331.
[24] Yuille A L, Hallinan Peter W, Cohen David S. Feature extraction from faces using deformable templates[J]. International Journal of Computer Vision, 1992, 8(2): 99-111.
[25] Michael Kearns, Leslie G. Valiant, Learning Boolean formulae or finite automata is as hard as factoring[R]. Technical Report TR-14-88, Harvard University Aiken Computation Laboratory, August 1988.
[26] R. E. Schapire. The strength of weak learn ability[J]. Machine Learning, 1990, 5(2): 197-227.
[27]路刚,陈永,范永欣,胡成. Boosting算法研究[J],电脑知识与技术, 2008, 36.
[28] Yoav Freund. Boosting a weak learning algorithm by majority[J]. Information and Computation, 1995, 121(2): 256-285.
[29] P. Viola, M. J. Jones. Robust real-time object detection[J]. Technical Report Series. Compaq Cambridge Research Laboratory, CRL 2001.
[30] R. Lienhart, J. Maydt. An Extended Set of Haar-like Features for Rapid Object Detection[C]. IEEE ICIP. 2002, (1): 900-903.
[31]黄超,谢康林,杜平.基于Adaboost的快速人脸跟踪算法[J].计算机工程, 2004, 12, 30: 373-374.
[32]郭磊,黄秋光. Adaboost人脸检测算法研究及OpenCV实现[J].哈尔滨理工大学学报, 2009, 05.
[33]何斌,马天予等. Visual C++数字图像处理(第二版)[M].北京:人民邮电出版社, 2001.
[34] Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 603-619.
[35]佘九华,王敬东,李鹏.基于Camshift的人脸跟踪算法[J].计算机技术与发展, 2008.09.
[36] M. C. Santana, M. H. Tejera, J. C. Gamez. Encara: real-time detection of frontal faces[C]. IEEE ICIP. 2003, (3): 881-884.
[37]王保保.彩色序列图像中的人脸检测与跟踪[J].西安电子科技大学, 2004.
[38] OpenCV概述[EB/OL]. http://www.opencv.org.cn/index.php/.
[39]边肇祺,张长水,张学工.模式识别[M].北京:清华大学出版社, 2000.
[40]冈萨雷斯.数字图像处理(第二版)[M].北京:电子工业出版, 2005.
[2]张翠平,苏光人.人脸识别技术综述[J].中国图像图形学报, 2000, Vol. 5(A版) (11): 885-89.
[3]梁路宏,艾海舟等.人脸检测研究综述[J].计算机学报, 2002, 25(5): 449-458.
[4] A. Azarbayejani, B. Horowitz, A. Pentland. Recursive estimation of structure and motion using the relative orientation constraint[C]. In Proceedings of the Computer Vision and Pattern Recognition Conference, 1993: 294-299.
[5] G. R. Bradski. Computer vision face tracking for use in a perceptual user interface[J]. Intel Technology Journal, Q2, 1998: 1-15.
[6] T. Maurer, C. Malsburg. Tracking and learning graphs and pose on image sequences of faces[J]. IEEE Proc. Of ICFQ1995: 176-181.
[7] A. Colmenarez, R. Lopez, T. San Huang. 3D model-based heard tracking[J]. Visual Communication Jose, CA, 1997: 426-434.
[8] M. J. Black, Y. Yacoob. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion[J]. International workshop on Automatic Face and Gesture-Recognition, Zurich, July 1995: 20-23. B.K.P.
[9] S. Basu, L. Essa, A. Pentland. Motion regularization for model-based head tracking[J]. In Proceedings of the International Conference on Pattern Recognition, 1996: 611-616.
[10] M. H. Yang, D. Kriegman, N. Ahuja. Detecting Faces in Images: A Survey[J], IEEE Trans. Pattern Analysis and Machine Intelligence, 2002, Vol. 24(1), 34-58.
[11] G. Yang, T. S. Huang, Human Face Detection in Complex Background, Pattern Recognition, 1994, Vol. 27(1), 53-63.
[12] C. Kotropoulos, I. Pitas. Rule-Based Face Detection in Frontal Views, Proc. Int’l Conf. Acoustics, Speech and Signal Processing,1997, Vol. 4,2537-2540.
[13] S. A. Sirohey. Human Face Segmentation and Identification[J]. Technical Report CS-TR-3176, Univ. of Maryland, 1993.
[14] Han C-C, Liao H-Y, Yu G-J, Chen L-H.Fast face detection via morphology-based pre-processing[J]. Pattern Recognition, 2000, Vol. 33(10): 1701-1712.
[15]梁路宏,艾海舟,何克忠.基于多模板匹配的单人脸检测[J].中国图象图形学报, 1999, 4A(10): 823-830.
[16] Turk M, Pentland A P. Eigen faces for Recognition[J]. Journal of Cognitive Neuroscience, 1991,Vol. 3, 72-86.
[17] S. J. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, H. Wechsler. Tracking Groups of People[J]. Computer Vision and Image Understanding, October 2000, 80(1): 42-56.
[18] Hariharakrishnan K, Schonfeld D, Raffy P, et al. video tracking using block matching[C]. In: IEEE International Conference on Image Processing, 2003, 945-948.
[19] Kjeldsen R, Kender J. Finding skin in color image[A]. In: Proceedings of International Conference on Automatic Face and Gesture Recognition[C]. Killington, Vt. 1996, 312-317.
[20] Eleftheriadis A, Jacquin A. Automatic face location detection and tracking for model-assisted coding of video teleconferencing sequences at low bit-rates[J]. Signal Processing: Image Communication. 1995, 7: 231-248.
[21] Santana M C, ejera M H, Gamez J C. Encara: real-time detection of frontal faces. In: IEEE International Conference on Image Processing, 2003, (3): 881-884.
[22]向桂山,王宣银,梁冬泰.基于人脸肤色和特征的实时检测跟踪算法[J].光电工程, 2007. 04.
[23] M. Kass, A. Witkin, D. Terzopoulos. Snake: Active contour model[J]. Computer Vision, 1987, 1: 321-331.
[24] Yuille A L, Hallinan Peter W, Cohen David S. Feature extraction from faces using deformable templates[J]. International Journal of Computer Vision, 1992, 8(2): 99-111.
[25] Michael Kearns, Leslie G. Valiant, Learning Boolean formulae or finite automata is as hard as factoring[R]. Technical Report TR-14-88, Harvard University Aiken Computation Laboratory, August 1988.
[26] R. E. Schapire. The strength of weak learn ability[J]. Machine Learning, 1990, 5(2): 197-227.
[27]路刚,陈永,范永欣,胡成. Boosting算法研究[J],电脑知识与技术, 2008, 36.
[28] Yoav Freund. Boosting a weak learning algorithm by majority[J]. Information and Computation, 1995, 121(2): 256-285.
[29] P. Viola, M. J. Jones. Robust real-time object detection[J]. Technical Report Series. Compaq Cambridge Research Laboratory, CRL 2001.
[30] R. Lienhart, J. Maydt. An Extended Set of Haar-like Features for Rapid Object Detection[C]. IEEE ICIP. 2002, (1): 900-903.
[31]黄超,谢康林,杜平.基于Adaboost的快速人脸跟踪算法[J].计算机工程, 2004, 12, 30: 373-374.
[32]郭磊,黄秋光. Adaboost人脸检测算法研究及OpenCV实现[J].哈尔滨理工大学学报, 2009, 05.
[33]何斌,马天予等. Visual C++数字图像处理(第二版)[M].北京:人民邮电出版社, 2001.
[34] Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 603-619.
[35]佘九华,王敬东,李鹏.基于Camshift的人脸跟踪算法[J].计算机技术与发展, 2008.09.
[36] M. C. Santana, M. H. Tejera, J. C. Gamez. Encara: real-time detection of frontal faces[C]. IEEE ICIP. 2003, (3): 881-884.
[37]王保保.彩色序列图像中的人脸检测与跟踪[J].西安电子科技大学, 2004.
[38] OpenCV概述[EB/OL]. http://www.opencv.org.cn/index.php/.
[39]边肇祺,张长水,张学工.模式识别[M].北京:清华大学出版社, 2000.
[40]冈萨雷斯.数字图像处理(第二版)[M].北京:电子工业出版, 2005.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |