基于瞳孔空间形态的双眼视线跟踪方法
|
摘要
针对目前人眼视线跟踪的人机交互技术尚不成熟的现状,提出了一种基于立体视觉的瞳孔空间形态的桌面式双目视线跟踪方法。根据低灰度值分布初步定位瞳孔中心;利用瞳孔区域径向导数极坐标图提取瞳孔边缘点坐标,并使用随机样本一致性(RANSAC)对瞳孔边缘进行椭圆拟合;采用定向二进制简单描述符(Oriented brief,ORB)算法配准双目瞳孔边缘点坐标;通过双目立体视觉模型计算得到瞳孔边缘空间点坐标,最后采用最小二乘法计算瞳孔空间形态并解算出视线方向。实验结果表明,瞳孔中心定位速度达300frame/s,双眼视线跟踪速度达15frame/s,视线跟踪最大误差为2.6°。本方法具有较好的准确性、稳健性、实时性,可应用于人眼视线跟踪的人机交互领域中。
Aiming at the status of immaturity for the human-machine interaction technique of eye gaze tracking,a tabletop two-eye gaze tracking method is proposed based on pupil shape in space of stereo vision.With the low grey value distribution,the pupil center is located preliminarily.The radial derivative polar diagram in pupil area is used to extract the pupil edge point coordinates,and the random sample consensus(RANSAC)is used to fit the pupil edge with a suitable ellipse.The two-eye pupil edge point coordinates are matched using the ORB(Oriented brief)algorithm and the pupil edge point coordinates are obtained based on the two-eye stereo vision model.The least square method is finally adopted to calculate the pupil shape in space and the gaze direction is presented.The experimental results show that the positioning speed of pupil center is 300 frame/s,the two-eye gaze tracking speed is 15 frame/s,and the maximum gaze tracking error is 2.6°.It is verified that the proposed method has good accuracy,robustness and real-time performance,and it can be used in the field of human-machine interaction.
Aiming at the status of immaturity for the human-machine interaction technique of eye gaze tracking,a tabletop two-eye gaze tracking method is proposed based on pupil shape in space of stereo vision.With the low grey value distribution,the pupil center is located preliminarily.The radial derivative polar diagram in pupil area is used to extract the pupil edge point coordinates,and the random sample consensus(RANSAC)is used to fit the pupil edge with a suitable ellipse.The two-eye pupil edge point coordinates are matched using the ORB(Oriented brief)algorithm and the pupil edge point coordinates are obtained based on the two-eye stereo vision model.The least square method is finally adopted to calculate the pupil shape in space and the gaze direction is presented.The experimental results show that the positioning speed of pupil center is 300 frame/s,the two-eye gaze tracking speed is 15 frame/s,and the maximum gaze tracking error is 2.6°.It is verified that the proposed method has good accuracy,robustness and real-time performance,and it can be used in the field of human-machine interaction.
引文
[1]Hu J J,Li S M,Chang Y L,et al.Measurement of comfortable contrast range of stereo image based on salient region[J].Acta Optica Sinica,2018,38(2):0215001.胡佳洁,李素梅,常永莉,等.基于显著区域的立体图像舒适对比度范围的测量[J].光学学报,2018,38(2):0215001.
[2]Wang X J,Cai F F,Liu F,et al.Non-contact dynamic real-time eye tracking technology[J].Journal of Frontiers of Computer Science&Technology,2015,9(3):266-278.王向军,蔡方方,刘峰,等.非接触动态实时视线跟踪技术[J].计算机科学与探索,2015,9(3):266-278.
[3]Hou S W,Li B,Xia X B.One-point calibration method for head-mounted eye tracking system[J].Computer Applications&Software,2014,31(9):152-156.侯树卫,李斌,夏小宝.头戴式视线跟踪系统的一点标定方法[J].计算机应用与软件,2014,31(9):152-156.
[4]Wood E,Bulling A.EyeTab:Model-based gaze estimation on unmodified tablet computers[C]∥Proceedings of the symposium on eye tracking research and applications-ETRA′14,March 26-28,2014,Safety Harbor,Florida.New York:ACM,2014:207-210.
[5]Zhang H M,Luo P Y,Sui J F.Research on pupil size detection algorithm based on image processing[J].Semiconductor Optoelectronics,2017,38(1):136-141.张慧敏,罗珮允,隋建峰.基于图像处理的瞳孔大小检测算法研究[J].半导体光电,2017,38(1):136-141.
[6]Mao S B.Exact pupil detection algorithm combining Hough transformation and contour matching[J].Journal of Computer Applications,2016,36(5):1415-1420.毛顺兵.Hough变换和轮廓匹配相结合的瞳孔精确检测算法[J].计算机应用,2016,36(5):1415-1420.
[7]Gu K K,Fu W W,Dong Y F,et al.Rapid pupil detection based on improved contour tracking[J].Infrared Technology,2017,39(6):574-578.顾可可,付威威,董月芳,等.基于改进轮廓跟踪的瞳孔快速检测[J].红外技术,2017,39(6):574-578.
[8]Jiang T P,Wang X F,Pan X,et al.Method of improved exact pupil measurement based on face image[J].Journal of Chinese Computer Systems,2018,39(4):842-846.姜太平,汪小帆,潘祥,等.一种改进的基于人脸图像的瞳孔精确检测方法[J].小型微型计算机系统,2018,39(4):842-846.
[9]Chen J,Zheng S H,Pan L,et al.Pupil detection method based on ellipse fitting and parameter passing[J].Chinese Journal of Scientific Instrument,2014,35(8):1890-1899.陈健,郑绍华,潘林,等.结合椭圆拟合与参数传递的瞳孔检测方法[J].仪器仪表学报,2014,35(8):1890-1899.
[10]Liu R A,Jin S J,Li W Q,et al.Subpixel edge detection and center localization of the pupil[J].Computer Engineering&Applications,2007,43(5):200-203.刘瑞安,靳世久,李文清,等.瞳孔亚像素边缘检测与中心定位[J].计算机工程与应用,2007,43(5):200-203.
[11]Xia X B,Li B.A two-level pupil location method relying on 3Deyeball model[J].Journal of University of Science and Technology of China,2014,44(2):153-159.夏小宝,李斌.一种依赖3D眼球模型的两级瞳孔定位算法[J].中国科学技术大学学报,2014,44(2):153-159.
[12]Liu Y Y,Feng P,Long Z R,et al.Research and implementation of binocular location system based on region of interest segmentation[J].Laser&Optoelectronics Progress,2018,55(5):051102.刘远远,冯鹏,龙邹荣,等.基于靶标区域分割的双目定位系统研究与实现[J].激光与光电子学进展,2018,55(5):051102.
[13]Wen Q C,Da F P,Fang X.Calibration for gaze tracking system based on binocular stereo vision[J].Acta Optica Sinica,2012,32(10):1015002.温晴川,达飞鹏,方旭.基于双目立体视觉的视线跟踪系统标定[J].光学学报,2012,32(10):1015002.
[14]Wu Q Y,Zeng Z,Zhang B C,et al.A 360°threedimensional measurement system and its calibration[J].Chinese Journal of Lasers,2017,44(4):0404002.吴庆阳,曾增,张佰春,等.一种新的360°三维测量系统及标定技术[J].中国激光,2017,44(4):0404002.
[2]Wang X J,Cai F F,Liu F,et al.Non-contact dynamic real-time eye tracking technology[J].Journal of Frontiers of Computer Science&Technology,2015,9(3):266-278.王向军,蔡方方,刘峰,等.非接触动态实时视线跟踪技术[J].计算机科学与探索,2015,9(3):266-278.
[3]Hou S W,Li B,Xia X B.One-point calibration method for head-mounted eye tracking system[J].Computer Applications&Software,2014,31(9):152-156.侯树卫,李斌,夏小宝.头戴式视线跟踪系统的一点标定方法[J].计算机应用与软件,2014,31(9):152-156.
[4]Wood E,Bulling A.EyeTab:Model-based gaze estimation on unmodified tablet computers[C]∥Proceedings of the symposium on eye tracking research and applications-ETRA′14,March 26-28,2014,Safety Harbor,Florida.New York:ACM,2014:207-210.
[5]Zhang H M,Luo P Y,Sui J F.Research on pupil size detection algorithm based on image processing[J].Semiconductor Optoelectronics,2017,38(1):136-141.张慧敏,罗珮允,隋建峰.基于图像处理的瞳孔大小检测算法研究[J].半导体光电,2017,38(1):136-141.
[6]Mao S B.Exact pupil detection algorithm combining Hough transformation and contour matching[J].Journal of Computer Applications,2016,36(5):1415-1420.毛顺兵.Hough变换和轮廓匹配相结合的瞳孔精确检测算法[J].计算机应用,2016,36(5):1415-1420.
[7]Gu K K,Fu W W,Dong Y F,et al.Rapid pupil detection based on improved contour tracking[J].Infrared Technology,2017,39(6):574-578.顾可可,付威威,董月芳,等.基于改进轮廓跟踪的瞳孔快速检测[J].红外技术,2017,39(6):574-578.
[8]Jiang T P,Wang X F,Pan X,et al.Method of improved exact pupil measurement based on face image[J].Journal of Chinese Computer Systems,2018,39(4):842-846.姜太平,汪小帆,潘祥,等.一种改进的基于人脸图像的瞳孔精确检测方法[J].小型微型计算机系统,2018,39(4):842-846.
[9]Chen J,Zheng S H,Pan L,et al.Pupil detection method based on ellipse fitting and parameter passing[J].Chinese Journal of Scientific Instrument,2014,35(8):1890-1899.陈健,郑绍华,潘林,等.结合椭圆拟合与参数传递的瞳孔检测方法[J].仪器仪表学报,2014,35(8):1890-1899.
[10]Liu R A,Jin S J,Li W Q,et al.Subpixel edge detection and center localization of the pupil[J].Computer Engineering&Applications,2007,43(5):200-203.刘瑞安,靳世久,李文清,等.瞳孔亚像素边缘检测与中心定位[J].计算机工程与应用,2007,43(5):200-203.
[11]Xia X B,Li B.A two-level pupil location method relying on 3Deyeball model[J].Journal of University of Science and Technology of China,2014,44(2):153-159.夏小宝,李斌.一种依赖3D眼球模型的两级瞳孔定位算法[J].中国科学技术大学学报,2014,44(2):153-159.
[12]Liu Y Y,Feng P,Long Z R,et al.Research and implementation of binocular location system based on region of interest segmentation[J].Laser&Optoelectronics Progress,2018,55(5):051102.刘远远,冯鹏,龙邹荣,等.基于靶标区域分割的双目定位系统研究与实现[J].激光与光电子学进展,2018,55(5):051102.
[13]Wen Q C,Da F P,Fang X.Calibration for gaze tracking system based on binocular stereo vision[J].Acta Optica Sinica,2012,32(10):1015002.温晴川,达飞鹏,方旭.基于双目立体视觉的视线跟踪系统标定[J].光学学报,2012,32(10):1015002.
[14]Wu Q Y,Zeng Z,Zhang B C,et al.A 360°threedimensional measurement system and its calibration[J].Chinese Journal of Lasers,2017,44(4):0404002.吴庆阳,曾增,张佰春,等.一种新的360°三维测量系统及标定技术[J].中国激光,2017,44(4):0404002.