基于单目视觉的视觉伺服与位姿估计
|
摘要
机器人视觉伺服技术涉及到机器人技术,图像采集和处理,控制理论等多学科内容。随着计算机性能的增强以及图像处理硬件和CCD摄像机等硬件的快速发展,机器人视觉伺服技术在上世纪90年代获得了较快的发展。国内外的大批学者在摄像机的标定技术,快速图像处理和视觉伺服控制理论等方面进行了深入的研究。
本文针对机器人视觉伺服系统中的图像特征点的提取和匹配,基本矩阵估计,位姿估计和视觉伺服仿真实验几个方面作了研究。机器人视觉伺服系统的控制结构的控制策略主要有三种形式,不同的控制策略处理图像的反馈信息的方式不同。本文以VST仿真工具为基础,研究了IBVS控制的问题,建立了摄像机模型,通过simulink仿真实现了基于图像的视觉伺服控制;又结合了PUMA560工业机器人与极线几何原理为基础,建立了摄像机与PUMA560的模型。通过EGT工具箱设定控制算法,实现了机器人模型与摄像机模型的联合控制仿真。从仿真结果分析,这两种控制策略均能较好的满足实验要求。
本文对六轮移动平台的位姿估计与姿态调整进行了仿真实验。通过获取摄像机在不同位姿下获得图像,通过计算图像间对应特征点的信息来估计本质矩阵。通过本质矩阵分解获得六轮移动平台位姿参数。按照基于规则的姿态调整策略实现对移动平台的姿态调整,实验结果表明,姿态调整策略满足六轮移动平台稳定性要求。
The robot visual servoing is the fusion of multi-disciplinary contents including robot kinematics and dynamics, image acquisition and processing, control theory and so on. With the rapid development of the computer science as well as the enhancement of the hardware of image processing and the CCD Camera, the robot vision servo control technology has obtained the quick development since the 1990s. Depth study on camera calibration technique, rapid image processing and visual servoing control theory and relevant technology have been conducted by a large number of domestic and foreign scholars.
In this paper, image feature extraction and matching, fundamental matrix estimation, pose estimation and simulation of visual servoing have been studied. Robot obtains the informations of image features from image processing system that captured pictures from the CCD camera. There are three main forms of the strategies of the visual servoing control structure, The approach of the image feedback is different in different ways. A simulink experiment of IBVS control simulation is done based on Visual Servoing Toolbox. And combination with industrial robots puma560 and Epipolar Geometry Toolbox, a puma560 robot model and a canera model is set up ,realize the control simulation between robot model and camera. These two control strategies are able to better meet the test requirements, from analysis of simulation results.
Wheel mobile platform pose estimation and adjustment experiment simulation is the other part of this paper. Essential matrix is estimated from the matching images that captured by camera. Wheel mobile platform pose parameters can be calculated from the decomposition of the essential matrix. To adjust the pose of the robot, a rule-based pose adjustment strategy is used. The experimental results show that the rule-based pose adjustment strategy meets the stability requirements of the wheel mobile platform.
本文针对机器人视觉伺服系统中的图像特征点的提取和匹配,基本矩阵估计,位姿估计和视觉伺服仿真实验几个方面作了研究。机器人视觉伺服系统的控制结构的控制策略主要有三种形式,不同的控制策略处理图像的反馈信息的方式不同。本文以VST仿真工具为基础,研究了IBVS控制的问题,建立了摄像机模型,通过simulink仿真实现了基于图像的视觉伺服控制;又结合了PUMA560工业机器人与极线几何原理为基础,建立了摄像机与PUMA560的模型。通过EGT工具箱设定控制算法,实现了机器人模型与摄像机模型的联合控制仿真。从仿真结果分析,这两种控制策略均能较好的满足实验要求。
本文对六轮移动平台的位姿估计与姿态调整进行了仿真实验。通过获取摄像机在不同位姿下获得图像,通过计算图像间对应特征点的信息来估计本质矩阵。通过本质矩阵分解获得六轮移动平台位姿参数。按照基于规则的姿态调整策略实现对移动平台的姿态调整,实验结果表明,姿态调整策略满足六轮移动平台稳定性要求。
The robot visual servoing is the fusion of multi-disciplinary contents including robot kinematics and dynamics, image acquisition and processing, control theory and so on. With the rapid development of the computer science as well as the enhancement of the hardware of image processing and the CCD Camera, the robot vision servo control technology has obtained the quick development since the 1990s. Depth study on camera calibration technique, rapid image processing and visual servoing control theory and relevant technology have been conducted by a large number of domestic and foreign scholars.
In this paper, image feature extraction and matching, fundamental matrix estimation, pose estimation and simulation of visual servoing have been studied. Robot obtains the informations of image features from image processing system that captured pictures from the CCD camera. There are three main forms of the strategies of the visual servoing control structure, The approach of the image feedback is different in different ways. A simulink experiment of IBVS control simulation is done based on Visual Servoing Toolbox. And combination with industrial robots puma560 and Epipolar Geometry Toolbox, a puma560 robot model and a canera model is set up ,realize the control simulation between robot model and camera. These two control strategies are able to better meet the test requirements, from analysis of simulation results.
Wheel mobile platform pose estimation and adjustment experiment simulation is the other part of this paper. Essential matrix is estimated from the matching images that captured by camera. Wheel mobile platform pose parameters can be calculated from the decomposition of the essential matrix. To adjust the pose of the robot, a rule-based pose adjustment strategy is used. The experimental results show that the rule-based pose adjustment strategy meets the stability requirements of the wheel mobile platform.
引文
[1]张广军.机器视觉.北京:科学出版社, 2005
[2]马颂德,张正友.计算机视觉--计算理论与算法基础.北京:科学出版社, 1998
[3] P. I. Corke, Visual Control of Robots: High-Performance Visual Servoing. New York: Wiley, 1996
[4]王社阳.机器人视觉伺服系统的若干问题研究: [哈尔滨工业大学博士学位论文].哈尔滨:哈尔滨工业大学, 2006
[5]刘军传,张玉茹,李振.脑外科机器人视觉伺服研究.北京航空航天大学学报, 2007, 33(3): 370-37
[6] Hill J and Park W T. Real time control of a robot with a mobile camera. In Froc. 9t" ISIR. Washington. D. C, 1979: 233-246
[7] Weiss L E, Sanderson A C. Image-based visual servo control using relational graph error signals. Proc·IEEE, 1980: 1074-1077
[8] P. I. Corke. The Machine Vision Toolbox . Ieee Robotics& Automation Magazine, 2005: 16-25
[9] P. I. Corke. A robotics toolbox for MATLAB. Robotics and Automation Magazine, 1996, 3(1): 24-32.
[10] G. L. Mariottini, D. Prattichizzo. The Epipolar Geometry Toolbox (EGT) for Matlab v1.1. Technical Report 07-21-3-DII, University of Siena, July 2004 Siena, Italy
[11] G. L. Mariottini, D. Prattichizzo. The Epipolar Geometry Toolbox: multiple view geometry and visual servoing for MATLAB. IEEE Robotics & Automation Magazine, vol. 12, Dec. 2005
[12] Chaumette F, Hutchinson S. Visual servo control part I: basic approaches.IEEE Robotics & Automation Magazine , 2006 (12) : 82-90
[13] Chaumette F, Hutchinson S. Visual servo control part II: advanced approaches . IEEE Robotics &Automation Magazine , 2007 (3) : 109-118
[14] The Visual Servoing Toolbox, 2003. http://vstoolbox. sourceforge. net/
[15] Jacopo Piazzi, Domenico Prattichizzo, and Antonio Vicino. Visual Servoing AlongEpipoles. Control Problems in Robotics, 2003, 4: 215-231
[16] Paulo, A. Paris, C. Christo et al. Uncalibrated Visual Servoing in 3Dworkspace. in: A. Campilho and M. Kamel (Eds. ). ICIAR 2006, LNCS 4142. Berlin Heidelberg: Springer-Verlag, 2006. 225-236
[17]李国栋,周友行,邓胜达等.无标定视觉伺服机器人系统研究.机械制造与研究. 2007, 36 (2): 67-69
[18] E. Malis, F. Chaumette, and S. Boudet. 2-1/2D visual servoing. IEEE Trans. Robot. Automat. , 1999, 4(15): 238-250
[19] P. Corke and S. Hutchinson. A new partitioned approach to imagebased visual servo control. IEEE Trans. Robot. Automat. , 2001, 17(4): 507–515
[20]杨延西.基于图像的智能机器人视觉伺服系统: [西安理工大学博士学位论文].西安:西安理工大学, 2003
[21]徐庆坤.机器人无标定视觉伺服系统的研究: [西安理工大学硕士学位论文].西安:西安理工大学, 2007
[22]郭蓝彬.基于图像雅可比矩阵的无标定工业机器人视觉伺服: [西安理工大学硕士学位论文].西安:西安理工大学, 2007
[23]白强.基于视觉反馈的智能小车系统研究: [中南大学硕士学位论文].长沙:中南大学, 2007
[24]高健.基于视觉的移动机器人运动控制研究: [华中科技大学硕士学位论文].武汉:华中科技大学, 2005
[25]林靖,陈辉堂,王月娟.机器人视觉伺服系统的研究.控制理论与应用. 2000, 17(4): 476-481
[26]诸静.机器人视觉系统研究与开发: [浙江大学硕士学位论文].杭州:浙江大学, 2003
[27]宗晓萍,淮小利,王培光.基于图像的PUMA560机器人视觉伺服系统仿真,机床与液压. 2007, 35(10): 161-164
[28]赵杰,李戈,蔡鹤皋.基于PUMA机器人的视觉伺服控制实验研究.哈尔滨工业大学学报. 2002 , 34(5): 620-623
[29]刘锋.基于PUMA560机器人的视觉伺服控制系统的研究: [兰州理工大学硕士学位论文].兰州:兰州理工大学, 2007
[30] Weiss L E, Sanderson A C and Neuman C P. Dynamic sensor-based control of robotswith visual feedback, IEEE Journal of Robotics and utomations, 1987, 3(5): 404-417.
[31] S. Benhimane, E. Malis. Homography-based 2d visual servoing. IEEE ICRA’06, Orlando, Fl, May 2006
[32]黎志刚,段锁林,赵建英等.机器人视觉伺服控制及应用研究的现状.太原科技大学学报. 2007, 28(1): 24-30
[33]王麟昆,徐德,谭民.机器人视觉伺服研究发展.机器人, 2004, 26 (3):277-282 WAN G Linkun , XU De , TAN Min. Survey of research on robotic visual servoing. Robot , 2004 , 26 (3) : 277-282
[34]赵清杰,连广宇,孙增圻.机器人视觉伺服综述.控制与决策, 2001, 16(6): 849-853 ZHAO Qingjie , L IAN Guangyu , SUN Zengqi. Survey of robot visual servoing. Control and Decision, 2001, 16(6): 849-853
[35]方勇纯.机器人视觉伺服研究综述.智能系统学报, 2008, 3(2): 109-113
[36] S. Hutchinson, G. Hager, and P. Corke, . A tutorial on visual servo control. IEEE Trans. Robot. Automat. , 1996,12: 651-670
[37] B. Espiau, F. Chaumette, and P. Rives, . A new approach to visual servoing in robotics. IEEE Trans. Robotics and Automation, 1992,8 : 313-326
[38] E. Malis,“Improving vision-based control using efficient second-order minimization techniques,”in Proc. IEEE Int. Conf. Robot. Automat, 2004. 1843-1848
[39] R. Hartley and A. Zisserman. Multiple view in computer vision. Cambridge University Press, 2000
[40]胡凌山,朱齐丹.算机视觉中基本矩阵的估计方法.应用科技. 2005, 32(10):40-43
[41]单欣,王耀明,董建萍.基于RANSAC算法的基本矩阵估计的匹配方法.上海电机学院学报. 2006, 9(4):66-69
[42] P. D. Kovesi. MATLAB and Octave Functions for Computer Vision and Image Processing. School of Computer Science&Software Engineering, The University of WesternAustralia. Availablefrom:http://www.csse.uwa.edu.au/~pk/research/matlabfns/
[43]郭红玉,王鉴.一种基于基本矩阵估计的RANSAC图像匹配方法.红外. 2008, 29(2):5-8
[44]熊有伦.机器人技术基础.武汉:华中科技大学出版社, 2004
[45] P. Rives. Visual servoing based on epipolar geometry. In InternationalConference on Intelligent Robots and Systems, 2000, 1: 602-607
[46]杨逢瑜,王其磊,关红艳等.基于极线几何的机器人视觉伺服控制系统分析.西华大学学报2009, 28(1):14-16
[47]刘盾.六轮腿式移动机器人在非平整地面上的位姿分析、检测和控制: [南京理工大学硕士学位论文].南京:南京理工大学, 2007
[48]侯国庆.移动机器人行走系统的运动学分析和稳定性研究: [河北工业大学硕士学位论文].石家庄:河北工业大学, 2007
[49]赵锐.基于单目视觉的物体位姿侧量方法研究: [合肥工业大学硕士学位论文].合肥:合肥工业大学, 2005
[50]郝颖明,朱枫,欧锦军.目标位姿测量中的三维视觉方法.中国图象图形学报.2002, 7(12):1247-1251
[51]阮利锋,王赓,盛焕烨.基于标志点识别的三维位姿测量方法.计算机应用. 2008, 28(11):2856-2862
[52]王杨.基于单目视觉的刚体空间位姿估计研究: [华中科技大学硕士学位论文].武汉:华中科技大学, 2008
[53]张岩,廖士中.二维凸包算法Graham的设计与实现.牡丹江师范学院学报, 1999, 2:1-2
[54] http://hi. baidu. com/greation
[2]马颂德,张正友.计算机视觉--计算理论与算法基础.北京:科学出版社, 1998
[3] P. I. Corke, Visual Control of Robots: High-Performance Visual Servoing. New York: Wiley, 1996
[4]王社阳.机器人视觉伺服系统的若干问题研究: [哈尔滨工业大学博士学位论文].哈尔滨:哈尔滨工业大学, 2006
[5]刘军传,张玉茹,李振.脑外科机器人视觉伺服研究.北京航空航天大学学报, 2007, 33(3): 370-37
[6] Hill J and Park W T. Real time control of a robot with a mobile camera. In Froc. 9t" ISIR. Washington. D. C, 1979: 233-246
[7] Weiss L E, Sanderson A C. Image-based visual servo control using relational graph error signals. Proc·IEEE, 1980: 1074-1077
[8] P. I. Corke. The Machine Vision Toolbox . Ieee Robotics& Automation Magazine, 2005: 16-25
[9] P. I. Corke. A robotics toolbox for MATLAB. Robotics and Automation Magazine, 1996, 3(1): 24-32.
[10] G. L. Mariottini, D. Prattichizzo. The Epipolar Geometry Toolbox (EGT) for Matlab v1.1. Technical Report 07-21-3-DII, University of Siena, July 2004 Siena, Italy
[11] G. L. Mariottini, D. Prattichizzo. The Epipolar Geometry Toolbox: multiple view geometry and visual servoing for MATLAB. IEEE Robotics & Automation Magazine, vol. 12, Dec. 2005
[12] Chaumette F, Hutchinson S. Visual servo control part I: basic approaches.IEEE Robotics & Automation Magazine , 2006 (12) : 82-90
[13] Chaumette F, Hutchinson S. Visual servo control part II: advanced approaches . IEEE Robotics &Automation Magazine , 2007 (3) : 109-118
[14] The Visual Servoing Toolbox, 2003. http://vstoolbox. sourceforge. net/
[15] Jacopo Piazzi, Domenico Prattichizzo, and Antonio Vicino. Visual Servoing AlongEpipoles. Control Problems in Robotics, 2003, 4: 215-231
[16] Paulo, A. Paris, C. Christo et al. Uncalibrated Visual Servoing in 3Dworkspace. in: A. Campilho and M. Kamel (Eds. ). ICIAR 2006, LNCS 4142. Berlin Heidelberg: Springer-Verlag, 2006. 225-236
[17]李国栋,周友行,邓胜达等.无标定视觉伺服机器人系统研究.机械制造与研究. 2007, 36 (2): 67-69
[18] E. Malis, F. Chaumette, and S. Boudet. 2-1/2D visual servoing. IEEE Trans. Robot. Automat. , 1999, 4(15): 238-250
[19] P. Corke and S. Hutchinson. A new partitioned approach to imagebased visual servo control. IEEE Trans. Robot. Automat. , 2001, 17(4): 507–515
[20]杨延西.基于图像的智能机器人视觉伺服系统: [西安理工大学博士学位论文].西安:西安理工大学, 2003
[21]徐庆坤.机器人无标定视觉伺服系统的研究: [西安理工大学硕士学位论文].西安:西安理工大学, 2007
[22]郭蓝彬.基于图像雅可比矩阵的无标定工业机器人视觉伺服: [西安理工大学硕士学位论文].西安:西安理工大学, 2007
[23]白强.基于视觉反馈的智能小车系统研究: [中南大学硕士学位论文].长沙:中南大学, 2007
[24]高健.基于视觉的移动机器人运动控制研究: [华中科技大学硕士学位论文].武汉:华中科技大学, 2005
[25]林靖,陈辉堂,王月娟.机器人视觉伺服系统的研究.控制理论与应用. 2000, 17(4): 476-481
[26]诸静.机器人视觉系统研究与开发: [浙江大学硕士学位论文].杭州:浙江大学, 2003
[27]宗晓萍,淮小利,王培光.基于图像的PUMA560机器人视觉伺服系统仿真,机床与液压. 2007, 35(10): 161-164
[28]赵杰,李戈,蔡鹤皋.基于PUMA机器人的视觉伺服控制实验研究.哈尔滨工业大学学报. 2002 , 34(5): 620-623
[29]刘锋.基于PUMA560机器人的视觉伺服控制系统的研究: [兰州理工大学硕士学位论文].兰州:兰州理工大学, 2007
[30] Weiss L E, Sanderson A C and Neuman C P. Dynamic sensor-based control of robotswith visual feedback, IEEE Journal of Robotics and utomations, 1987, 3(5): 404-417.
[31] S. Benhimane, E. Malis. Homography-based 2d visual servoing. IEEE ICRA’06, Orlando, Fl, May 2006
[32]黎志刚,段锁林,赵建英等.机器人视觉伺服控制及应用研究的现状.太原科技大学学报. 2007, 28(1): 24-30
[33]王麟昆,徐德,谭民.机器人视觉伺服研究发展.机器人, 2004, 26 (3):277-282 WAN G Linkun , XU De , TAN Min. Survey of research on robotic visual servoing. Robot , 2004 , 26 (3) : 277-282
[34]赵清杰,连广宇,孙增圻.机器人视觉伺服综述.控制与决策, 2001, 16(6): 849-853 ZHAO Qingjie , L IAN Guangyu , SUN Zengqi. Survey of robot visual servoing. Control and Decision, 2001, 16(6): 849-853
[35]方勇纯.机器人视觉伺服研究综述.智能系统学报, 2008, 3(2): 109-113
[36] S. Hutchinson, G. Hager, and P. Corke, . A tutorial on visual servo control. IEEE Trans. Robot. Automat. , 1996,12: 651-670
[37] B. Espiau, F. Chaumette, and P. Rives, . A new approach to visual servoing in robotics. IEEE Trans. Robotics and Automation, 1992,8 : 313-326
[38] E. Malis,“Improving vision-based control using efficient second-order minimization techniques,”in Proc. IEEE Int. Conf. Robot. Automat, 2004. 1843-1848
[39] R. Hartley and A. Zisserman. Multiple view in computer vision. Cambridge University Press, 2000
[40]胡凌山,朱齐丹.算机视觉中基本矩阵的估计方法.应用科技. 2005, 32(10):40-43
[41]单欣,王耀明,董建萍.基于RANSAC算法的基本矩阵估计的匹配方法.上海电机学院学报. 2006, 9(4):66-69
[42] P. D. Kovesi. MATLAB and Octave Functions for Computer Vision and Image Processing. School of Computer Science&Software Engineering, The University of WesternAustralia. Availablefrom:http://www.csse.uwa.edu.au/~pk/research/matlabfns/
[43]郭红玉,王鉴.一种基于基本矩阵估计的RANSAC图像匹配方法.红外. 2008, 29(2):5-8
[44]熊有伦.机器人技术基础.武汉:华中科技大学出版社, 2004
[45] P. Rives. Visual servoing based on epipolar geometry. In InternationalConference on Intelligent Robots and Systems, 2000, 1: 602-607
[46]杨逢瑜,王其磊,关红艳等.基于极线几何的机器人视觉伺服控制系统分析.西华大学学报2009, 28(1):14-16
[47]刘盾.六轮腿式移动机器人在非平整地面上的位姿分析、检测和控制: [南京理工大学硕士学位论文].南京:南京理工大学, 2007
[48]侯国庆.移动机器人行走系统的运动学分析和稳定性研究: [河北工业大学硕士学位论文].石家庄:河北工业大学, 2007
[49]赵锐.基于单目视觉的物体位姿侧量方法研究: [合肥工业大学硕士学位论文].合肥:合肥工业大学, 2005
[50]郝颖明,朱枫,欧锦军.目标位姿测量中的三维视觉方法.中国图象图形学报.2002, 7(12):1247-1251
[51]阮利锋,王赓,盛焕烨.基于标志点识别的三维位姿测量方法.计算机应用. 2008, 28(11):2856-2862
[52]王杨.基于单目视觉的刚体空间位姿估计研究: [华中科技大学硕士学位论文].武汉:华中科技大学, 2008
[53]张岩,廖士中.二维凸包算法Graham的设计与实现.牡丹江师范学院学报, 1999, 2:1-2
[54] http://hi. baidu. com/greation