多重约束条件下的LBD描述子与直线段匹配
|
摘要
目的针对直线描述子匹配算法缺乏有效的几何约束,且易受弱纹理、尺度变化的影响,提出一种结合多重约束条件的LBD描述子的直线段匹配算法(LBDs)。方法该算法以LSD算法提取的直线段作为匹配基元,利用SIFT匹配得到的同名点构建同名三角网约束确定候选直线;参考影像上以目标直线段为中心轴建立该直线段的矩形支撑域;根据目标直线段端点及其支撑域四角点在搜索影像上的核线约束建立候选直线段的对应支撑域;利用仿射变换统一目标直线段及候选直线段支撑域的大小;将直线段支撑域分解为大小相等的条形带,通过计算每个条形带的描述符得到该直线段的描述子,依次完成目标直线段与候选直线段LBD描述子的构建;分别计算目标直线段与每个候选直线段描述子向量间的欧氏距离,将满足最近邻距离比准则的候选直线段作为匹配结果;最后选取角度约束对匹配结果检核,确定同名直线。结果实验选取网上公开的3组分别存在角度、旋转、尺度变换的近景影像对作为实验数据,采用LBDs分别对其进行直线段匹配实验,并与其他直线段匹配算法进行对比分析,实验结果表明,LBDs获取同名直线数目约为其他算法的1. 06 1. 41倍,匹配正确率也提高了2. 4 11. 6个百分点,从匹配效率上来看,LBDs更为耗时,但兼顾该算法匹配获得同名直线数目、匹配正确率及运行时间,LBDs的鲁棒性更强,匹配结果的准确性与可靠性较高。结论结合多重约束条件构建的LBD描述子对于存在角度、旋转和尺度变化的影像进行直线匹配过程中具有稳定性。
Objective A new straight-line matching algorithm for line band descriptors( LBDs) combined with multiple con-straints is proposed to solve the typical problems in many straight-line matching algorithms that use descriptors. Such prob-lems include insufficient information utilization between matching straight lines,which are effective geometric constraints,and vulnerability of matching straight lines to the influence of low texture and scale change of images during the matchingprocess. Method Straight line segments are extracted by using a line segment detector method as matching elements,and acorresponding triangulation network established by using SIFT matching points is then used as the constraint region to deter-mine the candidate lines in the searched image. After the candidate lines are selected,a region for band descriptor con-struction is constructed. The construction method is described as follows. A rectangular support region,in which the targetstraight line segment is the central axis in the region,is established in the reference image. Then,the corresponding sup-port region of the candidate straight line segment in the searched image is determined based on epipolar constraints,whichis calculated by the endpoints of the target straight line segment and four corner points of its support region in the referenceimage. The support regions of the target and candidate straight line segments are constructed with the same size by utilizingaffine transformation. After completing the support regions of straight line segments,the regions are divided into a set ofbands,where each band has the same size and the length of the band equals the length of the straight line segment,and theLBDs of straight line segment are obtained by calculating the information of each band in the support region. The descriptorsare calculated based on the gradient values of four directions of pixels,and each band weight coefficient that is along thevertical direction in the support region is controlled by using a Gaussian function. On the basis of the above methods,thematching descriptor construction of LBDs for the target and candidate straight line segments is completed in sequence. Fur-thermore,new LBDs combined with multiple constraints are normalized in obtaining a unit LBD to reduce the influence ofnonlinear illumination changes,and the descriptor is a 40 D vector. Euclidean distances are used as the similarity measurein our algorithm and are determined based on the calculated vectors between the target straight line segment and each candi-date straight line segment descriptor. The candidate straight line segment,which satisfies the nearest neighbor distance ratiocriterion of Euclidean distances,is the matching straight line. In this process,the minimum Euclidean distance and nearestneighbor distance ratio thresholds should be determined,which directly affect the matching performance of the algorithm.Thus,many experiments should be conducted to ensure the accuracy of multi-threshold. The angle constraint,which is be-tween the corresponding straight line and its corresponding epipolar line,is used to evaluate the matching result and deter-mine the final corresponding straight lines. Result Three typical groups of close-range image pairs with angle,rotation,andscale transformation are used as the experimental dataset,which is used to complete the straight line segment matchingexperiments by the proposed algorithm. In comparison with other straight line segment matching algorithms,the matchingresults show that the proposed algorithm is more suitable in different typical close-range image pairs. The conclusions basedon the result analysis are summarized as follows. The successful matches of the proposed algorithm have 1. 06-1. 41 timesmore lines compared with other straight-line matching algorithms,and the proposed algorithm can improve the accuracy ofstraight line matching by 2. 4% to 11. 6%. In terms of matching efficiency,although the proposed algorithm is time-consu-ming,it is robust and achieves accurate and reliable straight line matching results by synthesizing the relevant experimentresults on the number of corresponding matching straight lines,matching accuracy,and running time. Moreover,a highlyaccurate and reliable matching result is obtained. Conclusion Constructed LBDs combined with multiple constraints are sta-ble for line matching of close-range images with angles,rotation,and scale changes. The instability of other descriptorscaused by numerous factors in line matching is improved.
Objective A new straight-line matching algorithm for line band descriptors( LBDs) combined with multiple con-straints is proposed to solve the typical problems in many straight-line matching algorithms that use descriptors. Such prob-lems include insufficient information utilization between matching straight lines,which are effective geometric constraints,and vulnerability of matching straight lines to the influence of low texture and scale change of images during the matchingprocess. Method Straight line segments are extracted by using a line segment detector method as matching elements,and acorresponding triangulation network established by using SIFT matching points is then used as the constraint region to deter-mine the candidate lines in the searched image. After the candidate lines are selected,a region for band descriptor con-struction is constructed. The construction method is described as follows. A rectangular support region,in which the targetstraight line segment is the central axis in the region,is established in the reference image. Then,the corresponding sup-port region of the candidate straight line segment in the searched image is determined based on epipolar constraints,whichis calculated by the endpoints of the target straight line segment and four corner points of its support region in the referenceimage. The support regions of the target and candidate straight line segments are constructed with the same size by utilizingaffine transformation. After completing the support regions of straight line segments,the regions are divided into a set ofbands,where each band has the same size and the length of the band equals the length of the straight line segment,and theLBDs of straight line segment are obtained by calculating the information of each band in the support region. The descriptorsare calculated based on the gradient values of four directions of pixels,and each band weight coefficient that is along thevertical direction in the support region is controlled by using a Gaussian function. On the basis of the above methods,thematching descriptor construction of LBDs for the target and candidate straight line segments is completed in sequence. Fur-thermore,new LBDs combined with multiple constraints are normalized in obtaining a unit LBD to reduce the influence ofnonlinear illumination changes,and the descriptor is a 40 D vector. Euclidean distances are used as the similarity measurein our algorithm and are determined based on the calculated vectors between the target straight line segment and each candi-date straight line segment descriptor. The candidate straight line segment,which satisfies the nearest neighbor distance ratiocriterion of Euclidean distances,is the matching straight line. In this process,the minimum Euclidean distance and nearestneighbor distance ratio thresholds should be determined,which directly affect the matching performance of the algorithm.Thus,many experiments should be conducted to ensure the accuracy of multi-threshold. The angle constraint,which is be-tween the corresponding straight line and its corresponding epipolar line,is used to evaluate the matching result and deter-mine the final corresponding straight lines. Result Three typical groups of close-range image pairs with angle,rotation,andscale transformation are used as the experimental dataset,which is used to complete the straight line segment matchingexperiments by the proposed algorithm. In comparison with other straight line segment matching algorithms,the matchingresults show that the proposed algorithm is more suitable in different typical close-range image pairs. The conclusions basedon the result analysis are summarized as follows. The successful matches of the proposed algorithm have 1. 06-1. 41 timesmore lines compared with other straight-line matching algorithms,and the proposed algorithm can improve the accuracy ofstraight line matching by 2. 4% to 11. 6%. In terms of matching efficiency,although the proposed algorithm is time-consu-ming,it is robust and achieves accurate and reliable straight line matching results by synthesizing the relevant experimentresults on the number of corresponding matching straight lines,matching accuracy,and running time. Moreover,a highlyaccurate and reliable matching result is obtained. Conclusion Constructed LBDs combined with multiple constraints are sta-ble for line matching of close-range images with angles,rotation,and scale changes. The instability of other descriptorscaused by numerous factors in line matching is improved.
引文
[1]Li D R,Zhou G Q.CAD-based line photogrammetry for automatic measurement and reconstruction of industrial object[J].Int Arch Photogramm Remote Sens,1994,30(5):231-240.
[2]Fu D,Wang C,Xu Y D,et al.A new algorism of matching line segments[J].Journal of National University of Defense Technology,2008,30(1):115-119.[傅丹,王超,徐一丹,等.一种直线段匹配的新方法[J].国防科技大学学报,2008,30(1):115-119.][DOI:10.3969/j.issn.1001-2486.2008.01.024]
[3]Shao Z F,Chen M.Line-based matching for high-resolution images with robustness for scale,rotation and illumination[J].Optics and Precision Engineering,2013,21(3):790-798.[邵振峰,陈敏.尺度、旋转以及亮度稳健的高分辨率影像直线特征匹配[J].光学精密工程,2013,21(3):790-798.][DOI:10.3788/OPE.20132103.0790]
[4]Wang Z H,Wu F C,Hu Z Y.MSLD:A robust descriptor for line matching[J].Pattern Recognition,2009,42(5):941-953.[DOI:10.1016/j.patcog.2008.08.035]
[5]Wang J X,Wang W X,Li X M,et al.Line Matching Algorithm for Aerial Image Combining image and object space similarity constraints[J].ISPRS-International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,XLI-B3:783-788.[DOI:10.5194/isprs-archives-XLI-B3-783-2016]
[6]Schmid C,Zisserman A.The geometry and matching of lines and curves over multiple views[J].International Journal of Computer Vision,2000,40(3):199-233.[DOI:10.1023/A:1008135310502]
[7]Zeng F Y,Zhong R F,Song Y,et al.Vehicle panoramic image matching based on epipolar geometry and space forward intersection[J].Journal of Remote Sensing,2014,18(6):1230-1236.[曾凡洋,钟若飞,宋杨,等.车载全景影像核线匹配和空间前方交会[J].遥感学报,2014,18(6):1230-1236.][DOI:10.11834/jrs.20144025]
[8]Wu J,Yao Z X,Cheng M M.Airborne oblique stereo image dense matching by integrating SIFT and SGM algorithm[J].Journal of Remote Sensing,2015,19(3):431-442.[吴军,姚泽鑫,程门门.融合SIFT与SGM的倾斜航空影像密集匹配[J].遥感学报,2015,19(3):431-442.][DOI:10.11834/jrs.20154132]
[9]Song W D,Zhu H,Wang J X,et al.Feature line matching based on the principal component similarity constraint of line segment primitive support region[J].Journal of Signal Processing,2016,32(8):904-910.[宋伟东,朱红,王竞雪,等.线段元支撑区主成分相似性约束特征线匹配[J].信号处理,2016,32(8):904-910.][DOI:10.16798/j.issn.1003-0530.2016.08.04]
[10]Song W D,Zhu H,Wang J X,et al.Line feature matching method based on multiple constraints for close-range images[J].Journal of Image and Graphics,2016,21(6):764-770.[宋伟东,朱红,王竞雪,等.多重约束下的近景影像线特征匹配方法[J].中国图象图形学报,2016,21(6):764-770.][DOI:10.11834/jig.20160609]
[11]Wu B,Zhang Y S,Zhu Q.Integrated point and edge matching on poor textural images constrained by self-adaptive triangulations[J].ISPRS Journal of Photogrammetry and Remote Sensing,2012,68:40-55.[DOI:10.1016/j.isprsjprs.2011.12.005]
[12]Fan B,Wu F C,Hu Z Y.Line matching leveraged by point correspondences[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2010:390-397.[DOI:10.1109/CVPR.2010.5540186]
[13]Fan B,Wu F C,Hu Z Y.Robust line matching through linepoint invariants[J].Pattern Recognition,2012,45(2):794-805.[DOI:10.1016/j.patcog.2011.08.004]
[14]Liang Y,Sheng Y H,Zhang K,et al.Linear Feature matching method based on local affine invariant and epipolar constraint for close-range images[J].Geomatics and Information Science of Wuhan University,2014,39(2):229-233.[梁艳,盛业华,张卡,等.利用局部仿射不变及核线约束的近景影像直线特征匹配[J].武汉大学学报:信息科学版,2014,39(2):229-233.][DOI:10.13203/j.whugis20120611]
[15]Wang J X,Song W D,Wang W X.Line matching algorithm for aerial image based on corresponding points and Z-Plane constraints[J].Acta Geodaetica et Cartographica Sinica,2016,45(1):87-95.[王竞雪,宋伟东,王伟玺.同名点及高程平面约束的航空影像直线匹配算法[J].测绘学报,2016,45(1):87-95.]
[16]Hu H X,Li G.Line matching based on binary relations of geometric attributes[J].Journal of Image and Graphices,2014,19(9):1338-1348.[胡海霞,李钢.几何特性二元关系的直线匹配[J].中国图象图形学报,2014,19(9):1338-1348.][DOI:10.11834/jig.20140911]
[17]Kim H,Lee S.Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs[J].Pattern Recognition Letters,2012,33(10):1349-1363.[DOI:10.1016/j.patrec.2012.03.014]
[18]Wen G J.A global algorithm for straight line Stereo matching based on feature grouping[J].Journal of Software,2006,17(12):2471-2484.[文贡坚.一种基于特征编组的直线立体匹配全局算法[J].软件学报,2006,17(12):2471-2484.][DOI:10.1360/jos172471]
[19]Wang J X,Zhu Q,Wang W X.Reliable line matching algorithm for stereo images with topological relationship[J].Acta Geodaetica et Cartographica Sinica,2017,46(11):1850-1858.[王竞雪,朱庆,王伟玺.顾及拓扑关系的立体影像直线特征可靠匹配算法[J].测绘学报,2017(11):1850-1858.][DOI:10.11947/j.AGCS.2017.20170162]
[20]Schmid C,Zisserman A.Automatic line matching across views[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Puerto Rico:IEEE,1997:666.[DOI:10.1109/CVPR.1997.609397]
[21]Bay H,Ferrari V,van Gool L.Wide-baseline stereo matching with line segments[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Diego,CA,USA:IEEE,2005:329-336.[DOI:10.1109/CVPR.2005.375]
[22]Wang Z H,Wu F C.Mean-standard deviation descriptor and line matching[J].Pattern Recognition and Artificial Intelligence,2009,22(1):32-39.[王志衡,吴福朝.均值-标准差描述子与直线匹配[J].模式识别与人工智能,2009,22(1):32-39.][DOI:10.3969/j.issn.1003-6059.2009.01.006]
[23]Verhagen B,Timofte R,van Gool L.Scale-invariant line descriptors for wide baseline matching[C]//Proceedings of IEEEWinter Conference on Applications of Computer Vision.Steamboat Springs,CO,USA:IEEE,2014:493-500.[DOI:10.1109/WACV.2014.6836061]
[24]Zhang L L,Koch R.An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J].Journal of Visual Communication and Image Representation,2013,24(7):794-805.[DOI:10.1016/j.jvcir.2013.05.006]
[2]Fu D,Wang C,Xu Y D,et al.A new algorism of matching line segments[J].Journal of National University of Defense Technology,2008,30(1):115-119.[傅丹,王超,徐一丹,等.一种直线段匹配的新方法[J].国防科技大学学报,2008,30(1):115-119.][DOI:10.3969/j.issn.1001-2486.2008.01.024]
[3]Shao Z F,Chen M.Line-based matching for high-resolution images with robustness for scale,rotation and illumination[J].Optics and Precision Engineering,2013,21(3):790-798.[邵振峰,陈敏.尺度、旋转以及亮度稳健的高分辨率影像直线特征匹配[J].光学精密工程,2013,21(3):790-798.][DOI:10.3788/OPE.20132103.0790]
[4]Wang Z H,Wu F C,Hu Z Y.MSLD:A robust descriptor for line matching[J].Pattern Recognition,2009,42(5):941-953.[DOI:10.1016/j.patcog.2008.08.035]
[5]Wang J X,Wang W X,Li X M,et al.Line Matching Algorithm for Aerial Image Combining image and object space similarity constraints[J].ISPRS-International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2016,XLI-B3:783-788.[DOI:10.5194/isprs-archives-XLI-B3-783-2016]
[6]Schmid C,Zisserman A.The geometry and matching of lines and curves over multiple views[J].International Journal of Computer Vision,2000,40(3):199-233.[DOI:10.1023/A:1008135310502]
[7]Zeng F Y,Zhong R F,Song Y,et al.Vehicle panoramic image matching based on epipolar geometry and space forward intersection[J].Journal of Remote Sensing,2014,18(6):1230-1236.[曾凡洋,钟若飞,宋杨,等.车载全景影像核线匹配和空间前方交会[J].遥感学报,2014,18(6):1230-1236.][DOI:10.11834/jrs.20144025]
[8]Wu J,Yao Z X,Cheng M M.Airborne oblique stereo image dense matching by integrating SIFT and SGM algorithm[J].Journal of Remote Sensing,2015,19(3):431-442.[吴军,姚泽鑫,程门门.融合SIFT与SGM的倾斜航空影像密集匹配[J].遥感学报,2015,19(3):431-442.][DOI:10.11834/jrs.20154132]
[9]Song W D,Zhu H,Wang J X,et al.Feature line matching based on the principal component similarity constraint of line segment primitive support region[J].Journal of Signal Processing,2016,32(8):904-910.[宋伟东,朱红,王竞雪,等.线段元支撑区主成分相似性约束特征线匹配[J].信号处理,2016,32(8):904-910.][DOI:10.16798/j.issn.1003-0530.2016.08.04]
[10]Song W D,Zhu H,Wang J X,et al.Line feature matching method based on multiple constraints for close-range images[J].Journal of Image and Graphics,2016,21(6):764-770.[宋伟东,朱红,王竞雪,等.多重约束下的近景影像线特征匹配方法[J].中国图象图形学报,2016,21(6):764-770.][DOI:10.11834/jig.20160609]
[11]Wu B,Zhang Y S,Zhu Q.Integrated point and edge matching on poor textural images constrained by self-adaptive triangulations[J].ISPRS Journal of Photogrammetry and Remote Sensing,2012,68:40-55.[DOI:10.1016/j.isprsjprs.2011.12.005]
[12]Fan B,Wu F C,Hu Z Y.Line matching leveraged by point correspondences[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2010:390-397.[DOI:10.1109/CVPR.2010.5540186]
[13]Fan B,Wu F C,Hu Z Y.Robust line matching through linepoint invariants[J].Pattern Recognition,2012,45(2):794-805.[DOI:10.1016/j.patcog.2011.08.004]
[14]Liang Y,Sheng Y H,Zhang K,et al.Linear Feature matching method based on local affine invariant and epipolar constraint for close-range images[J].Geomatics and Information Science of Wuhan University,2014,39(2):229-233.[梁艳,盛业华,张卡,等.利用局部仿射不变及核线约束的近景影像直线特征匹配[J].武汉大学学报:信息科学版,2014,39(2):229-233.][DOI:10.13203/j.whugis20120611]
[15]Wang J X,Song W D,Wang W X.Line matching algorithm for aerial image based on corresponding points and Z-Plane constraints[J].Acta Geodaetica et Cartographica Sinica,2016,45(1):87-95.[王竞雪,宋伟东,王伟玺.同名点及高程平面约束的航空影像直线匹配算法[J].测绘学报,2016,45(1):87-95.]
[16]Hu H X,Li G.Line matching based on binary relations of geometric attributes[J].Journal of Image and Graphices,2014,19(9):1338-1348.[胡海霞,李钢.几何特性二元关系的直线匹配[J].中国图象图形学报,2014,19(9):1338-1348.][DOI:10.11834/jig.20140911]
[17]Kim H,Lee S.Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs[J].Pattern Recognition Letters,2012,33(10):1349-1363.[DOI:10.1016/j.patrec.2012.03.014]
[18]Wen G J.A global algorithm for straight line Stereo matching based on feature grouping[J].Journal of Software,2006,17(12):2471-2484.[文贡坚.一种基于特征编组的直线立体匹配全局算法[J].软件学报,2006,17(12):2471-2484.][DOI:10.1360/jos172471]
[19]Wang J X,Zhu Q,Wang W X.Reliable line matching algorithm for stereo images with topological relationship[J].Acta Geodaetica et Cartographica Sinica,2017,46(11):1850-1858.[王竞雪,朱庆,王伟玺.顾及拓扑关系的立体影像直线特征可靠匹配算法[J].测绘学报,2017(11):1850-1858.][DOI:10.11947/j.AGCS.2017.20170162]
[20]Schmid C,Zisserman A.Automatic line matching across views[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Puerto Rico:IEEE,1997:666.[DOI:10.1109/CVPR.1997.609397]
[21]Bay H,Ferrari V,van Gool L.Wide-baseline stereo matching with line segments[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Diego,CA,USA:IEEE,2005:329-336.[DOI:10.1109/CVPR.2005.375]
[22]Wang Z H,Wu F C.Mean-standard deviation descriptor and line matching[J].Pattern Recognition and Artificial Intelligence,2009,22(1):32-39.[王志衡,吴福朝.均值-标准差描述子与直线匹配[J].模式识别与人工智能,2009,22(1):32-39.][DOI:10.3969/j.issn.1003-6059.2009.01.006]
[23]Verhagen B,Timofte R,van Gool L.Scale-invariant line descriptors for wide baseline matching[C]//Proceedings of IEEEWinter Conference on Applications of Computer Vision.Steamboat Springs,CO,USA:IEEE,2014:493-500.[DOI:10.1109/WACV.2014.6836061]
[24]Zhang L L,Koch R.An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J].Journal of Visual Communication and Image Representation,2013,24(7):794-805.[DOI:10.1016/j.jvcir.2013.05.006]