| Vision is the way creatures perceive the world through their eyes.For a long time,people hope to give computers the ability to simulate biological vision,that is,computer vision.With the development of sensor technology and information science,the amount of information that computer vision needs to process is increasing.How to establish the association between visual information is a key problem.Image matching aims to establish visual information consistency between two digital images,serving as a building block in many fields such as autonomous driving,computational photography,and multimedia.In order to achieve accurate image matching,it is necessary to accurately describe the feature of visual information.In recent years,neural networks have become the mainstream feature descriptor.Based on their excellent ability to describe geometric structure features and semantic content features,this thesis uses neural networks and focuses on two specific image matching tasks,namely stereo matching and semantic matching.Stereo matching can parse the depth information of the scene from the images collected by the binocular camera,which is the basis of stereo vision and 3D reconstruction.However,how to improve the matching accuracy is a bottleneck problem in stereo matching.By contrast,semantic matching gets rid of the reliance on binocular cameras,and only requires the same semantic content between images,so the sources of the images are more diverse.However,semantic matching faces the matching challenge caused by the non-rigid appearance differences of objects in the same semantic category.In addition,semantic matching lacks dense training labels,because it is unrealistic to label each pixel manually.In order to improve the performance of stereo matching and semantic matching and break through their technical bottlenecks,this paper studies the matching performance improvement algorithms after in-depth theoretical analysis of related technical issues.The main contributions of this paper are as follows:1.Considering the unlimited search range of candidate labels,this paper proposes a stereo matching algorithm based on adaptive label expansion.It combines edge perception,label propagation,and global optimization to adaptively propagate accurate labels on the image to reduce the workload of label search.Through the proposed rolling optimization strategy,the global optimal label is determined for each pixel,and then the accurate disparity is calculated.Experimental results show that the pixel-level and sub-pixel-level average matching error rates of the proposed algorithm on the Middlebury 3.0 dataset are 13.4% and 38.1%,respectively,smaller than other stereo matching algorithms in the same period;besides,the proposed algorithm can deal well with the matching problem in weak texture regions.2.To establish accurate and smooth semantic matching,this paper proposes two local semantic matching methods.The first method is based on semantic segmentation features and spatial context consistency.On the one hand,introducing semantic segmentation information into the feature can enhance its semantic distinguishability;on the other hand,convolving local context information of semantic matching clues can better guide the matching decision.Experimental results show that the spatial context consistency can bring at least 10.3% accuracy gain,and the semantic segmentation information can further improve the matching accuracy by at least 6.7%.The second method is based on adaptive spatial context aggregation,which uses adaptive aggregation instead of convolution to dynamically aggregate the neighborhood of semantic matching clues.Experimental results show that,compared to convolution operations,the dynamic aggregation is more helpful to achieve a good balance between matching accuracy and smoothness.3.Considering that using convolution or neighborhood aggregation will ignore important global context information,this paper proposes a semantic matching algorithm based on global exploration.By using the attention mechanism,the proposed algorithm on the one hand endows features with global perception capabilities,and on the other hand,it optimizes the direct clues of semantic matching decisions globally.Besides,for different work scenarios,this paper proposes corresponding training strategies,so that the proposed algorithm can serve different scenarios and help them to establish effective matches.The experimental results show that the global exploration mode is more beneficial to semantic matching than the local methods by 0.9%-5.2% accuracy gain,and the proposed algorithm achieves excellent performance in multiple datasets and scenarios. |
