Research On Binocular Vision Matching Algorithm And Its Key Technologie

As one of the key technologies of 3D reconstruction,stereo vision is widely used in the fields of automatic driving,robot positioning and industrial intelligent manufacturing due to its simple hardware equipment and strong robustness to scene changes.The matching algorithm is the core technology and research focus in stereo vision.Its purpose is to extract high-precision disparity map,and then obtain the accurate depth information of 3D scene.However,with the rapid upgrade of hardware devices,there are many challenging problems to be solved in stereo vision,such as the matching algorithm is prone to mismatching in weak texture,no texture,repeated texture,illumination changes,occlusion and depth edges regions in high-resolution images,high computational complexity of global algorithms,poor cross-domain generalization,unsupervised traditional algorithms and deep learning methods can not be efficiently integrated,etc.These problems make the matching algorithms unable to quickly obtain high-precision disparity maps.This paper conducts research on the above problems,the main innovations and contributions include the following aspects:(1)Aiming at the difficulty of matching the illumination changes,reflections,weak textures,no textures,and repeated texture areas in high-resolution images,a local extended global matching algorithm(HLocal Exp-CM)based on the fusion of confidence and multi-scale context information is proposed,which significantly improves the matching effect and disparity accuracy of the above ill-conditioned regions.First,the algorithm integrates local multi-scale context information on the grid structure to enhance the feature representation ability for the above ill-conditioned regions.Secondly,according to the change of the energy value of data term in the global algorithm integrating local multi-scale context information,a confidence module is designed to select the accurate confidence disparity planes,and the disparity planes with high confidence are used the input of the global algorithm for optimization.Finally,a two-stage layer-by-layer optimization strategy is proposed,the local algorithm integrating the multi-scale context information is first used to narrow the search range of the disparity plane,and then the confidence estimation module and the global algorithm integrating the multi-scale context information are used to iterative optimization alternately to obtain high-precision disparity map.The whole algorithm is optimized on multi-layer grouped grid structure.The experimental results show that the algorithm has high overall accuracy and strong robustness to various scenarios.(2)Since the HLocalExp-CM algorithm has high computational complexity,the poor processing for depth edges,and its matching effect is limited by the patch size,etc.First,a local Patch Match algorithm based on superpixel cut(LPSC)is proposed.The algorithm is the first to propose a fusion of color,gradient and binary image information to calculate the weight between two pixels,which enhance the ability of algorithm to identify weak textures,repeated textures and edges.The algorithm quickly extract feature points with local plane representation ability in each superpixel,and optimizes the feature points by non-local matching algorithm through spatial propagation and plane optimization to obtain a set of candidate disparity planes.The set of candidate disparity planes is optimized by a non-local matching algorithm for all pixels in the corresponding superpixel to obtain the disparity of each pixel quickly.Moreover,in order to achieve the parallel accelerated computation of the algorithm,the local patches surrounding superpixels and mask operation are employed.Secondly,in order to further improve the matching effect for occlusion,weak texture and depth edge regions,a superpixel cut-based local expansion algorithm(LESC)is proposed.The algorithm combines the layer-by-layer optimization idea and local expansion method of HLocal Exp-CM on the basis of LPSC algorithm.The algorithm performs the non-local algorithm of neighborhood propagation and plane optimization in the feature point and superpixel optimization stage by layerby-layer,and then performs alternate optimization at the maximum layer superpixels integrating local expansion method.The experimental results show the effectiveness of the above two algorithms,both of which achieve a balance between accuracy and computational efficiency.When the accuracy is comparable to that of HLocal Exp-CM,the computational efficiency of the algorithm is greatly improved.(3)An unsupervised matching algorithm based depth edges optimization(SDCO)is optimization.The algorithm not only extracts and preserve the accurate depth edges efficiently,but also significantly improves the matching effect and disparity accuracy of object interior occlusions,textureless and weak texture areas.First,a new matching method is proposed,which fuses Census transform and the combination of color and gradient information.Then,in the optimization process of the algorithm,the images is segmented and optimized according to the color,gradient and spatial distance information,a depth superpixel structure is designed.According the feature of depth superpixel structure,a fronto-parallel disparity map and slanted plane disparity map optimization methods are proposed.Finally,a large number of experimental results show the effectiveness of the algorithm,which is significantly better than similar unsupervised algorithms.(4)A self-supervised binocular matching network SANet based on depth edges and structure optimization is proposed,which solves the problem that the deep learning method can obtain high accurate disparity map without Ground truth and only input images,as well as the poor cross-domain generalization.The SANet network fuses the unsupervised SDCO algorithm and a lightweight deep learning method to optimize and complete depth edges and object structures.SANet includes three optimize modules,i.e.,supervised learning,multi-scale reconstruction of left image,and multi-scale depth edges perception.According to the design of the network optimization modules,a multiscale loss function is proposed to train the network.Experiments show that SANet obtains high accurate disparities with various datasets,and exhibits good cross-domain generalization.