Research On Stereo Matching Based On Semantic Information

With the continuous development of technologies such as autonomous driving,robotics,and virtual reality,3D vision technology has received more and more attention in the field of computer vision.Image-based 3D vision is an important research direction of scene understanding in computer vision.Its main goal is to infer the 3D structure and content of real-world objects and scenes from RGB images.Semantic segmentation technology based on deep learning has better scene content interpretation and understanding capabilities.Stereo matching technology based on deep learning can recover the 3D information of the scene from a pair of stereo images,compared with traditional algorithms,it has the advantages such as higher accuracy and faster speed.With the further development of technical requirements such as smart sensors and scene understanding,the research of stereo matching based on semantic information is of great significance in scene understanding.This paper studies and designs a stereo matching algorithm based on semantic information.Based on deep learning technology,it mainly conducts in-depth research and discussion on the use of segmentation semantic information to guide disparity refinement and the modular design of network structure.The main research contents of this article are summarized as follows:(1)In view of the insufficient flexibility of the existing deep learning-based stereo matching network and the inability to balance the speed and accuracy indicators,this paper proposes an idea of modular design of the network structure.Based on a lightweight feature extraction module,a Semantic Fusion Module and a Disparity Fusion Module are designed,which are used to estimate the semantic segmentation map and the disparity estimation map,respectively.This kind of multi-vision task sharing network backbone design can improve the utilization efficiency of the module while reducing the amount of network parameters.Experimental results show that based on the lightweight network structure,adding these two modules can get better semantic segmentation and disparity estimation results.(2)Aiming at the problem of disparity optimization using semantic information in the existing deep learning methods for estimating stereo disparity,this paper proposes a method for disparity optimization based on semantic information in outdoor autonomous driving scenes,and designs and implements a Context Adjustment Module.This module selects the semantic segmentation map of the left-eye view to guide the optimization of the disparity map for stereo matching,and performs effective feature fusion on the segmentation map and the disparity map.Experimental results show that based on the original network structure,this module can effectively improve the accuracy of disparity estimation in some areas of the image.(3)Aiming at the problem of insufficient flexibility of the stereo matching network in the existing multi-task network structure,this paper draws on the modular expansion idea in the software system to realize the modular expansion and use of the stereo matching network structure.In response to different application requirements,this paper has designed four solutions.Under different accuracy and speed requirements,different schemes can be used,and suitable network modules can be selected for training and use to meet different application scenarios.