Real-time Stereo Matching Algorithm Based On Deep Learning And Its Application

Stereo matching is an important branch of computer vision task,whose task is to calculate the disparity of corresponding pixels in binocular images.Stereo matching algorithm is widely used in robot,automatic driving and other fields.With the integration of convolutional neural network,the accuracy of stereo matching has been significantly improved,but the algorithm also has the disadvantage of slow running speed.To solve this problem,a real-time stereo matching algorithm is designed and applied to stereo vision system.Firstly,this thesis introduces ghost convolution and group convolution to accelerate the algorithm in feature extraction and disparity refinement.At the same time,combined the traditional algorithm,this thesis uses the discrete and reliable disparity to replace the whole continuous disparity search space.Then this thesis designs a lightweight cost volume construction method,which reduces the cost volume and accelerates the speed of disparity estimation.The above improvements ensure the real-time performance of our algorithm.In addition,this thesis designs a double-layer cascaded stereo matching network structure.The inner layer network calculates the disparity confidence range,making the disparity search range of each pixel accurate;the outer layer network estimates the disparity on the disparity confidence range.The double-layer structure improves the efficiency of disparity estimation and makes up for the loss of precision caused by small cost volume.Finally,this thesis designs and implements a stereo vision system.This system realizes the functions of camera calibration,real-time stereo matching and 3D reconstruction.This system calibrates the binocular camera by photographing the chessboard and detecting the corner of the chessboard image.Then the calibration results are used for stereo correction,so that the camera can provide standard input image for the following stereo matching algorithm.In stereo matching module,this system uses census features to assist stereo matching algorithm,reducing the dependence of algorithm on training data.Finally,this system uses the disparity calculated by stereo matching algorithm and the camera internal and external parameters calculated by calibration algorithm to reconstruct collected images.