Research On Stereo Matching Algorithm In 3D Reconstruction

Stereo vision is a branch of computer vision and has a wide range of applications in the fields of drones,autonomous driving,robot navigation,3D measurement and so on.Stereo matching is the most critical part of stereo vision.Due to the morbidity of stereo matching,different stereo matching algorithms are designed to adapt to different applications.Therefore,the stereo matching algorithm has been a hot research in recent years.This paper proposes a stereo matching method for 3D reconstruction.The matching accuracy is improved by continuously optimizing the results of stereo matching from three steps:matching cost calculation,matching cost aggregation,and disparity map optimization.The main works in this paper list as follow:(1)A matching cost calculation method based on multi-scale convolutional neural network is proposed.Energy function is very sensitive to surface reflection area,litter feature area and same feature area during the calculation of stereo matching cost.And the convolution kernel size is set unreasonably in common convolutional neural network.A multi-scale convolutional neural network is proposed to calculate stereo matching cost in this paper.By training the multi-scale convolutional neural network with the exponentially decay learning rate,the parameters of the neural network will be optimized to the best value.So that the stereo matching cost calculated by the multi-scale convolutional neural network is reliable.(2)Since the design energy function calculates the matching cost,it is very sensitive to surface reflection,weak texture,texture repetition,and the like.The use of neural networks to calculate the matching cost has stronger generalization ability than the traditional algorithm,but there are still some mismatching point and area since the morbidity of stereo matching.This paper designs an adaptive cross-window matching cost aggregation method to modify some mismatching points,and the details are kept well.This paper designs a semi-global method based on the dynamic programming method to modify mismatching area,and the tailing phenomenon of dynamic programming is overcame well.By combining the adaptive cross-window method with the semi-global method,the reliability of the matching cost matrix in the non-occlusion region is greatly improved.(3)Since the matching cost of the occlusion area and the edge area is not modified,the predicted disparity map is processed by post-processing.All the points in the disparity map are divided into four classes by left and right consistency checking,and different processing methods are given for each class.In addition,the disparity map is refined and converted into sub-pixel levels,which improves the accuracy of the disparity map.(4)Evaluate the algorithm of this paper.According to the evaluation standard of the Middlebury stereo vision platform,the root mean square error and error rate of the disparity map obtained by the algorithm are evaluated and compared with other algorithms on the platform.(5)Apply the stereo matching algorithm of this paper in 3D reconstruction.The image pair is corrected by the mature camera calibration algorithm at present,and the disparity map is obtained by the stereo matching algorithm this paper proposed.According to the disparity map,the three-dimensional coordinates of the points in the scene are calculated,and the three-dimensional point cloud can be drawn.The feasibility of the algorithm is verified by compared the actual size of the object and the size of the 3D point cloud of the object.(6)Package and GUI design for stereo matching algorithm.According to the hardware of the computer,the software environment is built.All the parts of the stereo matching algorithm,evaluation of stereo matching algorithm and calculation of three-dimensional coordinates are included by the stereo matching system,so as to facilitate the application of the algorithm.