Research On 3D Reconstruction System Of Binocular Stereo Vision Based On Convolutional Neural Networks

3D reconstruction is a widely used technology in various fields,such as intelligent robots,unmanned driving,building modeling,and industrial inspection.Its purpose is to restore the three-dimensional structural information of an object’s surface based on the image data of the target object in a given scene.Among the methods used to achieve 3D reconstruction,stereo vision is one way that utilizes two identical cameras to capture two images of the same object surface from different angles,and then uses the principle of similar triangles and the disparity obtained through stereo matching to reconstruct the object’s depth information.This paper mainly studies the methods and techniques of 3D reconstruction based on binocular stereovision,including hardware system construction,stereo matching,and 3D reconstruction technologies,among which stereo matching is a key technology in binocular 3D reconstruction systems.The main work of this paper is as follows:(1)A binocular stereovision 3D reconstruction system is constructed,using two identical monocular cameras to build a binocular vision system.Based on secondary development of the camera SDK,we achieved time synchronization of the cameras.Then,The binocular camera system is calibrated using Zhang’s calibration method in opencv library to obtain camera intrinsic and extrinsic parameters and perform rectification of epipolar lines.Finally,the user interface is designed for the 3D reconstruction system using Python programming language to visualize the 3D reconstruction results.(2)In order to enhance the robustness of stereo matching algorithms and address the issue of poor adaptability of traditional algorithms in complex environments,this paper proposes an adaptive region-based stereo matching algorithm.In the cost calculation stage,the first step is to distinguish between strong textured regions,weak textured regions,and untextured regions.For strong textured regions,the absolute intensity difference truncated by pixel values is used as the cost function.For weak textured and untextured regions,the cost calculation is performed by combining the absolute intensity difference truncated by pixel values with image gradients.Then,an improved Rank Transform parameter is introduced to solve the problem of decreased matching accuracy caused by changes in the center pixel value due to exposure and other factors.In the cost aggregation stage,a minimum spanning tree algorithm based on multi-feature fusion is used to combine color intracorrelation information with three-channel color information between leaf nodes,which compensates for the instability of using color components as a single component in the original edge weight.(3)To address the complex problem of feature function selection in cost calculation for traditional algorithms,this paper studies a stereo matching method based on convolutional neural network(CNN).Firstly,a spatial pyramid module is used to fuse dilated convolutions and obtain multi-scale feature information in the feature extraction stage,enhancing the feature extraction ability in pathological regions.Then,the cascaded matching cost and the disparity-dependent difference-based matching cost are fused to form the final matching cost,ensuring rich similarity information is included in the cost aggregation.Finally,local contextual information from multiple scales is connected through a 3D encoder-decoder network structure.In order to verify the feasibility of the system constructed in this paper and the effectiveness of the proposed algorithms,comparative experiments and 3D reconstruction experiments were conducted for the two proposed methods.The experimental results have verified that the stereo matching algorithm based on adaptive regions performs well in weak texture regions and noise resistance.The average mismatch rate obtained on four stereo image pairs in the Middlebury 2006 dataset is 10.17%,and the reconstructed water cup error reaches the millimeter level.The CNN-based stereo matching algorithm achieved a three-pixel error of 2.88% on the KITTI 2015 test set,and the reconstructed cup data obtained from the method shows an improvement in accuracy of approximately 5mm compared to the adaptive region-based stereo matching method.