Binocular Vision Based 3D Reconstruction Key Technologhies Research

With the rapid development of China's national economy and the improvement of people's quality of life,the state has paid more and more attention to a series of intelligent technologies such as smart manufacturing,smart cities and smart parks.The binocular stereo vision that reflects artificial intelligence is becoming more obvious,in the fields such as intelligent production,traffic monitoring,machine navigation,aerospace,medical modeling,visual simulation,cultural relics restoration,non-contact high-precision measurement,etc.,it provides technologies including virtual visual simulation,visual recognition and positioning,which can effectively improve production efficiency and product quality,reduce operating costs and resource energy consumption.3D reconstruction based on binocular vision is an important part of graphic image processing and machine vision.Because the single image taken by the monocular camera is two-dimensional and there is no third-dimensional information,the binocular camera is used to simulate the human visual system,two two-dimensional images are taken,and the parallax of the point pair is achieved according to the two images using similar triangles to calculate the distance from the surface of the object to the line connecting the two optical centers,and the three-dimensional coordinate point cloud on the surface of the object.The three-dimensional reconstruction is to reconstruct the surface of the target object according to the three-dimensional coordinates of the point cloud on the surface of the object.3D reconstruction can restore 3D information of the scene,and assist the robot to complete specific tasks such as target recognition,positioning,measurement,navigate,crawling and tracking.Based on the three-dimensional reconstruction of binocular vision,it is more economical and practical to imitate the two eyes of human beings while observing the scene.The purpose of this project is to calculate the parallax through binocular visual image matching,calculate the surface point cloud of the object according to parallax and establish a three-dimensional model of the target object,and assist the robot numerical control system to identify,locate and grab the workpiece and other objects.The significance is to improve production's efficiency and quality,and to make industrial production and manufacturing more intelligent.This project relies on the national project: the national key research and development program of China ?research and application of key technologies for real-time fault diagnosis of intelligent production line based on industrial internet of things?(YS2017YFGH001945).The main contribution of the paper is to propose the following methods:1.Circumference binary feature extraction algorithmExisting gradient feature and binary feature extraction methods have the problems of large computational complexity and low inlier rate.Aiming at these problems,circumferential binary feature extraction algorithm for local regions of image key points is proposed.When extracting features,the Gaussian pyramid is used to simulate the human eyelet imaging model: the near target image is large,the distant target image is small,the near target image is clear,and the distant target image is blurred,ensuring the illumination,scale and blur invariance of the feature;The FAST operator detects the key points;uses the image feature point neighborhood gray center of gravity method to calculate the key point feature direction to ensure the rotation invariance of the feature;proposes the mirror invariance law and the circumferetial binary feature extraction algorithm to improve the mirror invariance.The extracted circumferential binary features are highly adaptable and its comparison is fast.2.Bitmap local sensitive hash for searching matching binary featuresAiming at the problem that the existing matching binary feature search algorithm has low efficiency and few inliers,a fast calculation bitmap algorithm and a bitmap local sensitive hash algorithm are proposed to search the matching binary features of the image.First,calculate the keyword of the feature bit vector of the left image;then,use the fast calculation bitmap algorithm to calculate the bitmap of the bit vector,extract the key according to the mask,and construct the local sensitivity hash table together with the identification of the binary feature as a map.The table is simultaneously stored in the bit set;finally,according to the binary feature corresponding keyword extracted from the right image,the bitmap is used to determine whether the feature ID exists in the hash table and optimize the inquire of the matching binary in the query hash table,which can improve the search efficiency and quality of matching binary features.Experiments show that the bitmap local sensitive hash algorithm improves the efficiency of binary feature neighbor search and increases the number of the inliers.3.Circumference binary feature extraction and search methodBinary features have the advantages of fast calculation,effective matching and easy storage in image matching and positioning.The existing binary feature extraction algorithm has poor image invariance,and the matching binary feature search algorithm has low inlier rate.Aiming at the two problems,combined with the circumferential binary feature extraction algorithm and the bitmap local sensitive hash algorithm,a circumference binary feature extraction and matching search method is proposed.4.Internal and external similarity aggregation stereo matching algorithmThe problem of the high error rate of the non-occlusion region and low efficiency of the parallax map extracted by current stereo matching algorithm: First,the internal similarity of the reference color image is defined;then,in the color and sub-pixel space of the reference image and the target image,the external similarity is defined between them;then,the internal and external similarity aggregation method(IESA)is proposed to aggregate the similarity of matching points in the left and right images,and the disparity map is calculated using the winner-take-all algorithm.Finally,the box plot filtering(BPF)is proposed to purify and smooth disparity image,and the internal and external similarity aggregation stereo matching algorithm in eight directions is given.The experimental results show that IESA algorithm has lower non-occlusion region error rate of the disparity map and high efficiency,when extracting disparity image.5.Three-dimensional reconstruction method of object surface based on disparity imageAccording to the two images acquired by the binocular camera,the parallax image is extracted,and the image segmentation algorithm based on the parallax histogram is proposed.The target region is segmented from the parallax image,and the target region image is divided into 5*5 squares,and each square is divided into two triangles using the corner points of the square.Calculate the three-dimensional coordinates of the corner points of each block of the target object or the image of the scene,according to the similar triangle,obtain the point cloud of the target object or the scene surface,and connect the three-dimensional points corresponding to the corner points of each block of the target area in a sequence to form a triangle in three-dimensional coordination,and divides the target area into a triangular mesh to model the target object.The method has high precision for the three-dimensional reconstruction of the target object and a small amount of calculation.6.Method of estimating the pose of the target objectAfter establishing a three-dimensional model of the surface of the target object,the target region of the reference image of the disparity map is segmented,and the reference image target region is used as a matching template to provide a relative shape of the target itself;the key point is detected on the template image and the feature is extracted,and the feature is to be gestured from the estimated image,uses the matching feature search algorithm to query the matching features of the two images,finds the feature point coordinates according to the feature,and obtains the three-dimensional coordinates of the target region point according to the template image matching point pixel coordinates and the template parallax image,so that the mapping of 3D to 2D coordinates is obtained,and the PNP algorithm is used to perform pose estimation on the target object.