3D Reconstruction Of Texture-Less Object Using Incremental Structure From Motion

3D reconstruction based on image sequence is an important research filed in computer vision.The technique has been applied in many areas,such as 3D scanning,object detection and recognition,augmented reality,etc.Structure from Motion(SFM)is the process of estimating 3D structure from images with local motion signals,and it can obtain camera parameters and sparse point cloud of a scene.Using Multi-View Stereo(MVS)algorithm based on SFM's outputs,we can estimate image depth maps and obtain dense point cloud.Since the method usually has an accuracy problem when dealing with texture-less object,the thesis studied the feature matching and image depth estimation part of the reconstruction process.In order to obtain more accurate reconstruction results,improved algorithms were presented based on the characteristic of texture-less object.The main work is as follows:1.The thesis presented a multi-scale binary descriptor based on ORB.Firstly,FAST algorithm was applied to find feature locations.The features are invariant to image scale by building a Gaussian Pyramid,which was also used to obtain multi-scale information later.The feature descriptor used binary tests between image pixels and gradients in image patches with different scales.All descriptors at different scales were aggregated to form into one final binary descriptor.The improved method achieved good performance on the Oxford dataset,it was more accurate and robust to several image transformations,including viewpoint changes,illumination changes,rotation and scale changes,image blur,etc.Therefore,the descriptor is helpful to improve reconstruction quality.2.The thesis used a fully convolutional network to predict image depth maps,which were employed to improve dense point cloud results.When using Multi-View Stereo algorithm for texture-less objects,the acquired depth maps were not satisfied and objects got information loss on their surfaces of point cloud.The thesis chose learning algorithm to do depth prediction.A fully convolutional network based on ResNet-50 was trained end-to-end to estimate depth map given a single RGB image,and its output resolution is the same as input.The experimental results showed that the method can reduce depth estimation error effectively.3.Texture-less object was completely reconstructed from image sequence.Firstly,the thesis used incremental structure from motion to do sparse modeling.The proposed binary descriptor was used for feature detection and matching.After the process of image registration and triangulation,3D structure and sparse point cloud of the scene was acquired.Then a fully convolutional network was used for image depth estimation,and the results were merged with MVS algorithm's outputs to improve accuracy.Finally,fusion of the improved depth results fulfilled the missing part of point cloud and produced a complete dense point cloud of the object.