Research On 3D Scene Reconstruction With Multi-view Based On The Patches

Image-based 3D reconstruction technology is a basic research area of computer vision,using a camera(or cameras) to shoot objects or scenes, according to the two-dimensional images obtained from the camera and camera parameters to estimate the coordinates of three-dimensional space points. It is widely researched because of its low cost, convenient operation, no professional technical personnel, and can realize the reconstruction of large scale complex. At present, the research results have been widely used in the fields of aerospace,medical, entertainment, the construction of digital cities, ancient cultural relics repair. This paper is about the research work of this technology. The main research contents include image feature points extraction and matching, camera self-calibration, robust parameter estimation of fundamental matrix, and reconstruction of dense 3D point cloud. The main innovations of this paper are as follows:(1) A novel Structure from Motion method is proposed to reconstruct sparse 3Dpoints.Firstly, detecting and matching feature points of the images by SIFT algorithm. Then using RANSAC algorithm to obtain the estimation of the fundamental matrix robust parameters, and initialize the camera internal parameters through embedded image EXIF information, according to the relationship of fundamental matrix and essential matrix to get essential matrix and the singular value decomposition to get external parameters of the camera. Then the coordinates of3 D points are obtained by triangulation. Finally, the initial reconstruction results are adjusted by minimizing the back projection error, and a complete 3D point cloud model is reconstructed.(2) A cluster-based PMVS algorithm is proposed. In view of the high complexity of time and space when using PMVS algorithm to reconstruct large image sets, in this paper, a new method of image selection and clustering is combined with the robust model fusion method, and the PMVS algorithm based on clustering is proposed. First of all, the image set with a big quantity, disorderly sequence and different quality is divided into a number of multiple compact size, appropriate size and small overlap. Then PMVS algorithm is used to reconstruct the image clusters, and the reconstruction process is run in parallel, so it can improve the running efficiency of the algorithm and reduce the memory consumption. Finally, the fusion algorithm of two kinds of filters r is used to fuse the reconstruction results of all the clusters into a full 3D point cloud model. The problem that the original PMVS algorithm is too large memory consumption and cannot reconstruct the large images is solved.(3) The PMVS algorithm with the space geometric constraint is put forward. The original PMVS algorithm using only current patches' information to initialize parameters of a new patch expansion process, which may cause a large deviation between the normal of new patches and the real normal. This problem becomes more serious under certain image capturing configuration such as downward-shooting or upward-shooting, a frequent practice in large scene reconstruction.This paper puts forward to add the space geometric constraint to PMVS algorithm under the consideration of adjacent patches' influence on the diffusion, the normal and quasi visual imageset, which improves the reconstruction accuracy and reconstructed surface is smoother.Furthermore,two kinds of PMVS algorithm are successfully combined in this paper. Firstly, we decompose the collection into different a set of clusters. Then a PMVS algorithm with geometric constraint is used to reconstruct each cluster through. Finally we merge all resulting reconstructions. The experimental results show that the proposed method not only can realize the3 D reconstruction of large image sets, but also improves the reconstruction accuracy.