Research On Dense Point Cloud Reconstruction From Multi-scale And Multi-view Images

With the development of the Internet, the image database on the Internet becomes more and more abundant. More and more researchers start to reconstruct the real scene model using the Internet images. This makes image-based modeling more meaninful. Since the images on the Internet usually come from different cameras, they usually have different noise levels and occlusions. This leads to the different appearance of the photos, even they are captured at the same scene. Although the image resource on the Internet brings some conveniences, it brings more challenges for reconstruction. This paper focuses on the key techniques involved in3D reconstruction from Internet images, and several novel and practically useful algorithms are proposed as follows:(1) A novel local invariant descriptor HRCRD is proposed by combining the intensity and color information. This descriptor is built based on two sub-descriptors:Haar wavelet response sub-descriptor, and color ratio invariant sub-descriptor. The color ratio invariant model is invariant to the changes of viewing direction, highlights, illumination direction, illumination intensity, and illumination color. This descriptor not only improves the describing speed of most existing descriptors, but also improves the discriminative power and robustness.(2) A new matching cost function is proposed by weighting the traditional function using the color component, direction component, and the distance component. This greatly reduces the error matching. It is further integrated with a proposed affine transformation based dense matching function to improve the matching accuracy at the sub-pixel level.(3) In order to eliminate the negative impact of the variant scale and baseline of the internet images, a neighboring view selection strategy is proposed to quickly and accurately track matching points in multi-views. A two layer iteration optimization algorithm is proposed to resolve the problems that the optimization process in the camera calibration cost high, or even fail due to the large amount of input images and3D quasi-dense points. In the inner layer, local photometric consistency and a global objective function are used to optimize3D quasi-dense points and camera parameters respectively, and the two processes switch iteratively. In the outer layer, the outliers are discarded by reprojection error in order to reduce the negative impact of the outliers. The proposed algorithms are tested with several image sets. The experimental results demonstrate that our algorithm performs better than SBA algorithm. In addition, our algorithm has more superiority when the number of images is small.(4) In order to deal with the internet images that have the characteristics of large amount, large scale invariant, and large resolution invariant, an input image grouping and view selection algorithm is proposed. It has three level, the scene level image pre-grouping, image level view selection, and point level view selection. For the different stages of the reconstruction, the images can be arranged more effectively. The scene level image pre-grouping algorithm employs the global GIST features to roughly group the image, and eliminate the unnecessary groups. Then the local HRCRD features and epipolar geometry are employed to refine the groups. The images with low correlations to other images in the group will be discarded, and the remaining images will be further grouped according to their scales and views.On the basis of the feasibility and effectiveness of above methods and algorithms, we develop a multi-view and multi-scale3D dense point cloud reconstruction system which integrating all the algorithms proposed in this paper. Further experiments demonstrate that this system not only can reconstruct the point cloud for the outdoor multi-scale scenes, but also be applicable for the single scale scenes.