Study On Planar Scene Reconstruction Based On Multi-View Image

3D scene reconstruction is one of the typical tasks of machine vision. And it is also an important research in robot field. With applications of various robots in digital city and intelligent city for real-time monitoring, exploration, rescue, and even as a substitute,3D urban reconstruction based on the captured images by the vision system has caused widely attention in recent years. However, Accurate and realistic 3D reconstruction still faces many problems and challenges because such a scene is composed of complex and serious mutual occlusion structures, and the captured images contain many unstable illuminations, such as shadow, shading and highlight. Morverover, there are sparse and repeat texture, especially weak texture of windows on the planar regions of the common urban building.In this paper, we focus on the key problems of 3D reconstruction, including feature detection based on color image, discrete disparities optimization on plarar region, sparse point cloud, holes and bumps of planar reconstruction, and incomplete reconstruction of windows on building, to provide effective solutions to improve the quality and the completeness of 3D urban reconstruction. Its detailed research content and paper innovations are as follows:(1) Because of lost color information in image, gray-based detectors are sensitive to unstable illumination, such as shading, shadow and highlight. A color saturation invariant based on dichromatic reflection model is constructed, and synthesized with existing hue invariant to detect edge and corner features in color image. The detection method proposed here is more unaffected to the unstable illumination areas and achieve true target features.(2) Because the implicit assumption of a constant disparity value in support windows generally is invalid on slanted surfaces and boundary of two objects, where disparities are discontinuous and planar reconstruction resemble a staircase. Based on the principle of image smoothing, a disparity map optimization is proposed by using sparse gradient measurement under intensity-edge constraints. The model of the disparity optimization problem is then formulated as a constrained optimization objective function, which is finally solved via the half-quadratic splitting algorithm. Experiments on public test bench show that the proposed approach obtains accurate disparity maps with sub-pixel precision, and it solves the staircase problem of the disparity maps regarding slanted surfaces.(3) A piecewise multi-planar scene reconstruction is proposed without using depth map by combination multiple structures estimation based on 3D sparse points cloud with unsupervised 2D image segmentation. The proposed method first improves the J-linkage algorithm, with the stratified sampling, instead of the random sampling. We then fit the point cloud with planes by using the improved J-linkage algorithm, to obtain the multi-planar model of the scene. Finally, we extract and reconstruct the planar regions, with the multi-planar model, as well as an unsupervised segmentation algorithm. The proposed method not only can effectively overcome holes and jaggies problems, but also can models the complete planar regions.(4) Line features detected from images are matched precisely by using affine invariant descriptors. An initial candidate match is dispersed to a set of correspondences. As a result, the problem of inconsistent support region size is resolved because we need only construct descriptors of correspondence points instead of lines. In order to make the descriptor affine invariant, the dominant orientation and the scale of the descriptor are created according to the direction and the length of the line, and gradients of the discrete points set in the local neighborhood are calculated. To speed up line matching, epipolar constraint is used before constructing line descriptors, and the number of potential matches is limited. Then, line matching is preceded accurately by the nearest neighbor distance ratio approach. The experimental results show that the proposed descriptor has accurate line matching under the changes of affine, illumination, viewpoint, and partial occlusion. Furthermore, the result of line matching is applied to 3D line reconstruction, and it provides the foundation of windows reconstruction in building.(5) A method of window detection is proposed under edges and glass attribute constraints. Firstly, windows are basically located with edges in image. Next, color and texture features on the edge neighbor are extracted, and the dissimilarities between the two features are applied for windows recognition. Finally, windows are detected by using image segmentation algorithm. Experiments show that our proposed approach can efficiently detect windows in the case of unknowing the shape and structure of windows. Subsequently, the result of window detection using as 2D window localization in image and the result of lines reconstruction using as 3D line localization are applied to 3D window reconstruction. By this way, the incompleteness reconstruction of window area with highlight is improved.