Research On Three-dimensional Structured Scene Reconstruction From Multiple Images

Image based3D reconstruction is a research hotspot in computer vision. It is a kind of process to imitate human visual system. Its goal is to convert two-dimensional images into three-dimensional scenes world scenarios for all kind of use in various fields, such as computer-aided design, intelligent community, gaming entertainment, etc., which is very wide.Although researches on this topic have gained a lot of achievement, becaouse of noises in the image, there is still a lot of uncertainty existing in reconstruction results, so there are many problems to be solved."Structured scene" refers to scenes which are composed of three-dimensional point sets meeting some geometric rules (for example coplanar, parallel, vertical), such as buildings, bridges, stone blocks, piles and interior furnitures. This scenario is characterized by containing a lot of geometric primitives of mutually perpendicular planes. These primitive relationship such as vertical, parallel, etc., can be directly applied to camera self calibration. At same time, because there is a large ambiguity in automatic meshing process, automatic method can not always produce models which meet application request. Often, artificial means is needed to assist the process.This paper is aimed at the research of key technologies in automated and semi-automated scene reconstruction from multiple images. Main research routine is based on the multi-view geometry theory in computer vision, and combining with the pattern recognition clustering method, text retrieval fast similarity calculation method, visual perception method and geometric reasoning method. The start point is to apply the methods and theories in related fields to expand the traditional3D reconstruction method, and verify the feasibility of this expansion through experiments, to make a meaningful exploratory work for subsequent research. The main research contents and innovations include:1. Camera self-calibration formula is deduced for the pinhole camera model based on vanishing points. Camera self-calibration can be completed with three orthogonal vanishing points. For the problem that existing Manhattan direction vanishing point detection algorithms are generally less efficient, not suited for real-time calculation, this paper proposes a vanishing points detection algorithm MMVD based on multi-model estimation and entirely in image space. The method uses the detected straight line segments to complete initial line segment clustering and multi vanishing points estimation through a multi-model consistency estimation. And then through the EM algorithm to optimize the results, all possible vanishing points can be estimated. Among the detectd multiple vanishing points, according to the nature of the vanishing point, we can quickly find vanishing points with the Manhattan direction through some rules in different situations. At last, experiment verifys the efficiency and accuracy of vanishing points detection algorithm.2. Accroding to the problem that traditional iterative algorithms in image matching are very time-consuming and results always drift in iterative process, this paper proposes a fast iterative reconstruction algorithm FIRA based on agglomerate hierarchical tree. The algorithm is improved on Bundler algorithms which propesed by Snavely. In image similarity calculation stage, the algorithm refers to LSH mode in a massive text retrieval field, and uses minhash algorithm to complete the image similarity quick calculation. Using the similarity calculation results, agglomerative hierarchical tree structure is built to guide the iterative reconstruction process, making the reconstruction process controllable, efficient and can be executed in parallel. The main advantage of this algorithm is that the complexity of the algorithm is independent of the size the images, which greatly reduces the image matching time. And in iterative process, as agglomerative hierarchical tree can split big problem into more minor problems, drift problem can be resolved. FIRA algorithm and Bundler algorithm processes are analyzed and compared through experiments which proves the FIRA algorithm accurat and efficient.3. For the reconstructed geometric model complexity is too high and can not be directly applicated in real-time rendering system, in this paper, based on Garland’s QEM algorithm, EQEM is proposed to quickly generate LOD models. This simplified algorithm takes into account the geometry and texture characteristics. In the geometric construction of the operator, curvature is considered and a custom formula is proposed to reduce the curvature computational complexity. When building the visual measurement descriptor, edges texture attributes and edge texture differences are used to give each edge a visual importance wegith. Experiments show that the operator can guide each iteration step to ensure a more rational simplification. Some violent simplified model can maintain an acceptable appearance and shape characteristics. Meanwhile experiments also show that the operator can make the simplification process with good speed.4. In the paper, previous three algorithms, such as vanishing points detection algorithm MMVD, fast iterative reconstruction algorithm FIRA and simplification algorithm EQEM are integrated into a multiple images based three-dimensional scene reconstruction system, which is designed and implemented for the three-dimensional structure scene. According to problem that the meshing results always can not meet the applicaiton request, a semi-automatic interactive reconstruction method is proposed. When the camera has been calibrated, this method can take advantage of sequential method for three-dimensional geometry reasoning, complete geometric modeling through the extruding tools. When the geometric models have been aquired, texture mapping can be automatically completed with the use of camera’s internal and external parameters, without any human interaction. This method can quickly meet the application needs to generate an orthogonal structural model and automatically texturing. Compared with the traditional modeling process, it is more efficient, and more accurate in geometric modeling.