A Multi-primitive And Multi-view Image Matching Method Based On Self-adaptive Triangle Constraint

Along with the advancement in acquiring airborne or space-borne images, building models in cities and large scale city models can be produced such as those provided by Google Earth and Virtual Earth. However, three dimensional (3D) models with detailed geometry and textural imformation are still not available in most of the existing systems. This requires more close-range data, such as ground images, terrestrial laser scanning data, or oblique images from helicopter. Detailed3D modeling is a hot topic nowadays either for image-based modeling or LiDAR point cloud-based medeling. Image based modeling remains the most economical, flexible, convenient, and popular method for3D building modeling. One key issue in image-based modeling is image matching, which uses the computer to simulate human eyes to automatically determine corresponding primitives by stereo observing. Image matching is always the common academic frontiers and difficult problem in the photogrammetry and computer vision communities.Most existing image matching methods are based on an assumption of smooth surface continuity, which is difficult to adapt to the images acquiered in urban areas with dense buildings, high mountain areas with surface discontinuities, and other complex scene. Especially they are not able to obtain reliable matching results when encountering with poor textural considitions such as textureless, repetitive patterns and surface discontinuity. Critical edge features in buildings sometimes can not be matched successfully; the derived DSMs (Digital Surface Models) can not express the actual scene accurately, which limits the application of image-based modeling. Aimming at reliable image matching, this research introduces the edges into matching constraint (i.e., the triangulations) to constructing edge-constrained triangulations to support reliable image matching. This research also investigates other key aspects in reliable image matching including robust seed point determination on complex scene images, distinctive point and edge feature extraction, adaptive similarity measurement, and effective matching constraints. The main work and innovations of the research includes:1) Self-adaptive triangulation based multi-primitives image matching model: Aimming at the image matching on difficult textural conditions such as textureless, repeative patterns, and surface discontinuity, this research presents a hierachical multi-primitives image matching method. Firstly, a scale and orientation invariant feature extraction operator is employed to extracting features for image orientation and obtainning epipolar geometry, the obtained robust corresponding points after image orientation are used as seed points to contruct Delaunay TIN to constrain the subsequent image matching. Secondly, a feature-to-feature matching and a feature-to-area matching process are carried out to obtain more corresponding points. For reliable image matching, a trianguation-based disparity contraint and a triangulation-based gradient orientation constraint are developed to reduce the search range of finding corresponding points. Thirdly, a Shiftable Self-adaptive Normalized Cross-Correlation is developed to measure the similarity between the corresponding points, even in the image area with large perspective changes or surface discontinuity problems. The developed method is able to obtain dense and reliable corresponding points which is ideal for the subsquent3D modeling and re-construction.2) Edge (line) matching based on triangle constraint:Considering the rich line information in the images from human made object such as building facade, this research presents a simple but effective line extracting method and a line matching method based on triangle constraint. The combination of triangulation constraint, line attribution constraint, and line similarity constraint are used to reduce the search range of line matching. For reliable line similarity measurement, a Shiftable Self-adaptive Line Cross Correlation (SSLCC) is proposed. The experiments show that the proposed SSLCC performs better than the traditional line cross correlation methods. A line matching method based on RANSAC approach is proposed to process those lines parallel (or closely parallel) to the epipolar lines and to match those lines that no corresponding lines extracted on the searching image. Experiments using actual image data sets proved the good performace of the proposed method.3) Multi-primitive matching propagation based on the Constrained Delaunay TIN: Image matching in poor textural conditions (such as surface discontinuity, textureless, and repetive textural patterns) is very difficult. This research developed an integrated line and point matching propagation method to improve the effectiveness of image matching in poor textural conditions. Constrained Delaunay TIN is employed to organize the matched line and point together to support the subsequent matching. During the matching propagation, the matched lines are treated as feature lines and inserted into the Delaunay TIN dynamically. They can segment the images more effectively to support reliable image matching, especially when encountering with surface discontinuity problems. The matched lines are used to constrain the subsequent point matching, while inversely the matched points are used to constrain the subsequent line matching.4) Multi-view image mtaching based on self-adaptive triangulation constraint: When the overlap area among images becomes large, redundant information from multiple images is very useful to improve the matching reliability. Therefore, this paper presents a method to use the third image to improve the image matching reliability. For reliable propagation path, image segment information is used. The proposed method was compared to the state of the art image matching method from Computer Vision community. Experiments using actual multiple images proves the good performance of the proposed method.The theory and algorithm of the presented methods are implemented using Visual C++6.0. Experimental analysis revealed that the developed method is capable of generating dense and accurate point and edge matches preserving the actual textural features which are ideal for the subsequent3D modeling and re-construction.