Image Registration And Stitching With Geometric Constraints

Images with wide field of view and high resolution are the basis for scene understanding.Due to factors such as sensor size,field of view,and shooting conditions,there is a contradiction between wide field of view and high resolution.Thus,it is difficult to collect high-resolution images with wide format and large field of view.An effective solution may be image stitching,which is to continuously acquire images about the observation area under a certain overlap rate,and then synthesize it into a mosaic image with a wider field of view,thereby it can expand the field of view of the image.Image stitching is widely applied in photogrammetry and remote sensing,virtual reality,security surveillance and military reconnaissance.It is a research hotspot of photogrammetry,image processing and computer vision.The traditional image stitching method is generally based on the global transformation model.The image acquisition requirements are strict,that is,the camera is concentric when shooting(such as rotating shooting or shooting at the same viewpoint)or the shooting scene approximates the plane scene,which is often difficult to meet in actual data collection.The images acquired under the actual free viewpoint have parallax between the images.The depth of the scene changes greatly and cannot be approximated as a planar scene.Therefore,the global transformation model based on the plane hypothesis cannot correctly model the transformation relationship between images,resulting in the mosaic image with large alignment errors.In order to solve these problems,the researchers proposed to stitch parallax images based on based on the local transformation model,which divides the image into several local regions and estimates the best adapted transformation for the local regions to achieve accurate image stitching.However,the existing local stitching methods generally rely on sufficient and reliable matching point features and projection transformation models,which easily lead to inaccurate modeling of the relationship between images and projective distortions.To handle these problems of parallax image stitching,this paper combines a variety of geometric structure information to carry out the research on local registration and stitching of parallax images based on geometric structure information constraint,which improves the accuracy of parallax image registration and stitching.The research contents are as follows:(1)For the problem of insufficient matching features and inaccurate transformation models in parallax image mosaic,the image local registration method based on multifeature joint constraint is studied.The commonly used local transformation model relies on the registration of feature points.For low-texture regions or images,it is often difficult to provide sufficient and reliable matching points,which leads to inaccurate estimation of the transformation model.Moreover,the existing local transformation model is difficult to effectively characterize the transformation of parallax image,resulting in a large image registration error.To this end,this paper fully exploits the features of points,lines and junction structures in the scene and studies the local transformation model estimation of line feature constraints,so as to improve the accuracy of image registration.The paper constructs the structural constraint model based on line features,and combines with content-preserving warping model,to optimize the registration and maintain the image geometry simultaneously.The paper studies the joint registration method based on local transformation and mesh optimization model to improve the performance of parallax image registration.(2)It is difficult to accurately describe the mapping relationship between multiplane scene images,which leads to the problem of multi-plane image mosaic error.The multi-planar image mosaic based on smoothly planar homography model is studied.Multi-plane scenes have different planar structures.The local region partitioning in the local transform model cannot accurately acquire different planar regions,which leads to problems such as stitching misalignment in different regions between images.Fully considering the multi-plane characteristics of complex scenes,the multi-plane region detection method based on random clustering model is studied to extract the planar regions in the scene.The smoothly planar homography model for different planes is studied to realize the registration of complex scenes.The optimal stitching line search combined with registration error is conducted based on graph cut optimization to achieve seamless stitching of multi-plane scenes.(3)To handle the projective distortion(such as shape distortion and perspective distortion)existing in the stitching methods,the research on image distortion correction based on similar prior constraint is carried out.Since the existing image stitching generally adopts a projection transformation(e.g.homography transformation),its collinearity tends to cause the non-overlapping regions of the object to be non-uniformly stretched,thereby causing perspective distortion.The closer to the image boundary region,the greater the distortion.Considering the limitations of transformation model,the study combines similar transformations to constrain the distortion of projection transformation.The global similarity transform estimation based on random sampling consistency is studied.Based on the analysis of homography transformation error,a spatial distance weighting model based on distortion direction is constructed.The similar transformation and homography transformation are linearly combined to realize the smooth transition from perspective to similarity across the image.Thus the combined transformation eliminates the shape and perspective distortion of the non-overlapping regions and improves the quality of image stitching.In summary,this paper discusses and analyzes the related theories and methods of parallax image stitching technology,and studies the key issues about image registration and distortion removal,and proposes a series of parallax image stitching methods to improve the performance of image stitching.The paper also provides a theoretical basis for the further research and application of parallax image stitching,and has important academic and applied significance.