Reasearch On Multi-scale Visible Light And Infrared Image Registration And Fusion

Thermal infrared imaging and visible-light imaging are two of the most important detection methods,they had played an important role in military and civilian applications.Thermal infrared imaging obtains temperature and radiation information of targets,while visible-light imaging reflects texture and contour information of targets.Due to this difference in imaging mechanisms and application scenarios,their information can complement each other naturally.By merging infrared and visible information into a single image,it is possible to synthesize their unique information to achieve complementary outcome while reducing redundant information,thus this technique is of great importance.Most of the current fusion technologies require images of different source to have the same resolution and assumed that different sources have already achieved high-precision registration on a pixel-by-pixel basis.However,these assumptions are unrealistic.Limited by the development of devices and applications,the resolution of the visible-light camera is much higher than that of the infrared camera.Traditionally the fusion is performed by either down-sampling visible light image or up-sampling infrared image,which will inevitably lead to the loss of visible light image texture or blurring of infrared image.Therefore,it is important to study the fast,accurate,robust registration fusion algorithm for multi-scale visible light and infrared images.In this thesis,the multi-scale visible and infrared image registration fusion problem is deeply studied,new methods and algorithms for multi-scale visible and infrared image registration and fusion are proposed.The main contents and contributions of this thesis are summarized as follows:1.To meet the requirements of multi-scale visible light and infrared image registration,a SI-PIIFD(Scale-Invariant Partial Intensity Invariant Feature Descriptor)and LPM(Locality Preserving Matching)feature point mismatch removal method is proposed.Via this method,high-precision matching of multi-source visible/infrared image features is achieved.Firstly,Harris corner detection algorithm is used to extract control points from visible light and infrared images.In the constructed scale space,a multi-scale descriptor sequence of feature points is constructed by using partial intensity invariant features to achieve the scale invariance of feature points.A matching is performed between the multi-scale descriptor sets of the images to be registered,and then based on descriptor similarity,the initial matching relationship of the of the feature points is established on the optimal scale.By using the constraint of feature point neighborhood structure stability,the mismatches of the feature points are removed,and the proportion of the effective match is therefore improved.Experimental results showed that the proposed method can accurately and stably extract the features to be registered,has high effective matching ratio,as well it is robust in multi-scale changing scenes.2.To address the problem of high mismatch ratio of multi-scale visible and infrared image features,as well as the difficulty to construct correct registration relationship,an image space transform estimation method based on Gaussian mixed model is proposed,accurate spatial transformation relationship estimates between visible light and infrared image is realized.The multi-mode image registration process is modeled as a posterior probability maximization problem of effectively matched feature points.The Gaussian mixed model is used to characterize the joint probability density of registration points and outliers in the registration process.Based on the Bayesian framework,spatial transformation coefficients of the affine transformation are solved using the Expectation-Maximization algorithm.Experimental results showed that the proposed algorithm can rapidly and robustly estimate the correct feature correspondence and spatial transformation relationship of the image to be registered.3.To address the problem of texture information loss and thermal radiation point blurring in multi-scale visible and infrared image fusion,an image fusion algorithm based on total variational model is proposed,effective fusion of multi-scale infrared thermal radiation information and visible light texture detail information is obtained.According to the appearance characteristics of the two types of images,the fused image data is regarded as a composition of data fidelity and regularization terms.The data fidelity constraint assures that the fused image has a similar pixel gray value distribution with the original infrared image,and the regularization constraint assures that the fused image has a similar pixel gradient distribution with the original visible image,and an adjustment parameter is used to control the proportions of the two energy functions.Thus,the image fusion problem is transformed into a convex optimization problem,and can be solved by the Fast Iterative Shrinkage-Thresholding Algorithm(FISTA).Experimental results showed that the fused image obtained by this method can not only retain the thermal target brightness in the infrared image,but also preserve the texture information of the target in the visible image.