Research On Registration Method Of Infrared Image And Visible Image Based On Modal Transformation

Image registration technology is the key technology in the field of image processing,and the registration of infrared image and visible image is the key and difficult point.Infrared and visible light are in different bands,and their images belong to different modal images.However,it is difficult to match features between images of different modal,which leads to the low registration accuracy of existing methods.In order to solve the above problems,this thesis proposes an infrared image and visible image registration method based on modal transformation,which can effectively improve the accuracy of image registration while completing image registration.In this thesis,we first propose a modal transformation method based on generative adversarial networks.Based on the comparison of unpaired image transformation Network(CUT),a multi-scale attention U-net structure is optimized to replace the original discriminant network,in which one discriminator inputs a complete generated image to learn the detail texture,and the other discriminator takes the subsampled image as input to grasp the overall image,so as to improve the image quality after modal transformation.In addition,according to the contrast loss of CUT,a spatial correlation loss is proposed to constrain the modal transformation network.Through the experiment and analysis of the effect of modal transformation,it is found that it is difficult to achieve satisfactory transformation effect by modal transformation between images,and the resulting image will produce great distortion.On the other hand,the modal transformation of the image pair with data set can achieve a better transformation effect,and the resulting image is basically consistent with the original visible image in color,outline and appearance.By selecting some classical algorithms to analyze the feature matching results of images before and after modal transformation,it is proved that after modal transformation,the algorithm can obtain more matched feature point pairs and correctly matched feature point pairs.And the minimum number of correctly matched feature point pairs increases from 8 to 11,and the maximum increases from 6 to 111,and the number increased significantly.Therefore,it is proved that the modal transformation method in this thesis can effectively reduce the difficulty of feature matching,which is of great significance for multimodal image registration.Combined with the method of modal transformation,a method of infrared image and visible image registration based on modal transformation is proposed in this thesis.According to the data set of practical engineering application in this thesis,the parameters of registration network are optimized and retrained.The generated visible image obtained from the original infrared image through the modal transformation network and the original visible image are transported to the registration network,so as to output the corresponding matching relationship.Then the corresponding homography matrix is calculated and applied to the original infrared image to complete the registration.Combined with the relevant indexes and the classical registration algorithm,the minimum number of feature point pairs correctly matched by the method in this thesis is 76 and the highest is 97.The registration accuracy of the method in this thesis is 91.11% at the lowest and 96.23% at the highest.The algorithm in this thesis is superior to all kinds of comparison algorithms in the number of feature point pairs and the registration accuracy.It is finally verified that the registration method in this thesis has better registration performance,and can strictly complete the registration work of infrared image and visible image.Finally,for the specific image registration application of image fusion,the experiment proves that the direct fusion of unregistered images will produce ghosting,and the fusion effect is poor.After strict registration,good fusion results can be achieved,and it is verified that the registration method proposed in this thesis plays a positive role in image fusion.