Infrared Visible Image Registration Based On Modal Transformation

Infrared and visible image fusion technology can make good use of the complementarity of two different modal images,so it has been widely used in various fields,and the image registration is the most basic and critical preprocessing step before image fusion.Traditional image registration methods need to get enough feature points to pair,these feature points can be contour lines or points,need to manually complete manual marking,more time-consuming cost and prone to errors,in addition,due to the different principles of different sensors to obtain images,even in the case of fixed time and area and the relative position of the shooting equipment and subject,the final infrared image and the visible image also have a large difference in statistical characteristics.It is difficult to reliably extract detailed features such as key points or key textures,resulting in greater difficulty in the registration of infrared and visible images.In view of this difficulty,this paper studies the following:(1)In order to minimize the difference between infrared image and visible light image and reduce the difficulty of direct registration,this paper uses PatchGAN as the architecture of modal conversion to convert visible light image into false infrared image.At the same time,D-LinkNet with void convolution is introduced,and the final structure adds hop links and residual modules on the basis of the encoding-decoding system,which can increase the receiving domain of the network and retain the detailed information at the pixel level at different resolutions.In this algorithm,the discriminator receives multiple image blocks that represent the local truth of the image,and only needs to calculate the loss function on the scale of these image blocks,which can not only effectively reduce the number of parameters required,but also avoid losing too much image detail information.(2)The algorithm used for registration uses FlowReg-O.FlowReg-O is an improved algorithm based on the FlowNet optical flow algorithm.Since ground truth for optical flow is difficult to obtain,this paper adopts an unsupervised approach that does not require ground truth.In addition to the photometric loss function and smoothness loss function used in FlowNet,this article also adds a loss function that can add a global intensity constraint to the final result.Each of these three loss functions ensures the overall similarity between fixed and floating images,while also retaining data with small displacements.Experiments show that the method proposed in this paper has excellent performance in the registration of infrared and visible light images.