Infrared And Visible Image Registration Based On Visual Salient Features

The dissimilarities of sensors in imaging systems result in differences between visible image and infrared image, the fusion of which is conducive to the decision-making process. For the same scene, images captured by different imaging sensors are different in size, viewpoint, field of view, and so on. Therefore, the images must be geometrically aligned before fusion. However, for visible and infrared images, the relationship between intensity values of correspondences is complex; contrast reversal occurs frequently in some regions, it difficult to realize accurate registration by just using the common methods.In order to improve the precision of visible and infrared image registration, an image registration method based on visual salient features is presented. This method includes two parts: a visual salient(VS) feature detector and a descriptor-rearranging(DR) strategy.The VS feature detector based on the modified visual attention model is presented in this paper, and it makes use of both intensity and orientations information and extracts visual salient points in visible and infrared images. First, an intensity scale space and four orientation scale spaces are obtained through linear filters. Then the center-surround difference operation is used to extract early visual features from each image in scale spaces, and the four orientation feature maps are then integrated together to a single orientation feature scale space. After that, an iterative within-feature competition method is adopted to pop out the areas with higher saliency in the early visual feature maps and inhibits low-salient areas, and conspicuity maps are obtained. Finally, the Local maximum points are chosen from the conspicuity maps as the VS feature points. Because the iterative within-feature competition method used in visual attention model is very time consuming, an alternative fast visual salient(FVS) feature detector that uses average of local maxima as threshold to eliminate less-salient areas is proposed to make visual salient features more efficient in applications.A descriptor-rearranging(DR) strategy is adopted to describe feature points. This strategy combines information of both infrared image and its negative image by just computing the descriptors of salient points in infrared image, and the descriptors of its negative image can be obtained by rearranging these computed descriptors. These two types of descriptors are combined together as the final descriptor of salient points in infrared image. Then feature match is achieved, and image transformation parameters are computed to align visible image and infrared image.SIFT feature detector is used in this paper to make a comparison with the visual salient feature detector in performance. The standard dataset of Mikolajczyk et al. are used to evaluate the repeatability of VS, FVS and SIFT feature detectors, and the computational complexity of each feature detectors are also calculated. Three pairs of infrared and visible images are used to evaluate the registration results of different methods in registration accuracy, operation time, correct matching pairs and stability under noise. Experimental results show that both VS feature detector and FVS detector have higher repeatability score than SIFT. The combination of VS detector and DR registration strategy has higher match effectiveness and can achieve precise image registration, but its computational complex iterative method makes it time consuming. Whereas, the combination of FVS detector and DR registration strategy is more efficient because it can reach a good registration of visible image and infrared image with a shorter time.