Registration Of Cross-Spectral Images Based On Deep Learning

Image registration plays an important role in computer vision,which is widely used in image fusion,image stitching,change detection and etc.Image registration refers to the process of stitching and fusing of two or more images obtained under different imaging conditions.The image pairs to be registered can be either single-spectral images(from the same sensor)or cross-spectral images(from different sensors).This paper mainly focuses on cross-spectral image registration techniques.Cross-spectral image registration not only needs to deal with the problems of geometric deformation and illumination changes,but also needs to address the significant appearance differences caused by sensors with different imaging mechanisms.Aiming at cross-spectral image registration,this paper proposes three local feature methods based on convolutional neural networks.The detailed content is as follows:(1)Registration of cross-spectral images based on Multi-Scale convolutional neural network MS-Net.Patching matching methods based on metric learning model a non-linear similarity function through the network,which can establish more accurate matching correspondences.However,the existing metric learning methods lack a suitable hard mining strategy.To solve this problem,this paper proposes a three-branch-two-channel metric learning network named MS-Net,which can mine hard negative samples based on similarity scores.Furthermore,in order to enhance the scale invariance and discrimination of features,a lightweight Multi-Scale Convolution Module(MS-Module)is proposed in this paper.Experiments show that MS-Net not only achieves good performance on patch matching,but also can be successfully applied in cross-spectral image registration.(2)Registration of cross-spectral images based on attention model FocalNet.During the training process,the network can easily misjudge hard negative samples with high visual similarity.Attention mechanism has achieved remarkable results in many computer vision fields.This paper proposes FocalNet to introduce the attention mechanism to the image patch matching task.FocalNet can focus on important local detailed features and ignore unnecessary ones,which greatly enhance the discrimination of features.In addition,the widely used triplet loss only considers the distance between positive and negative pairs,and ignores the intra-class distances.Excessive intra-class distances damage the generalization ability of descriptors.To alleviate this problem,this paper proposes a margin loss,which further enhances the generalization ability of descriptors.Experiments show that the FocalNet achieves superior registration performance for cross-spectral images.(3)Registration of cross-spectral images based on manifold distance model ArcNet.Almost all existing descriptor learning methods adopt L2 distance to measure whether two patches are match.However,the descriptors are usually normalized to the hypersphere,thus the linear L2 distance does not make full use of the properties of the descriptor space.In this paper,the manifold distance(arc distance)is utilized to measure the similarity between patch pairs,thus a triple loss based on the arc distance margin(Margin ArcLoss)is proposed.Furthermore,since the optimal margin needs to be chosen through lots of experiments,this paper proposes an Adaptive Margin ArcLoss.The Adaptive Margin ArcLoss achieves the best performance on existing image matching and retrieval tasks.Meanwhile,the distance measurement proposed in this paper does not introduce additional computational costs.For cross-spectral image registration,the proposed method can realize sub-pixel level registration accuracy.