Deep Feature Representation Learning For Image Matching

Image registration,also known as image matching,is crucial in the field of image processing,as well as the basis and premise of other image processing tasks.The accuracy of image matching will directly influence the performances of subsequent image processing tasks.Thus,how to achieve accurate image matching is very important.At present,the traditional method based on handcrafted features is widely used in image matching.However,the handcrafted features lack adaptability for different images and are not suitable for the case of there are significant changes in appearances.In recent years,deep learning-based methods have achieved significant advantages in global feature learning,such as image classification and object recognition.However,the study of deep local feature learning in low-level computer vision tasks still needs further exploration.This paper aims to use the deep local feature representative learning method for improving the accuracy and adaptability of image matching.The main contents and contributions of this paper are summarized as follow:(1)This paper proposes a deep self-learning network for remote sensing image registration.Traditional handcrafted features lack adaptation,robustness and their matching accuracy need to be further improved.This paper unifies feature learning and feature matching in a deep network,which directly learns the mapping function between the image patch-pairs and their matching labels.The proposed deep network can adaptively extract features from remote sensing images and improve the accuracy and the robustness of remote sensing image registration.Meanwhile,we propose a self-learning method to automatically generate a lot of labeled training samples through images and their transformed copies,which can meet the training requirements of the deep network.Additionally,we use transfer learning to accelerate network convergence and reduce training costs.Experimental results show that this method can achieve a sub-pixel error or close to a sub-pixel error on a large number of remote sensing images.(2)This paper proposes a descriptor consistency learning method for multi-modal remote sensing image matching.Firstly,to alleviate the influence of the significant difference in multi-modal images,it uses the deep convolutional network to extract the shared features.Moreover,we propose a descriptor consistency loss,including the intra-modal consistency loss and the inter-modal consistency loss.It uses the learned knowledge from the high-level features to guide the low-level feature learning and realizes the self-distillation learning in the feature learning network.To keep the discriminant information in images as much as possible,we introduce a reconstruction loss to optimize the feature learning network.Experimental results on optical remote sensing image and SAR image matching have verified the effectiveness of this method.(3)This paper proposes the element-wise feature relation learning method for image matching.Existing methods focus on learning the invariant and discriminative features for individual image patches.However,the essence of the image patch matching task is to predict the matching relationship of patch-pairs,matching or non-matching.Therefore,this paper transforms feature learning to feature relation learning.Additionally,we propose two element-wise learning methods,the feature product and the feature difference,which can naturally capture the feature relation.Moreover,this paper aggregates feature relation from multi-levels,which integrates the multi-scale information to enhance the discrimination of features and achieves more precise matching.Experimental results on single-spectral image patch matching and cross-spectral image patch matching show the effectiveness of the proposed element-wise feature relation learning method.(4)This paper proposes a multi-relation learning network for image patch matching.It is difficult to accurately model the complex changes between images by the single feature relation learning model.Thus,we propose to fuse multiple feature relations and use their complementary advantages between different feature relations to improve the matching performance.Additionally,we propose to use the multiple losses based on different feature relationships for matching network optimization.Due to the rich supervisory information in multiple losses,they will enhance the network optimization,accelerate the network convergence,and improve the matching performance.Extensive experiments on single-spectral image patch matching,cross-spectral image patch matching,and multi-modal remote sensing image patch matching show the effectiveness of the proposed multi-relation learning network.(5)This paper proposes an end-to-end deep matching method to jointly train a learnable key-point detector and a learnable descriptor.Firstly,we propose a dual spatial-context detector(dual-det)to find discriminative and positional accurate key-points.Dual-det has a spatial path to extract rich spatial information and a context path to capture context features.Moreover,we propose a novel rank consistent loss to optimize key-point detection network.The rank consistent loss contains a score rank consistent loss and a score-discrimination rank consistent loss,which are used to ensure the key-points have high repeatability and high discrimination,respectively.In the training processing,the learned descriptor will guide the detector optimization,which will strengthen the collaborative training between the detector and the descriptor.Experimental results on illumination changes dataset and viewpoint changes dataset demonstrate that our proposed dual-det and rank consistent losses have significant advantages for image matching.