Cross-domain Image Matching Based On Deep Learning

Image matching is the process of converting different images into a coordinate system,which can be multiple photos from different sensors,different lights,or different viewpoints.The traditional matching algorithm matches the image in three steps.First,the key points are extracted from the image,then the key points are extracted.Finally,the perspective transformation matrix is calculated according to the similarity and robust estimation of the key points.A cross-source image refers to multiple photos from different sensors,different generation modes,or different modalities.Similar to image matching,the difficulty of matching is directly proportional to the intensity of image viewpoint and illumination intensity.Experiments have shown that most traditional algorithms have great limitations in cross-source image matching.Therefore,in order to further improve the accuracy of image matching,this paper studies the technology of cross-source image matching.The research is mainly divided into two parts:cross-source image block feature extraction and end-to-end image matching neural network.Firstly,in order to solve the difficult problem of cross-source image feature extraction,this paper proposes a deep neural network H-Net and H-Net++ based on automatic encoder and twin structure.The core of the network is to extract the features of the cross-source image block using an automatic encoder,and then restore the feature to the original image to extract a reliable feature description.The twin structure is used to train the network,so that the feature description of the network extraction is optimized in the dimension space toward the direction of "the intra-class spacing is as small as possible,and the extra-class spacing is as large as possible".Experiments show that H-Net and H-Net++ have an average accuracy of 8%improvement over the previous best method L2-Net in cross-source image block matching judgment and retrieval.Second,existing keypoint detection algorithms are difficult to extract satisfactory key points on cross-source images.Because the composite image used in this experiment has factors such as large image distortion and texture blur,how to extract reliable key points is a difficult problem.Inspired by the LF-Net end-to-end image matching network,this paper designs a keypoint detector based on the receptive field,and proposes a loss function term neighbor mask for the label ambiguity existing under end-to-end matching.Combining these two innovations,the end-to-end image matching network RF-Net proposed in this paper has an average of 20%higher than the previous best method LF-Net under multiple indicators of multiple public data sets.