Optical And SAR Image Registration Based On Deep Learning

The main goal of image registration is to realize the mapping,alignment and splicing of two or more images corresponding to the same target scene under the influence of different perspectives,sensors,lighting,etc.Image registration is widely used in geography,agriculture,military,remote sensing and biology.As a basic problem in the field of computer vision and image processing,image registration has been widely concerned.Many classical algorithms have been proposed to solve this problem.For example,SIFT,FAST and other algorithms.The common feature of these algorithms is that they need to design feature extraction methods manually.These methods have made great progress in the tasks of optical image and SAR image matching and registration.With the extensive application of deep learning technology,the method of manual feature design is gradually replaced by deep neural network.Many methods based on deep learning are proposed and applied in registration field.Therefore,this paper focuses on the application of deep learning in image matching and registration.In this paper,the key technologies of image registration task based on deep learning are studied,and the tasks of image patch matching,optical image matching,registration and optical and SAR image registration are analyzed and studied.The main work of this paper is as follows:1.An image patch matching method based on exponential loss is proposed.This method considers the weighting of different hard samples from the perspective of loss function,so that the network has a higher degree of concern for hard samples,and a smaller weight for easy samples,accelerating network optimization.Experimental results show that good results are achieved in metric learning and descriptor learning.2.A two-stage image matching network based on multi-scale and high-resolution is proposed to realize optical image matching.In order to make full use of the features of different deep networks,a feature learning network based on attention mechanism is proposed,and the number of repeatable feature points is further increased by weighted fusion.3.A simple and efficient feature extraction network CRNet is proposed.Experiments on multiple datasets show that the single-layer feature extraction network also has good matching results.Furthermore,the performance of the two-stage matching network is limited by the learning of image patch scale,angle and descriptor.4.By improving the single-stage detection network D2 Net,a multi-scale fusion method is proposed to aggregate feature points to achieve optical and SAR image registration.Furthermore,the influence of descriptors and types on registration results is analyzed,which improves the accuracy of optical and SAR image registration.