Remote Sensing Image Registration And Multiresolution Fusion Classification Based On Dual Branch Deep Neural Network

In the research of using remote sensing technology to detect the earth's surface,the matching of remote sensing images and multi-resolution fusion classification have always been a key research topic.On the one hand,with the development of information technology and the support of hardware equipment,people can obtain more and more high resolution and complex remote sensing images from satellite,aircraft and other remote sensing platforms.The unique characteristics of these remote sensing images,such as multi-source heterogeneous data,variable target structure and complex background,make it more and more difficult to meet the needs of efficient interpretation.On the other hand,the development of deep learning in the fields of natural images,video and voice is in full swing,which shows its powerful feature extraction ability for massive data,but its application in remote sensing is just beginning to show its brilliance.Unlike close-range natural images,remote sensing images have many characteristics,such as complex characteristics,large noise interference,distortion of local information,containing a large number of different scales of topographic information,and few sample markers.Therefore,in this work,we make full use of the special nature of remote sensing data,and design a variety of deep learning models specially for remote sensing data to complete remote sensing image registration and fusion classification tasks.In addition,these works can be applied to other related tasks independently,and have strong generalization performance.These achievements have also been recognized by domestic and foreign counterparts,the specific contents are as follows:1.Aiming at the data characteristics of SAR image in remote sensing data,a SAR im-age registration method based on multi-feature detection and arborescence networkmatching is proposed.Multi-feature detection strategy is helpful to retain two type-s of feature information at the same time.Compared with the traditional one typefeatures,this strategy can detect rich texture features and find stable corner features.It can expand the number of feature points and enrich the types of features,makingfull use of image information to prepare for the subsequent registration process.Con-sidering the speckle noise of SAR images,an exponentially weighted average ratiooperator based on statistics is used to calculate the gradients of the two detectors.Inthe process of feature matching,the proposed arborescence network matching algo-rithm is mainly composed of two parts: backbone network and branch network.Withthe construction of the network,matching pairs are searched.The algorithm combinesthe spatial relationship between feature points and feature constraints,and has morematching pairs and higher sub-pixel matching accuracy than the original algorithm.In SAR image registration tasks,the algorithm has better robustness and effectivenessthan traditional algorithms.2.Aiming at the characteristics of remote sensing images with higher resolution,largerscene and more complex structure,a feature matching algorithm based on dual-branchconvolution deep belief network is proposed,which transforms image registration taskinto a binary classification problem.In order to match two key points,two key point-centered image patches are input into the network.The aim of the network is tolearn the salient feature representation for image patch matching,so as to obtain morematching pairs while maintaining high sub-pixel matching accuracy.The network usestwo-stage training to process complex features of remote sensing images.In addition,an adaptive sample selection strategy is proposed to determine the size of each blockby the scale of its central key points,so as to determine the neighborhood of the sam-ple.Therefore,each patch can retain the texture structure around its key points,ratherthan all blocks have a predetermined size.In the stage of matching prediction,in orderto improve the matching efficiency and accuracy,the strategy of sample classificationbased on super-pixel and the strategy of ordered space matching based on super-pixelare designed respectively.The experimental results and theoretical analysis prove thefeasibility,robustness and effectiveness of this method.3.An adaptive feature fusion spatial network is proposed for high resolution remotesensing image registration.The network has the concept of multi-scale.It does notneed to consider that the domain scope is determined for each sample.It can adap-tively select appropriate neighborhood information for different samples.By fusingthe deep features with the shallow features and adaptively adjusting the fusion weightsaccording to the characteristics of the input samples,we can provide a robust featurerepresentation for the input samples.In addition,the idea of spatial transformation isembedded in the network,so that the two branches can be adjusted to the same coor-dinate system as much as possible before fusion,so as to improve the confidence ofmatching prediction.4.A two-branch feature fusion network is proposed for the fusion and classification ofmultispectral and panchromatic images.It aims to integrate feature level fusion andclassification into an end-to-end network model framework.Considering a large-scaleremote sensing scene,an adaptive sample selection strategy is proposed.In the net-work architecture,we propose a dual-path module,which can effectively mitigate thegradient explosion in residual paths while guaranteeing the maximum gradient infor-mation flow between layers in densely connection paths.This module can extractmore powerful features to deal with the complex features of remote sensing images.Finally,we integrate the characteristics of these two branches step by step in a progres-sive and cooperative way,so as to reduce the computational burden and improve theclassification accuracy.Experiments show that the algorithm performs well in remotesensing image fusion and classification tasks in large scenes.5.A dual-branch attention fusion depth network for multi-resolution classification of re-mote sensing images is designed.In the process of building training sample library,anadaptive center offset sampling strategy is proposed,which allows each image patchto determine its neighborhood adaptively by detecting the texture structure of the pix-els to be classified,unlike the traditional pixel center sampling strategy.However,the neighborhood range is not symmetrical with this pixel,so we hope to capture theneighborhood information which is more conducive to its classification.In networkstructure,based on the image patches captured by sampling strategy,a channel atten-tion module is designed for multi-spectral data,which highlights the advantages ofrich spectral information of multi-spectral data,while a spatial attention module forpanchromatic data is designed to highlight the advantages of high spatial resolution ofpanchromatic data.Then,the two features are fused to extract deeper features fromthe fused features for classification.The experimental results on high resolution re-mote sensing data sets demonstrate the effectiveness and robustness of the proposedmethod.