Object Detection Of Optical Remote Sensing Data Based On Spatio-Temporal Features And Keypoints Generation

With the rapid development of remote sensing technology,optical remote sensing data gradually plays a more and more important role in various fields.It is widely used in marine object monitoring,urban information monitoring,natural disaster response and military security.Optical remote sensing images and video not only contain rich spatial information,but also have abundant dynamic information in the temporal dimension,which is helpful to continuously obtain the information of the objects.In the research of optical remote sensing data processing and analysis,object detection is the basis of further mining remote sensing information.The combination of object detection and remote sensing data has important practical significance.This thesis focuses on object detection of optical remote sensing data based on spatio-temporal features and keypoints generation.By extracting and integrating the temporal features of remote sensing data,and combining with the keypoint-based object detection framework,the accuracy and robustness of optical remote sensing data object detection model are improved.The main contribution of this thesis is summarized as follows:(1)To avoid the over parameterization of oriented anchors and the instability of object angle learning process,a novel oriented object representation based on midpoints is proposed,and an anchor-and-angle-free detection network,Mid Net,is constructed.Mid Net predicts the coordinates and matching vectors of 5 keypoints(4 boundary midpoints and object center)of the oriented object,then suppresses and groups these keypoints and constructs the oriented bounding-box based on analytic geometry.To mine the discriminative features of midpoints in complex background,a novel symmetric deformable convolution structure is proposed based on the structural characteristics of symmetric objects.Mid Net provides a new perspective for coding the oriented object by keypoints,which avoids unstable angle prediction and over-parameterized anchor setting.(2)To improve the robustness of existing video inter-frame information extraction methods and the stability of deep learning model for small object size regression,a motion guided RCNN(MG RCNN)based on guided anchoring mechanism is proposed for small object detection in optical remote sensing video.MG RCNN consists of two main parts: motion augment network and mini region proposal network.In the motion augment network,the newly designed mean difference method is used to obtain the difference map which is insensitive to the background change and object moving speed,and then the backbone is used to generate the fusion feature of temporal and apparent information.In the mini region proposal network,one branch is responsible for predicting the location of the anchors,the other branch is responsible for predicting the width and height,and it achieves more stable and accurate object size regression with the novel smooth Io U loss.(3)To enrich the scant appearance information of objects through temporal and semantic information,and make the detector adapt to the various distribution patterns of objects in remote sensing data,a temporal and semantic-embedded density adaptive network for moving vehicle detection in satellite videos(SDANet)is proposed.By predicting clusterproposals and object centers,SDANet suppresses,matches and partitions the predictions according to the density distribution of moving objects hierarchically and recursively.Meanwhile,to reduce the interference caused by sensor displacement,illumination changes and other factors,a customized bi-directional conv-RNN module extracts the temporal information from consecutive input frames by aligning the disturbed background.In addition,the road information is integrated into the network end-to-end in a weakly supervised way.According to the semantic prior,SDANet narrows the search area and focuses on the regions of interest,reducing the false detection effectively.The above researches have achieved the state-of-the-art object detection performance in HRSC2016 dataset,FGSD2021 dataset,Jilin-1 video satellite dataset and Sky Sat video satellite dataset.