Detection And Multi-object Tracking In Satellite Videos Based On Deep Learning

With the rapid development of remote sensing monitoring technology to the ground and the more accessible of satellite video data,satellite video is gradually playing a more and more important role in various fields,which is widely used in marine target monitoring,urban information monitoring,natural disaster response,military security,etc.Satellite video not only has abundant spatial information,but also has more dynamic information in the time dimension,which is helpful to continuously obtain the valid information of interested moving objects in the ground.In satellite video research,object detection and multiobject tracking is the hot topic of satellite video interpretation.In this paper,object detection and multi-target tracking in satellite video based on deep learning is studied,which improve the accuracy and timeliness of detection and tracking in satellite video via extracting the motion and spatial features of satellite video object.The main contributions of this paper are summarized as follows:Firstly,a cross-frame keypoint-based detection network for satellite video is proposed.Considering the motion characteristics of satellite video object,cross-frame module is designed to extract and fuse the appearance features of the current frame and the temporal features between different frames simultaneously.It is beneficial to improve the recall of small and blurry object in satellite video.Besides,this module can be inserted into any current deep learning detection network.Then,soft or hard mismatch suppression strategy is devised to suppress the unreliable bounding boxes,which solves the problem of low precision of detection network based on keypoints in satellite video.That improves the detection precision of the network through avoiding using similar embedded vectors to match keypoints.Through experimental analysis,the excellent object detection ability of the proposed algorithm in satellite video is verified.Secondly,a spatial motion information-guided tracking network is proposed.In terms of multi-object tracking in satellite video,it is difficult to track the crowded similar vehicles stably only by using the appearance or motion information.To address these problems,a two branch LSTM network is designed.One LSTM is constructed to extract the spatial information between neighboring vehicles in the same frame,another LSTM establishes the motion model of vehicles via the movement velocity in consecutive frames.Then,the spatial information and motion information are combined to determine whether the tracking trajectory is a real tracking trajectory.At the same time,a virtual position is generated for missed or lost targets.In this way,it is conducive to tracking dense targets on satellite video and improving the stability and robustness of the network.The proposed algorithm skillfully makes use of the motion characteristics and spatial characteristics of satellite video object.The experimental results show that the proposed algorithm has better tracking performance.Finally,a cost and re-identification network for joint detection and tracking is devised.In satellite video,it is difficult for joint detection and tracking network to detect and track well under the conditions of large displacement.To deal with this issue,a correlation module is designed to extract the motion features of fast/slow targets between adjacent frames.Then,a cost-volume feature pyramid network is proposed to fuse the apparent features and motion features of the deep and shallow layers,which is beneficial to detect and track moving targets in satellite video.Due to the lack of apparent information of satellite video objects,the ReID features with apparent attributes has the weak discrimination when identifying targets.To address this problem,a learning mechanism of ReID features with motion and spatial attributes is proposed,which uses motion features and spatial features to improve the discrimination of ReID features.In this way,the ReID features has strong discrimination ability which can improve the network tracking ability in satellite video.Through experimental analysis,the proposed algorithm achieves end-to-end joint detection and tracking via a single network,and its feasibility and effectiveness in detection and tracking tasks is also verified.