Research On Multiple Object Tracking In Satellite Video

With the rapid development of remote sensing technology,the capabilities of satellite payloads has been significantly improved,achieving a leap from image to video for earth observation.Satellite videos not only contain rich land information,but also capture more temporal information,which plays an important role in many remote sensing applications,such as traffic control,motion analysis,and security monitoring.Multiple object tracking in satellite videos involves intelligent algorithms to detect objects of interest,analyze the motion patterns of the objects,and estimate their trajectories.It has important applications in urban monitoring,military security,and other fields.This thesis mainly focuses on the problem of multiple object tracking in satellite videos.To address the characteristics of satellite videos,corresponding algorithms are designed to improve the performance of tracking.Multiple object tracking in satellite videos mainly faces the following problems:First,satellite videos are captured from a long distance and cover a large area with complex geographic backgrounds.Consequently,it leads to the existence of a large number of small and feature-weak objects,making the process of object detection and data association difficult.Second,due to the small and feature-weak objects and motion blur caused by object motion,it is difficult to annotate satellite videos accurately.In addition,identity information of the objects needs to be annotated in the tracking of satellite videos,which undoubtedly increases the time and cost of annotation.Therefore,this thesis mainly focuses on the above two problems to research multiple object tracking in satellite videos,and the main research content and contributions are as follows:(1)To address the challenge of tracking small and feature-weak objects in satellite videos,this thesis proposes a multiple object tracking algorithm based on small and feature-weak object detection in satellite videos.The proposed algorithm follows the paradigm of tracking-by-detection and improves the handling ability of small and feature-weak objects in both the detection and association stages.In the detection stage,attention mechanisms are integrated into the feature extraction to improve the detector’s ability of feature representation.Moreover,an extra prediction branch that can produce a high-resolution feature map is added to the detector,which is more robust for small and feature-weak object detection.In the association stage,since the detection confidence of small and feature-weak objects is relatively low,both high and low confidence detections are considered to ensure that the detected objects can be associated with existing trajectories.The proposed method achieves 63.5 MOTA and78.9 IDF1 on the AIR-MOT dataset,which outperforms other multiple object tracking algorithms.(2)To alleviate the annotation problem of satellite videos,this thesis proposes a multiple object tracking algorithm in satellite videos based on multiple source domain adaptation.Firstly,a cross-domain detector is built based on the teacher-student model,and adversarial training is incorporated into the framework to learn domain-invariant features.To meet the real-time requirements of multiple object tracking,the one-stage detector is integrated into cross-domain detection.Secondly,multiple source domain knowledge is fused to improve the detection performance in satellite videos.Specifically,two identical teacher-student models are constructed,and the two teacher models take the same unlabeled satellite video as input to generate different knowledge.The two student models respectively learn domain-invariant knowledge from different source domains and different teacher models.Thus,multiple source domain knowledge is fused in a mutual learning manner.Finally,based on the proposed cross-domain detector,ID pseudo-labels are generated to train the Re-ID model.In the association stage,the motion information and Re-ID features are used to estimate object trajectories.Two cross-domain satellite video datasets were constructed,and the proposed method outperforms other methods on both datasets.