Research On Online Continual Object Detection For VHR Remote Sensing Images

With the rapid development of remote sensing technology,high spatial resolution remote sensing images provide a data basis for a large number of applications,and are of great value in both the civilian and military fields.The combination and application of object detection and high-resolution remote sensing images is an important research direction of remote sensing image processing and also an significant basis for dealing with complex visual tasks.Most of the existing object detection algorithms follow the offline learning mode,and improve the fitting result of the model by repeatedly learning the training set.However,advanced imaging techniques have reduced the difficulty of obtaining high-quality data,resulting in the continuous increase of remote sensing data.In the face of massive data,offline learning requires more time costs and computing resources.Online continual learning has become a research hotspot because it can quickly and effectively learn remote sensing data.This research studies online continual object detection for remote sensing images,which aims to improve the sustainable learning ability of the model while avoiding the catastrophic forgetting.The catastrophic forgetting issues stem from the overwriting of old parameters in models when learning a new task,resulting in the model being unable to detect past data.To deal with this problem,this study constructs a memory to save the learned data and selectively replay the data for training in the subsequent process.In addition,the imbalance problems have great impact on the model in the online continual learning mode,and this study solves the problems,including the class imbalance,the scale imbalance and the spatial imbalance.The main works are as follows:(1)For the problem of class imbalance,this study proposes a new online continual object detector based on the method of replay.In the online continual learning mode,the class imbalance has fundamentally changed that the priority of maintaining the balance of the number of images between classes is more important.The proposed approach includes two main modules: an entropy-based reservoir sampling algorithm and a priority assignment network for rehearsal imbalance,which are responsible for constructing memory and selecting old samples for replay respectively.The entropybased reservoir sampling algorithm introduces entropy as a metric to judge the balance in the memory and guide the memory to update data.The priority assignment network for rehearsal imbalance automatically assigns weights to images in the memory through a small-scale neural network,it aims to maintain the balance of replayed data in various situations.Our method outperforms other existing online continual learning algorithms that deal with class imbalance,and achieves 5.6%,2.7% and 2% mAP improvements on NWPU VHR-10,DOTA and DIOR datasets,respectively(2)For the serious scale and spatial imbalance problems in the DIOR dataset,this study improves based on the previously proposed detector.For the scale imbalance,the entropy-based reservoir sampling algorithm is further adjusted to maintain the scale balance in each class in the memory while maintaining the rehearsal balance.In addition,the feature extraction module adds a bottom-up path to reduce the feature transfer distance and reduce the loss of detailed information.For the spatial imbalance,the L1 loss is replaced by CIoU loss,and the similarity will be jointly represented by overlap area,aspect ratio and central point distance.This study compares the detection effect before and after the improvement based on the DIOR dataset.Our method obtains a5% increase in mAP; at the same time,the detection effect of small-scale objects is significantly improved,and the mAP increase reaches 5.2%.