The Foreground-Foreground Class Imbalance Problem In Object Detection Based On Deep Learning

Object detection is one of the basic tasks in the field of computer vision.It includes two subtasks: classifying objects in images and positioning objects.Before convolutional neural networks became popular,object detection has been regarded as a machine learning task.With the rapid development of deep learning technology,object detection has gradually moved from the period of detection algorithms based on traditional handcrafted features to the period of detection algorithms based on deep learning.Although the improvement of deep neural networks is the most important factor that promotes the development of object detection,the imbalance of different levels in object detection has also received great attention in recent years.At present,the imbalance problems in target detection can be divided into four categories: class imbalance,scale imbalance,spatial imbalance and objective imbalance.If this kind of imbalance is not resolved in time,it will have an adverse effect on the final performance of the detector.Among them,class imbalance is one of the most concerned issues.It appears when the number of samples in different classes is unevenly distributed.It can be divided into foreground-background class imbalance and foreground-foreground class imbalance.For the latter,it may be caused by the data set,or it may be caused by the training batch.Solving this kind of imbalance problem can not only improve the performance of the object detector,but also extend the results to other computer vision tasks,such as semantic segmentation.Therefore,we can study and solve the problem of foreground-foreground class imbalance in object detection based on deep learning.It is valuable and extremely challenging work.This article first researches and analyzes the imbalance caused by training batches,and then tries to improve the existing solutions.The main tasks and innovations are as follows:1.In order to control variables in the process of research,a balanced data set is constructed by combining several data sets.2.Two stage detector Faster R-CNN and random sampling method were used as the control group.In order to change the number distribution of foreground samples,online foreground balanced sampling(OFB)and absolute foreground balanced sampling method were set as the experimental group.The model was trained in different training batches and the experimental results were analyzed.The results show that the training batch does lead to the imbalance problem from foreground to foreground,and for the balanced data set,the more balanced the distribution of foreground samples in training,the better the performance of the detector.3.Aiming at the common data sets with long-tail distribution,an improved relative foreground balanced sampling method based on OFB is proposed.The overall data set distribution is included in the probability calculation of sample sampling.The results show that the improved sampling method can improve the performance of the detector.4.By analysing of the experimental results,it is found that in the two-stage detector training process,the proportion of the number of rounds with the threshold number of positive samples generated by the proposed region generation module in the total number of training rounds is too low,which may be the reason limiting the function of various foreground balanced sampling algorithms.Therefore,the boundary box generation method is introduced for further research.The results show that the introduction of this method can promote the two foreground balanced sampling methods,and the improvement of the improved relative foreground balanced sampling method is the biggest.