Research On Instance Segmentation Method For Long-tailed Distribution Scenario

With the booming development of artificial intelligence technologies,computer vision-related applications such as autonomous driving,face recognition and industrial inspection are increasingly appearing in people’s daily lives.Instance segmentation methods combine the three tasks of visual classification,detection and segmentation,and are one of the most challenging tasks in the field of computer vision.Currently,deep learning-based instance segmentation methods have achieved great success on datasets with relatively balanced number of class samples.In real natural scenarios,the number of samples from different categories is not evenly distributed,but tends to show a long-tailed distribution trend.Traditional instance segmentation methods perform poorly in such real scenarios,and the accuracy of the model is significantly reduced,which is obviously difficult to meet the requirements of practical applications.In this paper,by analyzing the characteristics and shortcomings of existing long-tailed instance segmentation methods,two new methods for long-tailed instance segmentation that can be used collaboratively are proposed on the basis of the Mask R-CNN framework.(1)A loss function called Foreground and Background Separation Loss(FBSL)is proposed.This function reweights the foreground instances and background in the overall data in a separated form asynchronously,and then combines the two weights after reweighting to calculate the loss,finally achieving the purpose of balancing the gradients of positive and negative samples of different categories.For foreground instances,this paper adopts a reweighting mechanism based on the ratio of positive and negative sample gradients,which will up-weight the positive samples of each instance class and down-weight its negative samples.In addition,it is argued that the large negative sample gradient brought by the background classes increases the pressure of category balancing training,especially for classifiers responsible for the tail classes which still run the risk of learning bias.Therefore,for background instances,this paper uses a weakening mechanism consisting of confidence and hyper-parameters to weight the background class instances additionally,weakening the importance of the background.With the synergistic effect of the two strategies,the FBSL method can induce a more balanced training of the network model,thus substantially improving the tail category performance.(2)A method called Feature Probabilistic Augmented Sampling(FPAS)is proposed to further improve the tail category performance.The method consists of a Feature Storage Module(FSM)and a Probabilistic Augmented Sampler(PAS).The FPAS generates a set of instance features from the region candidate network and stores them for reuse during the training iterations.Finally,in the prediction phase,the PAS performs targeted sampling of the stored instance features based on the average recognition probability of the sample features.In addition,to further improve the instance segmentation performance,this paper introduces a Refine Mask Module(RMM)in the mask segmentation network branch,which fuses fine-grained features in multiple stages and refines the segmentation boundaries to obtain high-quality instance segmentation results.Extensive experiments on LVIS v1.0 and COCO-LT datasets demonstrate that the proposed methods can effectively alleviate the long-tail distribution problem.Compared with the current mainstream long-tailed instance segmentation solutions,the approach incorporating FBSL,FPAS and RMM achieves the best detection and segmentation results.