Video Object Segmentation Based On Multiple Hypotheses Propagation

Semi-supervised video object segmentation(VOS)problem mainly studies that given the segmentation of a given target object in the first frame,automatically segment the target object in the subsequent frames.VOS is the basic task of a variety of widely application,including video editing,video summary and action recognition and so on.In recent years,with the rapid development of video-based applications(APP),VOS has been paid more and more attention.In the past two years,it has continued to attract a large number of researchers in academia and industry to engage in scientific research and industrial applications in this field.Although VOS has made great progress in recent years by using some semantic segmentation methods,it is still a very challenging task.The problems to be solved include the loss of the target object,the occlusion of the object,the huge deformation,the complex interaction between the objects,the fast movement and so on.In addition,because VOS itself is a semi-supervised learning(SSL)problem,the amount of data it can provide is very small,which further affects the performance of segmentation.This paper mainly studies some problems that VOS needs to solve at present,focusing on the Class-Agnostic Video Object Segmentation(CAVOS).A variety of VOS algorithms are proposed in this paper to solve the problems of object loss,object occlusion and so on.Firstly,combining artificial rules with deep neural networks,we propose a Class-Agnostic Video Object Segmentation without Semantic Re-Identification algorithm(CAVOS-NS).The important idea is to use a occlusion detection algorithm.The linear motion model of the method obtains the boundary box candidate of the object in the current frame by modeling the motion track of the object,and combined with the mask refinement CNN network and iterative spatio-temporal segmentation refinement module to get the final fine segmentation results.Occlusion detection algorithm uses manually designed detection conditions to monitor whether the object is occluded,which needs to be set separately on different data sets,and the generalization ability is not strong.Using the CAVOS-NS algorithm,we finished third in the DAVIS 2018 test-challenge competition,validating the performance potential of the category-independent VOS algorithm.Then,we propose a Multiple Hypotheses Propagation for Video Object Segmentation(MHP-VOS),the main idea of which is to use the object detection model to obtain all the object candidate frames for each frame.Then these candidates are combined into a hypothetical space tree growing in time dimension through the gate control algorithm,and then each segmentation path is scored with motion score and segmentation forward propagation score,and the optimal path is obtained by pruning continuously.This method uses long-term time information to make delay decision,in order to select the best candidate box,instead of the manual rules in CAVOS-NS algorithm.A large number of experiments show that MHP-VOS algorithm has achieved the best performance of video object segmentation,and can effectively solve the problems of object loss and occlusion in multi-target object segmentation.