Research On Video Object Segmentation Via Meta-learning

Video object segmentation(VOS)task is a fundamental but challenging problem in the field of computer vision.This task can be described as given the mask of the object in the first frame and gotten the separation of an object from the background in a video.As an important part of big data processing applications,video object segmentation technology will play an important role in future.And it can serve many higher level applications including video surveillance,virtual reality,anomaly detection,autonomous driving,etc.Conventional deep neural networks based video object segmentation method have two main problems to face: First of all,it is hard to segment the objects from the background in a video sequence under various challenging environments such as occlusion,large appearance variation,background clutter and illumination variation.The second point is that those methods are dominated by heavily fine-tuning a segmentation model on the first frame of a given video,which is time-consuming and inefficient.To deal with those problems,we propose a meta-learning based video object segmentation framework.Specifically,our approach builds a self-adaptive learning process by updating hyperparameter.And then we can improve our method from three aspects: confidence patterns,meta-learning strategy and network design.This work mainly includes the following three aspects:(1)Double flow confidence patterns based video object segmentation modelIn order to obtain more accurate confidence patterns in the online self-adaptive learning process,we introduce a double flow confidence patterns for modeling appearance information and motion information.Here we use a detection method to model appearance information and optical flow algorithm to model motion information.We then utilize both appearance and motion information to construct confidence patterns,which are helpful to further improve the performance of video object segmentation.(2)Faster video object segmentation based on updating hyperparameterWe know that the existing deep learning methods are mainly to fine-tune the segmentation model in the first frame of the test video sequence.It is time-consuming and can not be adapted well to the current target video.For better adapting to testing videos,we construct an online adaptive learning strategy to update hyperparameter which uses confidence patterns in different videos/frames.We then introduce a meta-learning strategy to accelerate model learning process.We use meta-learning strategy to evaluate the confidence patterns for the online adaptation of network,which can confidently online update negative/positive samples under the guide of motion information as well as appearance information.Meta-learning strategy includes four stages: pre-training stage,meta-training stage,meta-fine-tuning stage and metaadaptation stage.(3)Network structure self-optimization for video object segmentation network modelThe design of network structure is very important for deep learning methods.A reasonable network structure can improve performance a lot,so we had tried many methods to design the network structure.As far as we know,network architecture search is also a kind of meta learning.We have to find the best network structure by balancing model size and model performance.Finally,we use the gradient-based neural architecture search method to design the network.Comprehensive evaluations on four benchmark datasets demonstrate the superiority of our proposed framework when compared with other state-of-the-art methods in both single-target and multi-target segmentation tasks.