Research On Real-time Oriented Video Human Segmentation Algorithm

Human segmentation is a common task in computer vision,and plays an important role in the understanding of human activity for computer,widely used in the fields of video processing,video surveillance,human pose estimation,3D modeling and personal entertainment,etc.With the rapid development of computer vision and machine learning technologies in recent years,segmenting human part in a given image or video has attracted the attention of more and more computer vision researchers.Human segmentation based on deep learning models from a single image has obtained significant improvements.However,when directly adopting existing deep human segmentation model on video human segmentation task,the performance of the model often has great limitations,e.g.,the segmentation results of video frames are discontinuous,and the speed of segmentation process is slow,etc.To address these issues,this paper proposes a real-time video-based human segmentation algorithm framework which is applied to the field of single person video human segmentation.The main work of the paper is as follows:(1)A single person video dataset for human segmentation is built.Due to the lack of publicly available datasets for single person video human segmentation and to test the performance metrics of the proposed framework,such as accuracy as well as speed,we build a new single person video dataset for human segmentation of high resolution for the purpose of providing the research foundation for the research of video human segmentation.There are 1 000 video frames in total in the datasets with each frame annotated manually and pixelwise.(2)A video human segmentation algorithm framework based on the deep human segmentation model and the visual tracking method is proposed.The proposed framework consists of a deep human segmentation module centered on the fully convolutional network and a visual tracking module based on the level set method,where the deep learning model segments the human part in the first frame of the video sequence,and the visual tracking module obtains the segmentation results of the next frame using the segmentation result of the last frame.The deep learning model is trained using public human images datasets to ensure high accuracy of human segmentation.(3)A real-time video human segmentation algorithm system is designed and developed.Using the proposed real-time video-based human segmentation algorithm framework,a GPU accelerated real-time video human segmentation algorithm system based on C++as well as MATLAB languages and Caffe,OpenCV plus Dlib open source libraries is implemented under the operating system of Linux.The proposed algorithm framework has achieved the objective of video human segmentation.Experiments on the human video dataset show that the proposed algorithm framework can achieve very high segmentation accuracy on video and higher speed than the methods which use deep learning models to segment human in video frames individually.Besides,the deep human segmentation model significantly outperforms other algorithms in the paper on human image datasets in accuracy.