Research On Video Saliency Detection Method Based On Deep Learning

After long-term evolution,humans have developed a visual attention mechanism that can quickly and selectively find interesting targets in complex environments.How to simulate this visual attention mechanism is one of the important research contents in the field of computer vision.Video saliency detection can quickly find the region of interest from the video through computer simulation of human visual attention mechanism,which is widely used in target recognition,behavior analysis,image retrieval and other fields.This paper mainly studies the application of deep learning in video saliency detection.Firstly,we introduced the research background,current progress and related knowledge of saliency detection and deep learning,incuding human visual attention mechanism,image segmentation method,convolutional neural network and long shortterm memory network.Then,based on the related law on change of gaze point when people watching video,we proposed a video saliency detection algorithm based on deep learning.The algorithm combines convolutional neural networks and long short-term memory networks to design two deep neural networks model.The convolutional neural network is used to detect the saliency within the video frame,which is based on two networks,YOLO and FlowNet.The YOLO-based part detects the spatial features in the frame and focuses on object detection.The FlowNet-based part detects the time features in the frame,focusing on motion detection,At the same time,the features extracted by the YOLO network are standardized and used as the mask of the partial convolutional layer output of the FlowNet network to make it more focused on the motion detection of the object.Then,a feature fusion network is used to fuse the temporal features and spatial features generated by the two networks and generate an intra-frame spatiotemporal saliency map.In addition,the long short-term memory networks use the results of the convolutional neural network as input to detect the significant transfer between video frames,and add convolution operations to the long short-term memory networks to extract the spatial features between video frames,Meanwhile,a centerbased dropout is designed to prevent overfitting,and finally the final saliency map is generated by deconvolution.For the algorithm model proposed in this paper,several common saliency scoring methods are used to comprehensively evaluate the training database and other databases.The results show that the proposed algorithm has better performance in detecting video saliency than other algorithms,and its generalization is relatively good.