Video Saliency Detection Method Based On Visual Attention Mechanism

The human visual system has powerful information processing capabilities,and humans can find their areas of interest in a very short time.The technique of extracting salient areas in images or videos through computer vision-related algorithms is the saliency detection technique.In recent years,with the development of technology and the requirements for information processing speed,the saliency detection technology has attracted attention and made great progress.The traditional salient object detection method is dedicated to detecting the area where the salient object is located.When people are concentrating,they often only focus on a part of an object,not the whole object.Therefore,detecting the gaze position by detecting the eye movement can better simulate the attention transition.A model based on deep neural networks to predict attention transition is proposed.The model uses the spatiotemporal features to extract static saliency maps and dynamic attention maps respectively.They are fused into a final saliency map through a full convolutional network for predicting the attention transition.An eye tracking dataset is marked with eye tracking system.In this thesis,for the task of attention transition,the model combines two attention mechanisms,including the saliency detection of spontaneous eye movements and the task-related attention transition prediction.It can also consider the impact of image data information on visual attention,and the influence of cognitive factors and task factors on attention transition.In addition,a new eye movement data processing algorithm is proposed in this thesis,using eye tracking system to mark the eye movement dataset.The main work of this thesis is as follows:(1)In order to better simulate visual attention transition,an eye tracking system is designed for experiments,a dynamic scene video eye tracking data set is annotated,and an eye tracking data processing algorithm combined a probability distribution algorithm and a fuzzy C-means clustering algorithm was proposed,which help with the saliency model building.(2)A video saliency detection model based on multi-scale spatiotemporal features is proposed to predict attention shift in dynamic scenes.The model uses the spatial pyramid pooling model to extract spatiotemporal features,extract static saliency maps and dynamic attention maps separately,and use full convolutional networks to fuse them into the final gaze to predict attention transition.The model proposed in this thesis considers both spontaneous eye movements and command-driven gaze situations.