Prediction Of Sequence Of Eye Fixations Based On Machine Learning

In an rapid information growth era,human may receive hundreds of millions of information per second.It is generally believed that this order of magnitude information has far exceeded the limit of that people can process information in real time.The Human Visual System can extract valuable information from a large amount of input data for people to process in the cerebral cortex,such as visual memory,information classification,target tracking,etc.This mechanism for processing information is visual attention mechanism that helps people extract information and process it in complex scenes.Computer is one of the fastest tools to process information,we try to add visual attention mechanism into computer image processing,which can not only improve the ability of data extraction,but also improve the learning ability of the model and further guide computer to simulate human visual cognition process.When people obtain information,they will focus on the area they are interested in.As the observation area changes,people's attention also shifts.The changes in these gaze positions can be divided into two parts: the fixation point and the saccade path.The fixation point is the research goal in the Saliency Prediction.It reflects the most “conspicuous” and most compelling areas of the scene,reflecting the salient regions of the image with static saliency maps;however the saccade path is a dynamic process,reflecting how to choose the area of interest and the order between these areas.On the basis of investigating the existing theoretical knowledge and prediction models of saccade path,we propose a saccade path estimation method which based on machine learning.Meanwhile we construct the saccade path estimation model based on Recurrent Neural Network.The saccade path is the shift of human eyes between the fixation points.It's a time series.Compared with the existing model,our model based on Recurrent Neural Network whose construction is suitable for processing time series,and the saccade path belongs to time series problems,so modeling with Recurrent Neural Network can better simulate the process of people understanding scenes.Through the analysis of the process,we can understand the human visual cognitive mechanism.This paper builds a saccade path estimation model based on a recurrent neural network,and enhances the performance of the model by adding visual attention mechanism.The image is taken as the input of the convolutional neural network to extract the feature vectors of the input image.The extracted features include not only spatial information but also position information.At the same time,the input coordinates are coded to get a weight matrix that characterizes its features.The images' features and processed labels are used as the input to the decoding network to train the network.The trained network can predict the saccade path of input image.The work in this paper indicates that in the saccade path estimation model which based on machine learning,modeling based on the recurrent neural network can well simulate the human eyes' saccade path.