Fixation Prediction Based On Convolutional Neural Network

People usually give priority attention to the most interest area when they face the scene within the scope of complex visual area,the visual attention system of human brain can quickly select the fixation from the complex scene.The fixation prediction system simulates the human visual system through a certain algorithm to judge the probability that each pixel in the image is fixed by the human eye.Fixation prediction can be widely used in the preprocessing of complex tasks such as object recognition,video compression,visual tracking,image segmentation and so on.In recent years,deep learning has achieved good results in the field of target detection and image classification.It is proved that deep learning is an effective tool and method in the field of computer vision.The strong presentation ability of deep learning framework and the ability to extract complex features greatly promote the development of gaze detection.In this paper we summarize the predecessors of fixation prediction scientific researches and combine with state-of-the-art deep artificial neural research and technologies,we propose two method of fixation prediction based on local evaluation and global optimization and fixation prediction based on two-stream convolutional neural network.The former uses the existing neural network to calculate the deep feature as the evaluation feature of the images.The latter is a special fixation prediction network.The fixation prediction method based on local evaluation and global optimization is divided into two parts: local evaluation and global optimization.There are two ways to local evaluation: general learning and specific learning.Global learning refers to train evaluation classifier over the entire training set.Specific learning utilizing the semi-coupled dictionary learning algorithm,for each test image,we can reconstruct a specific evaluation classifier.Global optimization stage includes proposal subset clustering optimization and global context optimization.The sub cluster optimization can reduce the outliers,increase the confidence of the local evaluation,and it is easy to design a unified feature to describe the whole subset.The global context optimization uses the image context information to optimize the prediction results of the fixation on the global scale,and output the final fixation maps.The fixation prediction algorithm based on two-stream convolutional neural network is an end to end convolutional neural network model.One stream is used to predict eye fixations and the other stream is used to classify scene category.Multi-task learning is exploited to learn the relationship between fixation prediction and scene classification.We train the two streams simultaneously,so that we can get a representation that is helpful for both tasks.In addition,we design an adaptive fusion mechanism to integrate semantic information,calculated contrast and center priors.The adaptive fusion mechanism utilizes the features of the classification stream to judge whether contrast maps or center priors should be integrated.We test the neural network algorithm on MIT300,MIT1003,CAT2000 datasets,and test the local evaluation and global optimization algorithm on MIT300 and MIT1003 datasets.The local and global optimization algorithms have achieved good performance compare with the non neural network models.The two-stream convolutional neural network model outperforms other state-of-the-art methods widely,which proves that our model is effective.