Research On Image Memorability Prediction Method Based On Emotion Analysis

With the development of the Internet industry and the needs of various industries,more and more images that play the role of information transmission and publicity and education appear in people's vision.However,the degree to which images can be remembered varies.In the face of such a flood of visual information,how to choose images that can be remembered by human beings as illustrations for publicity and education is a problem that computer vision needs to solve.The prediction of image memorability can well predict the probability that the image can be remembered.This prediction method is applied to the selection of advertising,educational illustrations and photographic works,which provides convenience for the work of the staff.With the development of Deep Learning,more and more scholars use the Convolution Neural Network to extract image features.This method of extracting deep features has advantages in representing image and extracting methods.However,the prediction effects of existing image memorability prediction models are not very close to human consistency,and there is the problem of incomplete expression of image features.In view of the existing problems of the existing image memorability prediction model and considering the emotional factors of image expression,and inspired by Celikkale's model of fusing two effective features of image to predict image memorability.In this paper,an image memorability prediction model based on emotion analysis is proposed.The model consists of three parts: the first part is to get the image caption of the image and extract the emotional characteristics of the image through the image caption emotional tendency and emotional intensity algorithm.The second part is to extract the deep feature of the image through the deep network.The third part is to fuse the two image features and map them into image memorability through Support Vector Regression.Finally,in order to verify the validity of the proposed model.Firstly,in the Emotion image data set,we use the image caption emotional tendency and emotional tendency intensity algorithm to verify the accuracy of image emotion classification.In addition,we use La Mem image data set to carry out the prediction model with emotional characteristics and without emotional characteristics.The correlation between the experimental data and the ground truth in the data set is calculated by Spearman,and the two experimental results are compared and analyzed.Then,the image memorability prediction model proposed in this paper is tested on Sun and La Mem datasets respectively,and the prediction effect of the model is compared with that of the existing image memorability prediction model.The analysis of the experimental results shows that the algorithm proposed in this paper is effective.At the same time,image emotion as a feature of image memorability prediction model is effective.The prediction model of image memorability based on emotion analysis is effective and feasible,and the prediction performance is improved to a certain extent by the fusion of the two features.