Research On Image Memorability Prediction Method Based On Attention Mechanism

With the rapid development of modern information technology and the Internet,people will be exposed to thousands of images through electronic,social and print media.Despite the proliferation of various visual information,people can remember a large number of images that are only seen once,but not all images can be remembered with the same degree.Image memorability is proposed to measure the extent which images can be remembered by people and is of important research value in education,advertising and medicine.How to predict image memorability has become a concern.In early research,manual features and support vector regression were used to predict image memorability.Now,convolutional neural networks are often used to build predictive models to predict image memorability.To overcome the deficiencies of the basic network used by the existing models and the small scale of the image memorability dataset,based on the transfer learning method,the image memorability prediction model RMem Net based on Res Net50 and the image memorability prediction model SRMem Net based on SE-Res Net50 are proposed in this thesis.Both models proposed above transfer the knowledge learned in the image classification task to the image memorability prediction task,which significantly improves the prediction efficacy of the model.However,a single convolutional neural network model cannot fully express the memory related information in the image.To solve this problem,an image memorability prediction model CSRMem Net fused with semantic features,an image memorability prediction model ESRMem Net fused with emotional features,and an image memorability prediction model ECSRMem Net based on multi-feature fusion are proposed in this thesis.They use the SERes Net50-based image memorability prediction model proposed in this thesis to extract deep features of the memory using image description models and text embedding technology to extract semantic features,use an emotional classification network to extract emotional features.After fusing the features separately,different neural networks are trained to map the fused features to the image memorability to achieve prediction.In the end,to verify the feasibility and effectiveness of the multiple models proposed in this thesis,the Spearman rank correlation is used as the evaluation standard for this thesis.In the thesis,experiments are conducted on the RMem Net model and SRMem Net model on the La Mem dataset,and the results are compared with the model without pre-trained and the VGGMem Net and Mem Net models proposed by the predecessors.At the same time,experiments are carried out on the CSRMem Net model,ESRMem Net model and ECSRMem Net model on the La Mem dataset,and the results are compared with the DFO-Ens Mem Net model,EMNet model and AMNet model proposed in other literature.The experimental results show that the image memorability model based on transfer learning and the image memorability model based on multi-feature fusion proposed in this thesis are effective and feasible.