Image Sentiment Analysis Based On Sample Refinement And BERT Guided

Image sentiment analysis research has great economic value,social value and research value,so it is also a hot issue in the field of machine vision.Existing research faces the problem of sample scarcity,and does not make full use of the cross-modal sentimental semantics contained in multi-modal features,the complementarity of different features in decision-making,and the integration of multiple sets of prediction results.To this end,an image sentiment analysis model based on sample refinementand BERT guided is proposed,which includes methods such as sample refinement,cross-modal sentimental semantic mining,and two-stage decision model.The main contents are as follows:(1)Image sentiment analysis model based on sample refinement and cross-modal semantic mining: Aiming at the problem of scarcity of high-quality samples,the sample refinement model is designed to expand high-quality emotional samples on the basis of existing dat sets.Sift,Lbp,Gist,pre-trained VGG models are used to extract image features,and the cross-modal semantic mining algorithms is used for feature fusion,and finally a group ofclassifiers are utilized for emotion prediction.The experimental results show that the sample refinement model and the cross-modal semantic mining algorithmare both important and effective for image sentiment analysis.Among them,on the Twitter 1dataset,the Sift feature obtains the best performance due to its rich emotional semantics,on the FI dataset,the VGG feature achieves the best performance,and the Boosting-based classifiersareeffective for image sentiment analysis.(2)Image sentiment analysis model based on sample refinement by BERT features and image caption: The existing image sentiment analysis tasks mainly focus on the image modality,while the text modalityis rarely studied.To address this problem,we propose our model: the image captioning model is firstly used to generate the corresponding text description of the image.On this basis,text features are obtained through the pre-trained BERT model,and combined with the sample refinement model,could Multi-angle and in-depth mining of high-quality samples,and training of image sentiment analysis models based on BERT features.The experimental results show that the text features generated based on image descriptions have good predict accuracy.These BERT features aslohave good emotion expression abilities.However,simple text description is still insufficient for image sentiment analysis,and the comprehensive utilization of both image and text is the next key issue.(3)Multi-modal image sentiment analysis model guided by the BERT model: In order to make full use of the complementarity between image and text modalities,comprehensive refinement of samples is completed by optimizing BERT features,and then a multi-modal two-stage decision model is designed to complete image sentiment analysis.The experimental results show that the BERT feature can refined high-quality image samples;the multi-modal two-stage decision model improves the accuracy of the model.The model has high scalability.The main innovations are as follows:(1)Feature fusion through cross-modal semantic mining;(2)Extract text features and combine images for sentiment analysis;(3)Comprehensive selection of image and text modalities to improve the quality of the dataset.