The Research Of Image Emotion Analysis Based On Attribute Learning

With the prosperity of social media and the popularity of mobile terminals with camera functions,image data constantly flock into the network,which has become one of the main media for users to express emotion feelings.In this context,as a supplement to textual sentiment analysis,image emotion analysis has become a new research focus in the field of image semantic understanding.The research of image emotion analysis is challenging.How to bridge the gap between low-level visual features and high-level emotion is the key to image emotion analysis.Most of the existing studies utilize deep learning techniques to extract discriminative emotional features in data-driven ways.However,such methods require the support of large-scale and little-noise training data,while ignoring the correlation between mid-level semantic concepts and emotion and lack the interpretability.Aiming at the problems existing in current methods for image emotion analysis,this thesis proposes the concept of emotional attributes and constructs an image emotion analysis method based on attribute learning,which takes attribute mining,attribute prediction and attribute modeling in visual attribute learning as the basic research route.The main research contents and contributions are as follows:(1)A method of emotional attribute mining based on user metadata information is proposed in this thesis.Aiming at the problems of weak discriminative ability and limited semantic coverage of semantic concept sets for current image emotion analysis,an emotional attribute selection strategy is presented,which effectively utilizes user metadata to mine emotional attribute consistent with human emotional cognition.This method first defines the characteristics of emotional attributes,and designs a quantitative calculation method for each characteristic based on related theories.Secondly,a concept selection model is constructed to select the emotional attribute set from the user-generated tags through the constrained quadratic linear optimization method.Experimental results show that the emotional attribute selection method proposed in this paper can obtain a set of semantic concepts conforming to the definition of emotional attributes,and is more effective in image emotion analysis tasks than the semantic concept set used in previous methods.(2)A method for image label prediction based on fused neural network with matrix factorization is proposed in this thesis.Aiming at the incorrect and incomplete user metadata tags,a fused neural network with matrix factorization model is presented,which realizes the image label prediction based on user annotation and provides a semantic clue for image emotion analysis.Based on matrix factorization algorithm for label prediction,the visual and semantic factor matrix are obtained by integrating the convolutional neural network and graph convolution network,which is realized under the dual optimization of feature learning and matrix reconstruction.Moreover,to solve the noise problem,the construction of the latent shared space is constrained under the guidance of prior information in the images and labels in a weakly supervised environment.The experimental results show that the proposed method achieves good performance on the task of image label prediction based on user annotations,and also confirms the effectiveness of each part of the model.(3)A method for image emotion analysis based on joint attribute modeling is proposed in this thesis.Aiming at the disadvantages of emotional attributes such as insufficient semantic coverage,emotional latent attributes complementary to emotional attributes is proposed.Then a dictionary learning framework for joint attribute modeling is presented to construct joint emotional attribute space,within which can obtain mid-level rich and discriminative features to solve semantic gap of image emotion analysis.To ensure the effectiveness of the joint emotional attribute space for image emotion analysis,for one thing,a discriminative dictionary item is constructed to guide the learning of emotional attribute features,using emotional attribute labels as supervised information.The semantic representation ability is further enhanced through feature collaborative learning.For the other,based on laplacian eigenmaps,the geometric structure of the image in the emotion space is used as a constraint to mine the potential attributes of emotion.Experimental results on a number of widely used datasets confirme the superiority of joint sentiment attribute features as mid-level features in image emotion analysis tasks.