Visual Sentiment Analysis Using Neural Netwrok

With the explosive growth of social media and on-line visual content,understanding the emotions of images and videos has attracted significant attention.In recent years,lots of attentions have been paid to affective image classification.Most works focus on manual designing of features and classifiers for the global images.However,several issues remain in this area.Firstly,most work ignore the fact that the image always evoke the human sentiment by specific regions instead of the global appearance,while such regions usually contain more useful information than the background.Secondly,the emotion label of images has strong ambiguity.Even for the same image,people with different cultural backgrounds may have different emotional response.And thus the traditional single emotion problem cannot describe the image emotion clearly.To solve the problem of uneven distribution of the emotion in image region,we propose the concept of affective region.We define the affective region according to two properties,and design a system using deep framework to automatically detect the affective region.In addition,to verify the importance of the affective region,we enhance the emotional representation of images using the affective region,and outperform the state-of-the-art methods on several datasets.For the label ambiguity,we employ the label distribution learning paradigm.Since the existing datasets do not provide the ground truth distribution,we collect two large scale distribution datasets.We encode the label inputs of the conditional probability neural network model(CPNN)to learn the label information,which can effectively solve the problem of the mismatch between features and labels.In the light of the fuzziness of the emotion image labels,we enhance the distribution of the emotion labels of each image,so as to alleviate the problem that the neural network needs lots of strong labeled data during training.The effectiveness of the proposed method is assessed on the public available datasets as well as the collected distribution datasets.