Multi-view Learning For Web Image Sentiment Analysis

With the development of IT, more and more images were distributed in Internet. These images transmit some information in some aspect, such as news media, personal status and feeling, etc. What's more, Internet becomes an important platform of expressing emotions and opinions with the rise of social media. Research of minning emotion trends from Internet is important to social public opinion analysis, product design and event propagation. As image is the indispensable part of message of Internet, web image sentiment analysis become necessary. In view of the rich emotion expression in human face, one possible framwork to analysis web image sentiment is to divide iamge into two parts:including facial and without facial. The images in web have no tags mostly, if we want to analysis personal emotion with facial images, we need the facial pictures of that people. So the first issue is to cluster images with facial. As the normal pictures, we need to learn an effective image sentiment classifier. In this paper, we focus on these two problems:image clustering and image sentiment classification.In traditional image clustering and sentiment learning methods, single-view feature was used only. But the different the views describe, the different features are. With the development of computer vision, more and more feature extraction method proposed, such as low-level visual fetures, mid-level semantic features or deep learning. These views describe the same and complementary information, which would be good for describing the content of images. We mainly research the multi-view learning framework for image clustering and sentiment classification. Our contributions are threefold:(1) Inspired with the manifold learning, we proposed a multi-view learning framework with local graph regularization base on Non-negative Matrix Factorization (NMF). The inter-view consistencies were mostly considered in multi-view learning methods, but the inner-view local structures are also important to feature learning. In this paper, we introduced local graph regularization in multi-view NMF framework. We named this new algorathms MultiGNMF, and designed experiments of image clustering with it.(2) We generalize the MultiGNMF method to deal with features with negative numbers, named MultiGSemiNMF. The NMF-based methods can not handle matrix with negative numbers, which are nomal in image features such as DCT. Combined with SemiNMF method and MultiGNMF framework, we proposed a general multi-view feature learning method. We designed experiment in public datasets:UCI Digital and CMU PIE, the clustering results demonstrated the effectiveness of our algorithms.(3) We proposed a multi-view feature selection method to learn image sentiment classification better. As the abstraction of sentiment, there is a sentiment gap between feature and sentiment. We combined mid-level and aesthetic features with low-level features to learning classification. To deal with high dimensional challenges, the experiment on two public datasets:Artphoto and Abstract demonstrated that the feature selection method would improve learning performance.