Implementation And Research On Key Issues Of Text-image Matching By Combing Content And Emotion Consistence

With the rapid development of Internet and mobile devices,social network users have gradually evolved from the way of passive consuming content to the way that of actively creating and sharing content.At present,the combination of text and images is the popular presentation way on social network.It is a challenging task to quickly find images suitable for user generated content from large scale images on social network.This thesis conducted a related research on the Sina Microblog and Huaban social network.The content and emotion expression of Microblog text generated by users is analyzed,so as to search suitable images for users.The details are as follows:1.A large scale image dataset based on Chinese Emotion Word Ontology for emotion analysis is established.Firstly,using Chinese Emotion Word Ontology as keywords,we crawl images with Chinese information in Flickr.Then,a method of cleaning datasets is designed,which employs polar conflicts of emotional keywords,description text and tags to filter noisy images.2.A method of image sentiment analysis by combining local and global information is proposed.First,it is judged whether there are salient objects in an image or not.If there are,the detection window of the salient objects are adjusted based on the designed rules,and then local regions are extracted.The sentiment of local region is analyzed by the model trained on all local regions of training set.Predictions of sentiments from entire images and sub-images are then fused together to obtain the final results.If no salient object is detected in the images,sentiment predicted directly from entire images is used as the final result.The proposed method combines global and local information effectively,and improves the performance of emotion analysis.3.A text-image matching system by fusing the emotion and content similarity is developed.The system is developed on Django.It has the interface with Sina Microblog Platform.The content and emotion of the text published by the user is automatically analyzed.Then the candidate images are searched based on the emotion and content similarity.Finally,the top N images are recommended to the users.The text and the image user selected are published on the Sina Microblog simultaneously.