Research On User Interest Modeling Based On Multimodal Data

With the popularity of mobile devices and the continuous development and breakthrough of Internet technologies,the amount of data has grown exponentially.The entire Internet communication data is no longer a single text data,but coexistence of diverse forms of data,such as: images,audio and video.etc.The essence of the Internet is sharing,interaction,virtual,and service.In fact,these four items are inseparable from social functions.The essence behind it is "the flow of data." The Internet allow s data to flow freely between people and devices,connecting the world through data.This is a revolutionary change brought about by the Internet,which allows humans to connect,share information,find interest,and benefit together.Internet data is gene rated by humans,and these data are destined to be consumed by humans,because these multimodal data generated by humans often have a lot of interesting content or useful knowledge,which makes the availability and effectiveness of data mining in academia and industry.The industry has been extensively researched and recognized.Social media can generate a large amount of text and image data.By mining the multi-modal data of social platforms,it not only can well understand the user's interest or intention,but also plays a very important role in the semantic expression and fusion of multi-modal data.The important role,this is the focus of this paper.According to the characteristics of close semantic association between multi-modal data,this paper mainl y uses the concept of interest in social media user behavior data as the basis,and analyzes the user's text and image data,combined with deep learning method to multi-modality.The semantic features of the data are fused and expressed,and the multi-modal data fusion is deeply studied.The research work of the text mainly consists of six chapters:Chapter 1 Introduction: Studying the status quo and development trend of social media user interest modeling and user interest recognition research at home and abroad,mainly understanding the uniqueness of data types in social media user interest modeling at home and abroad,and discussing the background of the topic and the meaning of the topic are presented.The research methods,research ideas,research conte nts and research innovations are proposed for the background of the topic and the meaning of the topic.Chapter 2 Overview of Relevant Theories and Work: explain the related concepts and information dissemination methods of social media,analyze the relevance of domestic social media Microblog user images and text data,and collect and clean the work,mainly on Microblog.user data and image data are used for user interest modeling definition,model construction and user interest identification.Chapter 3 Based on the integration of Bi-RNN and attention mechanism.User interest modeling: A new Microblog short text training model is proposed for user blog posts in social media.In this chapter,the semantic information of Microblog user interest tags is introduced in the case of sparse and insufficient corpus of traditional traditional user Microblog texts,so that the blog vector model can obtain additional co-occurrence information,so that the whole word vector model can better identify semantics.The assoc iated words make the semantic words more similar in the vector space,and the interpretability is better.The bidirectional cyclic neural network is used to extract the high-level features and introduce the attention mechanism.This method has better text semantic feature extraction and expression ability,which is very good.construct and identify the user interest model of the Microblog text,and classify the users.Chapter 4 Based on CNN Microblog image user interest modeling: a user interest classification model based on convolutional neural network for single-mode and single-mode combined image fusion is proposed.A single-mode combined image detection method is constructed for single-mode images and single-mode combined images shared by Microblog users in social media.Then,by using convolutional neural network and pooling method to obtain image feature representation,the pooled matrix is tiled as the global representation of the image,which is more helpful for image user interest classification in Microblog.Chapter 5 Based on multimodal data fusion for Microblog user interest modeling: a multi-modal data-based Microblog user interest classification model is proposed.Through the fusion of text and image semantics,users can be well discovered or predicted.Interest categories.This method mainly uses the expression of the features of the image to calculate the inner product of all hidden states of the bidirectional cyclic neural network in the text,and then connects the softmax activation function to obtain the distribution of each hidden state,using all hidden state weighted summation,and finally the results are weighted and spliced for classification,and the performance of the user interest classification can be improved.Chapter 6 Research Summary and Prospects: summarize and forecast the content of the full-text research,summarize the research results and propose relevant conclusions of the research,analyze the research deficiencies,and propose future feasibility study suggestions and fu rther research directions.