Multi-Modal User Portrait Analysis System

With the booming development and popularity of the Internet,different social media networks are emerging for different needs.Users leave abundant user information and massive structured and unstructured data on different social platforms,which are spontaneous,genuine and diversified.Through the analysis of these multi-source and multi-modal data generated by users,user portraits can be more accurately and comprehensively understood.In our research,we proposed a modeling approach to analyze user's online behavior across different social media platforms.We can deeply mine the joint information,improve the utilization rate of multi-source data,and introduce personality dimension to build precise user portraits.The main work can be listed as follows.Firstly,the accounts of users on Sina Weibo and NetEase Cloud Music have been connected,and the data generated by users on these two platforms are crawled,including structured and unstructured data such as microblog text,personal information,music collection,etc.Then,regular multi-source and multi-modal real social datasets are constructed through text cleaning,tag extraction,preference clustering and other methods.Secondly,based on SC-LIWC dictionary,we extract psychological word frequency features from micro-blog text,establish a user personality mapping matrix and train a CNN linear regression network based on microblog text to calculate user's Big Five personality,so as to get rid of the traditional constraint of obtaining user's personality based on questionnaires.At the same time,the correlations between user personality and user attributes,user behavior and Weibo tags are analyzed to display micro-blog portraits in multi-dimensions.Thirdly,music genre preferences and music mood preferences are calculated based on music tags,and five typical music genre preferences and four typical music mood preferences are obtained with K-means method.Then,linear regression personality computing model based on music tags is trained to analyze the relationship between user's music preference,user's personality and basic information.Thus,the joint modeling of cross-platform and multi-modal social behavior data is completed,and a comprehensive user portrait is obtained.Finally,the multi-modal user portraits are visualized based on Flask framework,which makes the results more intuitive and completes the whole multi-modal user portrait analysis system.