Knowledge Embedding Based Topic Model For Multi-modal Social Event Analysis

With the development of mobile Internet and social networks,people's lifestyles are also changing.Many social networking sites(such as Facebook,YouTube and WeChat)have sprung up,which leads to the emergence of a large number of multimedia data(such as text,image,and video)of various events.And along with the accumulation of data on the Internet and the progress of representation learning techniques,the knowledge embedding learned from large-scale knowledge base has also been used for probabilistic topic models.The goal of this thesis is to efficiently mine event topics from a large number of unordered social media data,which is beneficial to search,browse and monitor significant social events for users or governments.However,it is very challenging to learn interpretable topics and discriminative event representations based on multi-modal information.(1)Firstly,this thesis proposes a knowledge-based multi-modal weighted topic model(KBMMWTM)for social event analysis.The proposed KBMMWTM model has the following advantages.1)The proposed KBMMWTM model can effectively utilize the multimodality of social event data.2)The proposed KBMMWTM model can improve the performance of event topic mining by taking the word correlation in the dataset as prior knowledge.Finally,we evaluate our KBMMWTM model on the real dataset.The complete experiments demonstrate that our model outperforms the state-of-the-art models.(2)Then,we present a knowledge embedding based topic model for multi-modal social event analysis,called KE-MMTM.Compared with other existing methods,our work has three main advantages.1)Our model can integrate the additional knowledge graph embedding as prior knowledge into a unified topic model in which knowledge embedding,max-margin classifier and multi-modal information are exploited to get more event description.2)We use the WN18 knowledge base(which contains 151,442 triplets with 40,943 entities and 18 relations)to learn the knowledge embedding vectors,and then incorporate the multi-modal data with a prior knowledge encoded by these entity vectors into the topic model to learn more coherent topics.3)For the purpose of event topic mining and classification research,this thesis collects and publicly publishes a large-scale multi-modal dataset(including 10 events,and each event contains about 7000 documents).Extensive experiments are conducted to demonstrate that the proposed model outperforms(with 83.3% classification accuracy)the state-of-the-art models in topic coherence.