Research On The Distributed Representation Learning Of Knowledge Graphs

With the advent of the era of cognitive intelligence,more and more intelligent applications are more concerned with the interpretability of results and the knowledge embedded in the data.To this end,knowledge graphs,with the large-scale and automated knowledge acquisition features,and interpretable and inferable knowledge application features,are widely used in scenarios such as smart search,question answering and personalized recommendations.However,traditional discrete symbolic knowledge representation methods are limited by the large-scale requirements of modern knowledge graphs,semantic relevance metrics and deep learning applications.In recent years,inspired by the distributed hypothesis in the field of natural language processing,the distributed representation of knowledge graphs is proposed to solve the above problems:embedding the corresponding semantic information into a dense and continuous vector space by representing entities and relationships as low-dimensional numerical vectors.It has the advantages of efficiently implementing semantic relevance computation,easily capturing tacit knowledge,and easily integrating with deep models.It has the advantages of efficient semantic relevance computation,easy to capture of implicit knowledge and easy to integrate with deep learning models.Therefore,the distributed representation learning of knowledge graphs has gradually become a popular research topic in the field of knowledge graphs at this stage.Inspired by the distributed hypothesis,distributed representation of knowledge graphs considers that the semantics of entities and relationships in knowledge graphs depends on their distribution.However,the data sparsity problem prevalent in knowledge graphs due to the"incompleteness" of the data prevents existing methods from learning the representation of long-tailed entities more accurately,and is often less accurate for knowledge inference related tasks.In addition,the large-scale requirements of modern knowledge graphs make existing methods still largely rely on the weak logical constraints,which cannot adequately represent the complex knowledge structures such as network and dynamic structures in knowledge graphs,brings challenges to distributed representation of knowledge graphs.To this end,after in-depth research and analysis,this paper proposes the following three innovative research works in the field of knowledge graph representation learning of knowledge graphs:(1)To address the problem of data sparsity,this paper proposes a distributed representation learning method of knowledge graphs that incorporating entity description information as an adjunct and supplement to the existing structured knowledge information,providing more in-depth descriptions and mining missed knowledge,and then enhancing the representation of long-tail entities.First of all,to address the problem of insufficient entity-related semantic extraction,a model for encoding entity description information based on hierarchical bidirectional long and shortterm memory networks and pre-trained language representations is proposed to effectively extract the rich semantic information contained in entity descriptions.Secondly,to address the fusion problem of text and knowledge space,a knowledge constraint and alignment method is proposed,which can simultaneously learn entity representations from structural triples and entity descriptions,and achieve interactive fusion alignment of text and knowledge space.The experimental results on the knowledge graph complementation task(all and long-tail case data)fully demonstrate that the above approach can help with better knowledge graph representations and enhance the representation of long-tail data.(2)To address the problem of network structures of knowledge,this paper proposes a distributed representation learning method of knowledge graphs based on graph neighbourhood structure information,which can portray the semantic network distribution of entities and relations richly from another perspective by considering the multi-step relational paths,node neighborhoods,and other graph structure features embedded in knowledge graphs.First of all,to address the problem of semantic extraction of multi-step relational paths,a multi-step relational path encoding model based on the local and global attention is proposed,that can consider the local and global attention of multiple paths between entity pairs on the basis of single path encoding.Secondly,to address the problem of neighbourhood information aggregation,a hierarchical structure model based on graph attention networks is proposed:neighbourhood-level attention→layer-level attention,which can effectively aggregates direct and multi-step neighbors surrounding entities.The experimental results on the knowledge graph complementation task(entity and relationship prediction)fully demonstrate that the above approach can effectively model the distribution of entities and relationships by integrating the graph structure information,and enhance the knowledge inference capability of the distributed representation of the knowledge graph.(3)To address the problem of dynamic structures of knowledge,this paper proposes a distributed representation learning method of knowledge graphs based on time-series historical memory,and solve the problem brought by the non-linear evolution and uncertainty of entities and relationships over time development to the knowledge graph representation from the perspective of dual-process theory.First of all,this paper proposes the importance of clue information based on direct historical clues and correlated historical clues as a predictor of future facts.Secondly,this paper proposes a filter function that can consider both the repetitions of historical cues and the timeline trend to achieve more detailed replication inference.Finally,this paper proposes an improved self-attentive generation mechanism that can prompt the model to focus on the entities that are more relevant to the event to be predicted,giving the model the ability of predicting from scratch.The experimental results on the temporal knowledge graph complementation task(unknown fact inference)provide ample evidence that the above approach can effectively enhance the knowledge inference capability of distributed representations of temporal knowledge graphs.