A Research On Knowledge Representation Learning Of Joint Text Based On Deep Learning

Knowledge representation is designed to transform knowledge into a form of data that computers can interact with.Although the method of knowledge representation based on the network is widely used,its computational complexity is too high,and it is unable to cope with the sparse data in the knowledge base.Representation learning based on deep learning technology has been paid lots of attention due to its excellent performance.Knowledge representation learning can encode the entity or the relationship into the low-dimensional continuous dense vector and has high application value in many fields such as knowledge reasoning and knowledge fusion.TransE is the most popular method among the knowledge representation learning models.It considers the relationship vector in the knowledge base as the translation from the head entity vector to the tail entity vector.Compared with other models,TransE has simple structure which is easy to expand and high computational efficiency.And it still shows good performance on large sparse knowledge base.However,it performs poorly in the face of the complex relationship,thus a large number of extended models have been proposed to deal with this situation.TransE and its extended models only use structured knowledge and cannot represent new entities outside the knowledge base.In order to solve this problem,some models introduce the description text of the entity to supplement the missing entity representation.However,these models do not filter text information,and have not proposed a good method to combine the two representations.The knowledge representation method of encoding text based on bidirectional LSTM solves the above problems and achieves the best effect in the related evaluation tasks.However,LSTM needs to generate hidden variables in order,which makes the model unable to train in parallel and affects the computational efficiency of the model.To solve the above problems,we study the related deep learning techniques including word embedding,the convolutional neural network,the attention mechanism and so on,and then propose the knowledge representation model named JCNN which joints text and structured information,and propose improved models including JCNNAttpool and JCNN-Attweight.Specifically,JCNN uses the convolutional neural network to encode entity description information,and uses TransE to encode the structured information of the entity,and then uses a weighted gate mechanism to fuse the two types of information.The training process uses the stochastic gradient descent to update the parameters of the models,and finally we obtain the learned vector representation of entities and relationships.Considering that not all textual information is useful,we introduce the attention mechanism to select the text's information which is most relevant to the triple' relationship.Both JCNN-Attpool and JCNN-Attweight are improved models based on JCNN.JCNN-Attpool replaces JCNN's largest pooling layer with the attention pooling layer,and JCNN-Attweight adds the attention weight layer after JCNN's convolutional layer.Experiments on the FB15 k and WN18 datasets show that our models have advantages in some ways including predicting missing entities,coping with complex relationships and classifying positive and negative samples.This shows that the use of deep learning techniques to encode entity descriptions,and the fusion of text representation with knowledge base representation can significantly help entity representation enhance the difference.In addition,it is further confirmed that the text encoder we propose can effectively capture the semantic information of the text;The attention mechanism we design can extract the key features;The fusion method we use has certain advantages in fitting the two information sources.