| Relation Classification(also known as Semantic Relation Classification)has become a task that has attracted a large number of researchers in recent years.It is not only play an important role in the construction of Knowledge Graph,but it is also of great benefit to many other tasks in the field of Natural Language Processing such as Automatic Question Answering,Information Extraction,Ontology Learning and so on.In recent years,models proposed by most scholars for relational classification tasks are based on deep learning models.The Att-BLSTM model is a representative work based on Long-Short Term Memory(LSTM)Recurrent Neural Networks(RNN)that incorporates Attention Mechanism(Att).The model makes use of the ability of LSTM-RNN to extract temporal features and long-distance dependence information in semantic relational sentences.It also uses Attention Mechanisms to increase the degree of attention of the model to the more relevant parts of the final result of the semantic relationship in the sentence.Finally,the model achieved an excellent result without using any artificial features.However,the Att-BLSTM model in some cases makes similar vocabulary expressions at different times in the same sentence get close attention weights,although the degree of correlation to the final classification results between these similar vocabulary expressions is not the same.This is mainly because the Att-BLSTM model calculates the degree of attention only through the vocabulary itself at every moment in the sentence.And in semantic relationship classification,two entities with a semantic relationship in a sentence usually have one active entity and the other passive entity.That is,the semantic relationship has directionality.Considering above issues,this article has improved on the basis of the Att-BLSTM model,which mainly includes the following works:1.We improve the attention layer in the Att-BLSTM model as follows:1)Change the module for calculating the attention weight in the attentional layer to a feedforward neural network with a hidden layer.2)Let the attention level use the vocabulary of each moment in the sentence and the embedded representation of each semantic relation to calculate the attention weight distribution at that moment.2.We added a relationship and direction representation layer to the Att-BLSTM model to learn the sentence's global semantic relation representation and relational directionality representation.The approach is as follows:1)Design the relation and direction representation presentation layer as a bidirectional LSTM-RNN layer.2)Connected the forward and backward final cell states of LSTM-RNN,the connected bi-directional cell states are representation as relationships and directions.3)Incorporate the relation and direction representation with the sentence representation of the original model to classify the semantic relations.3.In order to verify the above two improvements for Att-BLSTM model,we carried out several comparative experiments on the standard dataset semEval dataset:1)Comparison of the Att-BLSTM model with the modified impAtt-BLSTM model.2)The Att-BLSTM model is compared with the Att-Dir-BLSTM model to which the relationship and direction presentation layers are added.3)Comparison of the impAtt-BLSTM model with the Att-Dir-BLSTM model.4)Comparison of the Att-BLSTM model with the impAtt-Dir-BLSTM model after comprehensive improvement.The experimental results show that the performance of the model with improved attention layer and the model with additional relational and directional presentation layer have achieved better results compare to the original model,and the performance of the all-improved-merged model is further improved.At the same time,in order to further improve the performance of the model,we also conducted an anti-overfitting experiment and a comparative experiment to replace the LSTM hidden layer unit.Finally,in order to test the generalization performance of the model,comparativeexperiments were also conducted on the KBP37 data set. |
PDF Full Text Request