Semantic Relation Classification Algorithm Based On Hierarchical Recurrent Neural Network

Relation classification is one of the important methods to realize text structure in the field of information extraction,and it has been widely used in many natural language processing tasks such as machine translation,information retrieval,automatic question and answer,and knowledge base construction.Breakthroughs have been made in many tasks in the field of natural language processing with deep learning,people began to explore the combination of deep learning and relation classification,which has become one of the research hotspots.The existing relation classification based on deep learning construct network model on two-stage tasks of relation classification respectively,including entity recognition and semantic relation classification.However,this method exists the following problems:(1)The existing method adopts the step-by-step training method in two stages,so the errors generated in the entity recognition stage will be propagated to the semantic relation classification stage,causing the problem of error accumulation,thereby affecting the final classification effect;(2)There are structural differences in different syntactic structures.Due to the problem of poor robustness,network models based on specific syntactic structures cannot be directly used on other structures.Regarding the problems above,we propose a semantic relation classification algorithm based on hierarchical recurrent neural network.The model framework of the algorithm is divided into two layers with different network,the sequence prediction layer and the relation prediction layer,and the two layers are nested for end-to-end training.Main tasks as follows:(1)Sequence prediction layer: We construct a network of Bi-LSTM-CRF fusion attention mechanism.Firstly,Bi-LSTM is used to encode the bidirectional hidden state information of the sequence.Secondly,it is re-learned under the attention mechanism to adjust the weight distribution.Finally,the CRF layer decodes the information based on the BILOU annotation mode to predict the entity label information.This layer is designed to enhance the model's attention to the key information on the sequence,and make full use of the context information for decoding,thus improving the entity recognition effect.(2)Relation prediction layer: We construct a Bi-Tree-LSTM network with multiple syntactic structures.By weighting the Fulltree,Subtree and SDP structures into the same network,the layer fully learns the structural information in the topdown and bottom-up directions respectively and obtains the candidate relation of the tri-tuple structure information.Finally,the category label of candidate relation is predicted by the SoftMax classifier.This layer aims to enhance the robustness of the model by weighting multiple syntactic structures in the same network.(3)By nesting the relation prediction layer on the sequence prediction layer and using the shared parameters to perform end-to-end training,the two phases promote each other,which improves the classification effect of the whole model.Experiments on the SemEval-2010 Tak8 dataset show that our proposed algorithm achieves a state-of-the-art result which the F1 value is 86.3.At the same time,the experimental results of the sequence prediction layer and the relation prediction layer are analyzed,which the effectiveness and robustness of our algorithm are verified.