Research On Entity Relation Extraction Based On Multi-Granularity Attention Mechanism

With the advent of the 5G era and the continuous advancement of Internet technology,the amount of information generated by the Internet every year is increasing exponentially.However,most of the massive data exists in the form of unstructured data,so that the computer cannot directly obtain the structured information that people need from the unstructured data.With the application and development of deep learning technology,the task of entity relation extraction has been transformed into a computer learning the characteristics of relational instances by itself,and extracting structured information from massive data in a way that computers can understand and recognize.For deep learning models,a large number of labeled datasets are required for iterative optimization of parameters.In order to save the labor cost of manually labeling datasets,the research adopts Distant Supervision(DS)to obtain the labeled datasets,that is,annotated datasets are obtained by aligning knowledge bases and unstructured external documents.In order to eliminate the noise introduced in the distant supervision method,Multi-instance Learning(MIL)is used in the research to train the model,and at the same time,it combines various granularity-level attention mechanisms to extract more important information.First,a relation extraction model that uses Convolutional Neural Network(PCNN)and Bi-directional Long Short Term Memory(Bi LSTM)to jointly encode the input sequence is proposed,which solves the problem of the limitations of traditional neural network extraction features and long-term dependencies within sequences.In addition,in the process of preprocessing the input sequence,the feature information of the position embedding is added to highlight the importance of each word in the instance relative to the entity pair.Secondly,in order to eliminate the noise existing in distant supervision,on the basis of multi-instance learning,CNN in the model is combined with the bag-level selection attention mechanism and the sentence-level multi-head attention mechanism;Bi LSTM is combined with sentence-level word attention mechanism.Finally,the feature vectors of the overall model are represented by assigning different weight coefficients to the feature information extracted by the three modules.Finally,the proportional relationship between the parameters of each module of the model is determined by the parameter verification experiment;the key role of attention mechanisms at various granularity levels in eliminating noise in distant supervision is verified through horizontal comparison experiment;and on the distant supervision dataset NYT10,multiple entity relation extraction methods based on distant supervision methods are selected as baseline methods for comparative experiments.The AUC index of the model proposed in this paper reaches 0.461,which is 0.121 higher than the earliest model combining the attention mechanism with the convolutional neural network.