Research On Semantic Role Labeling Based On Deep Neural Network

Semantic analysis is an important means to understand unstructured text,which is of great significance to the application of natural language processing.Semantic role labeling is an important tool for text semantic analysis,and as a shallow analysis technology in semantic analysis,semantic role labeling has always been a research hotspot in the field of natural language processing.The main goal of semantic role labeling is to recognize the “predicate-argument” structure in each sentence and assign semantic roles to all arguments in the event the predicate refers to.Previous researches on semantic role labeling are mostly based on traditional machine learning algorithms,which rely on the results of syntactic analysis,and then select effective features and feature combinations.Feature engineering is a time-consuming and laborious task,and the error information of syntactic analyzer will affect the performance of semantic role labeling model.With the strong rise of deep neural network,more and more researchers begin to use deep neural network to complete the task of semantic role labeling,and reduce the dependence on artificial features by automatically extracting features from models.Although many existing neural network models have achieved good results in semantic role labeling,there are still some problems in these models: firstly,how to make better use of the semantic dependence between part-of-speech and semantic roles;Secondly,how to effectively add the dependency syntactic information in the dependency tree to the task of semantic role labeling;Thirdly,previous works have neglected that selective preference information can provide rich semantic information for semantic role labeling.Aiming at the problem of "how to make better use of the semantic dependence between part-of-speech and semantic role",we propose the auxiliary deep neural network model,which can jointly mark part-ofspeech and semantic role.First,the model uses Bi-directional Long short-term memory network to encode sentence to obtain context representation.The alternation training of two annotation tasks enables the effective feature information to be shared through the network structure of hard parameter sharing;then,in the training process of the main task-semantic role labeling,the argument recognition layer is introduced,which helps the predicate select the information related to the argument.Finally,the model was trained and adjusted by using the shared dataset Co NLL2005,and the experimental results were compared with the existing excellent models.The experimental results show that the model achieves the F1 value of 89.0% on the test set WSJ,which is 0.8% higher than the existing best model,and effectively models the semantic dependence between part-of-speech and semantic roles.Aiming at the problem of "how to effectively add the dependency syntactic information in the dependency tree to the semantic role labeling",we propose a Bi-directional graph convolution network model based on attention mechanism.Firstly,the part-of-speech and word vectors are spliced as the input of Bidirectional Long short-term memory network,so the part-of-speech features are added to the feature representation of words;Then,attention mechanism is used to select the dependency between predicate and argument;Finally,the selected feature information and the direction matrix of the syntax dependent edge are input into the Bi-directional graph convolution network layer for secondary encoding.The top layer of the model uses the conditional random field to get the optimal labeling sequence.The model also uses the shared dataset Co NLL2005 for training and prediction.And the F1 values of 88.9% and 79.5% in WSJ and Brown test sets are obtained respectively,which are superior to the best existing model.Compared with the above excellent models,this model effectively utilizes the rich dependency syntactic information in the dependency tree,and uses the attention mechanism to emphasize the important role of predicates in semantic role labeling.Aiming at the problem of "ignoring the selective preference information can provide rich semantic information for semantic role labeling",a Transformer model based on predicate-argument selective preference is proposed.First,the model uses a sentence encoder composed of multi-layer stacked transformer to encode the vocabulary and semantic information in the sentence;Then,the selective preference encoder composed of multi-layer integrators integrates the predicate-subject and predicate-object selective preference information in SP-10 k dataset into the embedding representation of sentences;Finally,the embedding representations are input into the Soft Max layer after linear processing to complete the role classification.In this model,F1 values of 89.9% and 80.4% are obtained on the test set WSJ and Brown respectively,which are improved by 1.7% and 1.1% compared with the existing best results.The experimental results show that transformer can obtain the dependency and semantic information between words without considering the distance between words,and then the token embedding contains more context information;This model also integrates the selective preference information into the embedding representation,focusing on the selective preference between predicate-subject and predicate-object,and uses more selective preference relations to assist the semantic role labeling,thus effectively utilizing the rich semantic information in selective preference.The auxiliary deep neural network model based on the joint labeling of part-of-speech and semantic role is used to model the semantic dependency between part-of-speech and semantic role,the Bi-directional graph convolution network model based on attention mechanism is used to make better use of the dependency syntactic information in the dependency tree,and the transformer model based on predicate-argument selective preference is used to make better use of the semantic information in selective preference information.The above three models are used to make up the shortcomings of the existing semantic role labeling models,which respectively correspond to three application scenarios with part-of-speech tags,dependency syntactic analysis results or rich selective preference information.