| Named entity recognition is crucial in natural language processing.Currently,deep learning methods are mainly used,such as BiLSTM-CRF model.Although BiLSTM structure in this model is capable of capturing bidirectional long-distance dependency in sentences,following problems exist: 1)Document-level label consistency is an effective indicator that different occurrences of a particular token sequence are very likely to have the same entity types in a document.But BiLSTM-CRF model itself has sequential nature,labeling only at the sentence level.The constraint prevents the full utilization of document-level label consistency.2)Only by simply tuning the hyperparameters or changing the network structure,the performance of the model can easily reach the bottleneck,and the quality of the recognition results cannot be further improved.To deal with above problem,the main work of this thesis is as follows:(1)Aiming at the problem of unable to take full advantage of document-level label consistency,we add key-value memory network(KVMN)structure to the BiLSTM-CRF model and propose BiLSTM-KVMN-CRF model.KVMN is adopted to memorize all the hidden state vectors and their corresponding label embeddings of the entire document,which the BiLSTM networks yield by taking a document as input.Before CRF decoding,using the multi-head attention mechanism to extract the context information and label embeddings of other occurrences of the word in the document,generating document-level context representation and document-level label embedding,and merge it with the hidden state vector of the current word.The experimental results show that,compared with classic BiLSTM-CRF model,the F1 value of BiLSTM-KVMN-CRF model on Co NLL-2003 dataset is 91.48%,increased by 0.28%;on Onto Notes5.0 dataset,the F1 value is 87.42%,increased by 0.43%.In addition,we also conduct ablation experiments to analyze the contribution of the context representation and the label embedding in KVMN to the BiLSTM-KVMN-CRF model.The results show that both can improve the performance of the model.And when both are used at the same time,the improving effect is greater than the sum of the improving effects used separately.(2)Aiming at the problem of existing performance bottlenecks in the BiLSTM-KVMNCRF model,we add label correction based on deep reinforcement learning to BiLSTM-KVMNCRF model and propose BiLSTM-KVMN-CRF-DRL model.Agent based on deep reinforcement learning is used as a label corrector,setting label correction threshold,to correct the labels with uncertainty greater than threshold in the labeling results of the BiLSTM-KVMNCRF model.The experimental results show that,compared with BiLSTM-KVMN-CRF model,the F1 value of BiLSTM-KVMN-CRF-DRL model on Co NLL-2003 dataset is 92.35%,increased by 0.87%;on Onto Notes5.0 dataset,the F1 value is 88.05%,increased by 0.63%.In addition,experiments on the impact of different label correction thresholds on model performance show that setting a suitable label correction threshold can effectively avoid incorrect correction processing of the correct label. |
