Research On Chinese Named Entity Recognition Based On Improved Transforme

Named entity recognition can intelligently identify various entity information contained in the text,which is a basic and important research direction in the field of natural language processing.Provide research basis and key information for other tasks such as event extraction,relation extraction,knowledge graph construction,and question answering.At present,deep learning technology has made breakthroughs in the field of named entity recognition,but the effect of named entity recognition for Chinese text is still not ideal.The main reason is that Chinese has a unique language structure with words as units,and there are no obvious boundary symbols before and after words in the text,which affects the performance of the model for entity recognition in Chinese texts.At present,the common solution to the above problems is mainly to use the lattice structure to effectively use the lexical information in the text.Transformer has become the most commonly used neural network model in the field of natural language processing.Thanks to its powerful global feature extraction and parallel computing capabilities,Transformer-based models have achieved the best research results in many natural language processing fields.However,the existing attention mechanism cannot effectively utilize the location information of the lattice structure,so the accuracy of named entity recognition still needs to be further improved.This will affect the research effect of downstream tasks such as event extraction and knowledge graph.How to improve the effect of named entity recognition in Chinese text has become a key scientific problem to be solved in the field of natural language processing.In response to the above problems,this paper conducts research on improving the Transformer to make full use of the position information in the lattice structure.The research work includes the following two parts.1.The self-attention mechanism variant currently used in the Transformer-based model has the problem of insufficient utilization of position information in the lattice structure.The position information in the lattice structure is usually extracted as a distance feature information stream and input to the Transformer.By using a variant of the self-attention mechanism proposed in Transformer-xl,when calculating the attention score between two characters or words,the influence of the distance feature on the previous characters or words is also included.However,this selfattention mechanism variant has the problem of insufficient use of distance features.This dissertation proposes a modified self-attention mechanism variant,which also considers the impact of distance on the back characters or words,making full use of the distance feature.The modified Transformer model based on this mechanism has a better ability to utilize position information in the lattice structure.The performance of the model is tested on two open-source datasets,confirming the effectiveness of the improved attention mechanism proposed in this dissertation.2.When using the above-mentioned improved self-attention variant,since the source of the distance feature and the semantic feature are independent of each other,the influence of the distance feature in different heads is different under the multi-head mechanism.This reduces the ability of the model in focusing on local information on semantic information.This dissertation proposes a hybrid-head mechanism that combines the multi-head mechanism and the single-head mechanism.This method uses a single-head mode when calculating the influence of distance features,and uses a multi-head mode when calculating the attention score between semantic information,which effectively solves the problem of noise interference caused by different influences of distance features on different heads.This dissertation proves the effectiveness of the hybrid-head attention mechanism proposed in this paper by testing on three authoritative public datasets,and has achieved a relatively advanced recognition effect in the field of Chinese named entity recognition.