Research On Named Entity Recognition Combining Residual Structure And Attention Mechanism

Named entity recognition has always been a focus of natural language processing research,and it is intended to identify person name,place name,organization name and other entities from text.As a fundamental task in natural language processing,named entity recognition plays an important role in some tasks such as question answering and relation extraction.At present,Chinese named entity recognition models mainly use external knowledge to obtain character embeddings containing more semantic information,and then use recurrent neural network to obtain sequence information.However,external knowledge may be not easy to obtain and its quality is hard to evaluate,and the recurrent structure of the network leads to low model efficiency.Therefore,it is of great significance to design a practical and efficient named entity recognition model,but it also faces many challenges.Through in-depth analysis of the existing problems and research status of named entity recognition,this thesis improves the existing named entity recognition models.The contributions of this thesis are as follows:(1)A named entity recognition model based on data augmentation and Multi-level CNN with Residual Structure called RS-MCNN model is proposed.RS-MCNN model first uses data augmentation method to increase the number of training samples,which not only improves model performance,but also improves the model practicability.Secondly,the model uses different character embedding look-up tables to construct multimodal embeddings for abundant semantic information.Then,the multi-level residual convolution is used to extract features,capturing and fusing the context information of different scales for each character.The experimental results on many benchmark datasets show that compared with the existing named entity recognition models based on recurrent neural network,RS-MCNN model achieves better recognition results and effectively improves model efficiency.(2)A named entity recognition model based on Multi-level CNN with Residual structure and Attention mechanism called RA-MCNN model is proposed.Although RA-MCNN model has large receptive field and obtains long-distance context information through stacked convolutional network,it is still difficult to obtain global context information.Therefore,this thesis proposes a character-sentence attention mechanism to construct the relationship between each character and its sentence for global context information.Adding attention mechanism to RS-MCNN model can make up for the deficiency of multi-level CNN to a certain extent.The experimental results on different datasets show that RA-MCNN model can effectively improve the RS-MCNN model and further improves the accuracy of entity recognition.(3)A named entity recognition model based on pre-trained model BERT and Multi-level CNN with Gated mechanism and Attention mechanism called BERT-GA-MCNN is proposed.Considering that the pre-trained model can learn more semantic information and has strong generalization ability,the embedding look-up tables in RA-MCNN model are replaced by BERT to generate character embeddings.In addition,in order to enhance the ability to evaluate the importance of features,this thesis also uses the gate mechanism to improve the multi-level CNN and realizes the adaptive fusion of features.The experimental results show that both pretrained model and gate mechanism make improvements on the model and further improve model performance.