Research On Named Entity Recognition Based On Transfer Learning

Named Entity Recognition(NER)is one of the mainstream methods that transform unstructured text into structured.The performance of the Natural Language Processing(NLP)downstream tasks is affected by NER model.For deep learning,sufficient label dataset determines model performance.Recently,in order to obtain NER model of better performance,most of the methods have added auxiliary features.However,the auxiliary features require resources to construct,obtain difficultly,there are large differences of auxiliary features in different fields,and the performance of NER model is affected by the quality of auxiliary features.Therefore,this paper focuses on the text itself,does not introduce auxiliary features,uses different feature encoders to obtain multiple types of feature distributions,uses transfer learning to make the features interactivity and sharing,to compensate for the lack of auxiliary features.The main research contents are as follows:(1)Construct a Chinese NER model with gated multi-feature extractors based on BERT(Bidirectional Encoder Representations from Transformers)transfer model.The model uses BERT to obtain word embeddings and transfer them to target model.Using three feature extractors to build two-layer feature extraction layer.Based on the multihead self-attention mechanism,two feature extractors with different encoding mechanisms were used to jointly excavate the text information output by BERT,and share-feature extractor is used to encoding features output by the first feature extraction layer.At the same time,gating was introduced to iterated dilated convolutions,which can control flow and expand data circulation channels.In addition,two conditional random fields were constructed to predict label of sentence.Finally,conducting our experiment on two different widely used Chinese NER datasets and compared with advanced NER model,the experimental results demonstrate that our proposed model can improve the ability of recognizing entities.(2)Adopting adversarial transfer learning to construct a Chinese NER framework.This model builds two NER sub-models,the two NER sub-models jointly encode word embedding output by language model,then use adversarial transfer to learn and transfer the knowledge between the NER sub-models,to obtain feature distribution with more diverse and stronger characteristics.Skip connection was introduced into feature encoder,to reduce the adverse effects produced by network degradation.In addition,the adversarial loss was added to encourages the feature generator to generate highquality shared features.Validation experiments were conducted on various fields datasets,and compared with representative NER models such as LR-CNN,Lattice LSTM,FLAT,etc.The experimental results showed that the F1 produced by our model highest increased by 4.17 percent,and the rate of improvement is higher than other comparison models,indicating that our proposed adversarial transfer NER model significantly and consistently improved in terms of generalization,robustness,and evaluation index.