Research On NER Algorithm Combining Pre-training Model And Adapters

In the filed of Natural Language Processing(NLP),Named Entity Recognition(NER)is one of the essential problems.However,the methods of solving NER depend on the corpus and language.NER has experienced three stages of development: machine learning,deep learning and pre-training model.These three stages of development also reflect the development of NLP.Machine learning methods such as HMM and CRF have achieved good results on some NER datasets.But its application is restricted because the model uses the wrong generation probability for calculation.The deep learning method represented by LSTM relies on semantic learning.The text sequence needs to pass through the embedding layer before entering the model.This embedding layer maps the sequence to the word embedding vector.When each word in the sequence is mapped into a word embedding vector,then the model can learn the function with the help of the nonlinear fitting ability of the neural network.And use this function which is to predict each label of the token.So pre-training models such as ELMo and BERT came into being.The pre-training model is designed to effectively represent the text.It trains the model to learn the semantics of the text through the tasks within the model,and obtains the word embedding vector representation.In order to improve the performance of the Bi LSTM-CRF model on named entity recognition task,we add ALBERT model on the basis of Bi LSTM-CRF and proposes a new significant ABC model.In the ABC model,first ALBERT can generate a word embedding vector with features according to the context semantics,and then pass the word embedding vector to the Bi LSTM model for NER.Finally the CRF layer constrains and optimizes the labeled sequence.ABC model can effectively improve the F1 of NER.The generation of the pre-training model has a huge impact.It can solve different types of problems in the field of NLP by combining with different downstream models.The era that one model solves one problem is gone forever.However,the pre-training model also brings many problems,such as the huge model scale,the large amount of parameters,and more adjustment parameters after connecting to the downstream model.These all bring about the problem of large time and space consumption for the training model.In order to solve the above problems,this article continues to expand the ABC model and introduces the Adapter layers to generate a new significant Adapters-ALBERT-Bi LSTM-CRF(AABC)model.Different from the ABC model,the AABC model uses the Adapters module to adjust the parameters of the overall model.The Adapters module remains unchanged during the model training phase,and only adjusts the parameters in the Adapters module during the fine-tuning.In order to verify the reduction of the comprehensive tuning parameters,the model was tested on the datasets of three types of tasks:named entity recognition(NER),sentiment analysis(SA),and natural language inference(NLI).The experimental results show that the AABC model achieves good results on 3 public NER datasets;it achieves optimal results on one SA datasets and 4 NLI datasets.The AABC model combines the advantages of classic machine learning,deep learning and pre-training models,and can increase the F1 value by up to 8% on the NER dataset.Compared with the BERT-BC model,the parameters of AABC are reduced by up to 97%on the SNLI dataset.It is a comprehensive model with great potentials.At the same time,named entity recognition is one of the important steps in the construction of knowledge graph.In addition to named entity recognition,the important steps of knowledge graph construction also include entity linking and relationship extraction.Therefore,in order to verify the actual effect of the AABC model in named entity recognition,this paper applies the AABC model as one of the modules to the construction of small-scale medical diseases,and realizes simple question-and-answer operations through probability model calculations,and finally connects to the graph database Neo4 j for visualization.