Chinese Word Sense Disambiguation Based On Transformer And LSTM Model

Word sense disambiguation is a common problem in the field of natural language processing.The purpose of the word sense disambiguation problem is to identify correct word sense from multiple word sense lists of ambiguous words.The disambiguation model often needs to judge target vocabulary’s sense based on its context.Word sense disambiguation affects the performance of many tasks in natural language processing,including m achine translation,text classification,sentiment analysis,and speech recognition.The low accuracy of disambiguation is an urgent problem to be solved in the field of natural language processing.In order to solve the problem of low disambiguation accur acy,this article uses transformer model,LSTM model,and attention mechanism to construct a disambiguation network.Before training a deep learning model with corpus,the data must be preprocessed.That is,the vectorization tool is used to transform the corpus into word vectors.The specific process is to map the words into a vector space.The words with the same context may have similar vector distances.This article uses a supervised method to train the model.The article first introduces the application background and usage scenarios of word sense disambiguation,as well as the research trends of domestic and foreign scholars in the field of disambiguation,and expounds the algorithm and framework which they proposed.Then it describes the preparation and preprocessing process of the corpus,including the use of vectorization tools,word segmentation of the corpus,and feature labeling.The principle of word sense disambiguation by transformer model is analyzed in detail,including the structure of trans former model,the calculation process of the data in the model,and the classification process.Then it introduces the principle of word sense disambiguation of the serialization LSTM,and mathematically analyzes various gate structures of LSTM network.At the same time,an improved version of BiLSTM is introduced.In order to improve the disambiguation ability of LSTM network,this paper introduces a single-layer attention mechanism in the LSTM network.Finally,transformer network and LSTM network are merged to deal with word sense disambiguation.Transformer is a multi-layer network based on multi-head attention mechanism,which can be seen as an extension of single-layer attention mechanism.The fusion model can make full use of the long-term and short-term dependence characteristics of LSTM network and the parallel processing mechanism of transformer network.The fusion model is trained in supervised manner in experiment.The model calculates the loss of the classification result,and then the optimizer is used to perform gradient descent on the loss to optimize the model parameters.Finally,test corpus is used to test the optimized model.Experimental results show that the disambiguation method proposed in this paper performs better than a single deep l earning model.