Multi-prototype Word Vector Based On Context Word Embedding

Deep learning plays an increasingly important role in the field of Natural Lan-guage Processing.Word embedding,as the most commonly used expression method in deep learning models,has a direct impact on representing the text.However,in most word embedding models,word uses the same word embedding in different contexts,that is,single-prototype word embedding.In actual language expression,some words often express different meanings in different contexts.At present,word embedding mostly adopts single-prototype word embedding,which can not well express the dif-ferent usage of polysemous words in different contexts.To overcome this problem,a multi-prototype word embedding based on context semantic embedding is proposed to improve the current word embedding model.The work of this paper mainly includes the following parts:1)Because of the semantic diversity of different words,the number of clusters K should be determined according to the complexity of word meanings when clustering the semantics of different words.In this paper,we use two-parameter Dirichlet Process to construct non-parametric bayes to determine the number of word meanings in terms of the number of meanings and the unbalanced distribution of word meanings.2)In order to prove the feasibility of two constrution methods of two-parameter Dirichlet Process,two complete semantics induction algorithms are proposed according to the differences of the two proposed construction methods of two-parameter Dirichlet process.Gibbs sampling is used for parameter solution.The experiments are set up to verify the effect of the proposed methods for SemEval-2010.3)We propose a new neural network model to solve the problem of word sense induction,word sense represention and word sense disambiguation.The model we proposed solves the problem that word2vec word vector model can not represent pol-ysemous words well.Long Short-Term Memory(LSTM)and Neural Tensor Network(NTN)are used to train the context vectors.Two-parameter Dirichlet Process is used to train the semantic clusters of the context vectors.Finally,the neural network model is used to train the semantic vectors.4)We use Wikipedia 2018 data set to train the multi-prototype word embedding model.The words with more occurrences are extracted to calculate multi-prototype vectors.Word similarity calculation experiment is carried out on WSCS dataset and WordSim353 to detect the improvement of the current algorithm.In addition,in order to prove the effectiveness of the proposed algorithm in natural language processing ap?plications,multi-prototype word vectors are applied to the task of named entity recog-nition and sentence-level sentiment analysis,and a variety of contrast algorithms are used to verify the effectiveness of the proposed algorithm.