Research On Text Filling Algorithm Based On Deep Learning

Text filling,also known as missing text generation,is a research field of natural language processing(NLP),whose main task is to fill in missing fragments of text.In recent years,with the rapid development of deep learning,text filling task has made some achievements.However,there are still some problems.For example,the information of the missing part of the filled text is not consistent with the context semantic information,which makes the coherence and fluency of the filled text poor.The diversity and complexity of filling the missing part of the text,as well as the consideration of grammatical,syntactic and contextual semantic information,make the filling task more difficult.In order to solve the problems mentioned above,this paper studies the text filling algorithm based on deep learning.First of all,in this paper,the semantic similarity work has carried on the related research,semantics is the core of natural language processing.In order to solve the problem of insufficient information interaction and semantic feature loss in semantic similarity matching task.Is proposed based on an intensive connection network and multidimensional feature fusion method of semantic similarity matching.Secondly,based on the study of semantics,this article addresses the problem of poor semantic coherence of the filled text due to the loss of semantics in the text filling process.Proposes a fusion and semantic similarity based on prediction network loss of text filling method.The model reduces semantic loss through cross entropy and similarity fusion.Finally,the model proposed in this paper is tested and compared with other models in various actual data.The experimental results show that the effect of the proposed method in this paper is better than other models.The main contributions of this article are:1.This paper proposes a semantic similarity text matching method based on the fusion of dense connection network and multi-dimensional features.The dense connection method can closely connect the word embedding feature at the bottom layer and the dense module feature at the top layer,which enriches the semantic features of the sentence.Secondly,a multi-dimensional feature fusion method is designed so that the model can capture more semantic information between sentence pairs.By comparison with other semantic text matching algorithm.The model in this paper is evaluated on three different tasks,namely natural language inference SNLI and Sci Tail,interpretation and recognition Quora,and four benchmark datasets of Ant Financial.The accuracy rate has been increased by 0.3%,0.3%,0.6% and 1.81%,respectively.Indicating that the performance of the method proposed in this paper is better than other strong baseline models:(1)Different from the traditional residual connection network method,the dense connection network can be used to closely connect the output features of multiple dense modules with the original features(word embedding features),which enriches the semantic features of sentences.(2)Using multi-dimensional feature fusion,on the basis of word-level interaction based on the attention mechanism,the similarity feature,difference feature and key feature between sentence pairs are multi-dimensional feature fusion.The model is able to capture enough sentence semantic relationship between.2.This paper proposes a text filling method based on the fusion loss of prediction network and semantic similarity.The prediction network uses a sequence-to-sequence neural network model to reduce semantic loss through the fusion of cross-entropy and similarity.First,the encoding end encodes the missing text to obtain contextual semantic features.Second,the decoder uses the Transformer network to fill in the missing text.Finally,the loss of semantic similarity is integrated to make the semantics of the filled text more coherent and smooth.By comparing with other filling methods,different missing rates and blocking strategies were used on Yelp,Grimm and NBA news data sets to evaluate,and the accuracy and fluency have been significantly improved.Shows that the method proposed in this article is significantly better than other methods in text filling tasks:(1)The prediction network consists of two parts: the codec,the Bi-LSTM network and the Transformer network.The missing text can be filled in through the prediction network.In the Transformer network on the decoder side,the multi-head attention mechanism can capture the contextual semantics of missing text fragments during the text filling process.(2)On the basis of the original cross-loss entropy,the semantic similarity loss is fused.The two losses learn different semantic features,the cross-entropy loss learns local semantic features,and the similarity loss learns global semantic features of the text.Finally,the weighted fusion of the two losses is used as the loss function of the network.