Research On Text Representation Model And Application In Text Classification And Natural Language Inference

Due to huge application potential,natural language processing has received extensive attention from researchers in different fields.It is a core issue on how to effectively represent words,sentences,or documents to describe their semantic information in the field of natural language processing.There are various text representation strategies for different application purposes,such as sentence relationship reasoning,text classification,sentiment prediction,entity and relationship recognition,etc.However,we found the following two problems:(1)As the most popular text representation network at present,the bidirectional LSTM network ignores the contribution of each word vector and the text vector in each direction to the overall text representation,which affects the final encoding result.(2)The current multi-task jointly training model for text representation does not take into account the effect of jointly training different datasets under the same task,and ignores that the data enhancement may enhance the effect by joint training of different datasets in the same task.In response to the above problems,the main research work of this article is as follows.(1)A sentence representation model based on directional attention mechanism is proposed.In the traditional bidirectional LSTM network,we use a word attention layer and an adaptive direction weight layer to learn the contribution of each word vector and text vector in each direction to the overall text representation.The vectors are weighted by the contribution degree to improve the encoding performance of the text representation model.This model is applied to sentence representation learning.Based on the ESIM model,the a ESIM model is proposed for processing natural language inference tasks.The experimental results show that the a ESIM model performs better than the original model.At the same time,it demonstrates that the bidirectional LSTM network based on the directional attention mechanism is superior to the traditional bidirectional LSTM network.(2)A document representation model based on directional attention mechanism is presented.We used a bidirectional LSTM network based on the directional attention mechanism to construct a text classification model based on the directional attention mechanism,and the results are better than the common text classification models.Through multiple sets of ablation experiments,it is proved that multiple network components used for document representation learning can improve the ability of document representation learning.(3)A multi-task learning scheme for jointly training of homologous datasets is explored.We classify different datasets under the same task according to the semantic similarity of different text datasets,introduce the concepts of homologous datasets and non-homologous datasets,and jointly train the text representation model using a multi-task learning framework.Experimental results show that the text representation model based on the jointly training of homologous datasets has better performance than the text representation model trained on a single dataset.But for the text representation model obtained by jointly training of non-homologous datasets,the performance has not been improved.