Research And Implementation Of Sentence Pair Modeling Method Based On Deep Learning

In the information age,the Internet has a large amount of text data.In order to facilitate the retrieval and use of these text data,it is necessary for the computer to understand the text.The core of understanding the language is to understand the language semantics.Considering that language semantics are difficult to express,a method is needed to judge the accuracy of computer understanding semantics.Sentence pair modeling task is proposed as a kind of task to evaluate language comprehension ability because of its simple form,which can indirectly test artificial intelligence's understanding of semantics.The main form of sentence pair modeling task is to predict the semantic relationship between a pair of sentences(or a group of sentences),which is a general term for a series of semantic understanding tasks.It is difficult for neural networks to extract features with strong generalization performance based on a small amount of data,so neural network models often perform poorly on small-scale data sets.In this thesis,firstly,a set of feature set is constructed to measure the semantic similarity of sentences based on multiple word representation methods and distance / similarity function,in order to improve the performance of neural network model.Secondly,we use CNN,RNN and attention mechanism to construct multiple neural network to extract different forms of features.Finally,several different neural network models are integrated to improve the performance of the model.We performed experiments on the relevant dataset of CHIP 2018 Task 2,and the results show that this method can effectively improve the performance of the neural network model.The shallow neural network is difficult to extract complex features.Simply increasing the model depth can increase the complexity of the model,but it will make the model difficult to train and performance degradation.Inspired by the residual connection network,this thesis introduces the residual connection into the model,which improves the depth of the model and makes the model still easy to train,alleviates the problem of model performance degradation,and finally improves the model performance.Experimental results show that the model proposed in this thesis achieves the best performance on the SNLI dataset when using tokenbased word embedding,and related experiments based on BERT show that the model proposed in this thesis can make better use of word embedding information and get more performance improvements.In addition,the form and the goal of each subtask of the sentence pair modeling task is similar.In order to reduce repetitive work and help researchers build and train the neural network model under this task better and faster,this thesis builds a general framework for sentence pair modeling task based on configuration.The framework is based on Pytorch and built with the tensorboard toolkit.It can complete the construction,training,and visualization of the training process of neural network models for modeling tasks through parameter configuration and simple data flow custom functions.Because the parameterization is very intuitive,the framework reduces the workload and improves the efficiency of work.The work of this thesis is based on this framework.