| To compare a pair of sentences is a fundamental technology in many NLP tasks,such as answer selection,textual entailment,and paraphrase identification.The rapid development of deep learning overturns the era of manual design features for semantic matching.Cooperating with Siamese network,we use CNN,RNN and attention mechanism to design a semantic matching model for different scenes.As a general pre-training model,Bert helps natural language processing go further in most scenes.Based on this,we combine Siamese network with Bert to deal with the problem of long text matching better.Siamese network which has two subnetworks with the same structure and shared weights is a general framework for calculating semantic similarity.Two inputs of the twin network receive one text information at the same time.The two texts are transformed into vectors by the sub network sharing weight,and then the distance between two text vectors is calculated by a certain distance measurement algorithm.Two inputs ends of the Siamese network accept two text messages at the same time,and the two text are converted into vectors through a subnetwork that shares weights.Then a distance measurement algorithm is used to calculate the distance between two text vectors.In this paper,three text matching algorithm models are proposed:(1)The first research view of this paper is how to build a more efficient semantic matching model.SWEM is a simple model based on word vector and pooling technology.The module itself has no parameters,so using this module to build twin network has fewer parameters and can be trained faster.The experimental results show that the Siamese network based on SWEM has only 1/3 parameters and 12 times faster than the Siamese network based on RNN,while the experimental results are almost the same.At the same time,we do further research on the impact of text word order on the model and the impact of semantic matching aggregation methods on the model.(2)The second research view of this paper is how to better handle short text matching.According to the difference between the pair of sentence,we divide semantic sentence matching into two situations: Situation A is that the pair of sentences are worded with a context relationship,Situation B is that two are equal in semantics.Models for Situation A works in Situation B too,so prior deep work mostly model each sentence's representation considering the interaction of the other sentence simultaneously.However,models designed for Situation A bring redundant information for Situation B,the models become more overweight and complex.In this paper,for sentence pairs with equivalence,we present a deep architecture with comparison-interaction separated to match two sentences,which based on Siamese network for comparison and multi-head attention for interaction information between sentence pairs.Experimental results on four latest Chinese sentence matching datasets outline the effectiveness of our approach.(3)The third research view of this paper is how to better handle long text matching.CNN,LSTM or GRU have been konwledged that they cannot handle long text matching perfectly due to their difficulty in memorizing long-distance information.The presence of Transfomer improves long text matching.BERT,as a pre-training model based on Transfomer,has achieved the best results in many natural language processing tasks.Using BERT and Siamese network,we construct a long text semantic matching model.Compared with the direct use of BERT for semantic matching,the model we built is more effective and stable.At the same time,we also compare the effects of different pre-trained models on our models.All the experimental results in this paper are based on Chinese datasets,including four short text semantic matching datasets and one long text semantic matching dataset.The experimental results show that the three models designed in this paper can effectively improve the semantic matching effect of the existing models.Because Chinese word segmentation is different from other language,the three models designed in this paper are mainly for Chinese semantics matching,and experiments based on other language need further study. |
