Research On The Generation Method Of Short Text Dialogues With Fusion Of Positions

In recent years,neural conversation models have achieved huge success in both academic and commercial worlds.While previous work ignores the underlying stance in conversation.However,the stance is an important factor in human conversation,where we often have a stance towards the input sentence firstly,and then make a reply carrying the stance.Hence,in this paper,under the methodology of deep neural networks,we explore how to enable neural conversation models to exploit the stance during conversation.More specifically,we study it in the following three aspets:(1)Stance detection for short-text conversation: Based on 16772 input-response pairs that are annotated with stances by humans from Weibo short-text onversation dataset,we use Bi-LSTM and Transformer to implement the stance detection model,respectively.Besides,to make stance classifiers more robust,we also train language models.Experimental results show that when using pretrained language models,both models achieve better stance detection performance,and Transformer achieves the best performance.(2)Short-text conversation response generation conditioned on the given stance: We use the Transformer stance detection model pretrained with the language model to automatically label the stance pseudo-labels for the Weibo short-text conversation dataset.Then,upon the dataset with pseudo-labels,we train the LSTM-based sequence-tosequence conversation model,and compare three stance fusion methods,e.g.,fusion in the encoder,fusion in the decoder and fusion in both the encoder and decoder.Besides,during decoding,we use beam serach and top-k sampling,respectively,and compare their BLEU scores,Distinct scores and stance F1-measures.Experimental results show that beam search achieves higher BLEU scores and stance F1-measures,while top-k sampling achieves higher Distinct scores.(3)Short-text conversation response generation conditioned on the autonomous stance: Based on the stance-conditioned short-text conversation generation,we further explores problems of autonomous stance generation and fusion.First,based on the dataset of the input sentence and stance pairs,we use Bi-LSTM to build autonomous stance generator.Then we use the response retrieval model to retrieve reasonable responses conditioned on the autonomous stance.Finally,we use the retrieve-and-refine model to copy,modify,or rewrite the retrieved response to generate a more reasonable response.Besides,we use beam search and top-k decoding,respectively,to compare the response generation model,response retrieval model,and the retrieve-and-refine model.Experimental results show that the autonomous stance generator can generate reasonable stances,and compared with the generation model,the retrieval model has higher Distinct scores but lower BLEU scores,yet the retieve-and-refine model can improve BLEU scores of the retrieval model.