Design Of Dialogue System Based On Transformer

In recent years,with the continuous development of big data and deep learning technology,the dialogue system has attracted more and more attention in various fields.The dialogue system can be roughly divided into two types: task-oriented dialogue system and non-task-oriented dialogue system.The task-oriented system is designed to help users complete practical and specific tasks.The non-task-oriented dialogue system interacts with humans to provide reasonable response and entertainment functions.Usually,it focuses on talking with people in an open field.Although the non-taskoriented system seems to be chatting,it plays a role in many real-world applications.In the non-task-oriented dialogue system,the Seq2 Seq chattering robot is more common.However,the ordinary Seq2 Seq model may have problems such as negative emotional responses,questionable sentence responses,and low diversity of responses,resulting in poor user experience or inconsistent conversation context.At the same time,the existing Seq2 Seq model is not satisfactory for multi-turn dialogue systems.In multiturn answering choices,it is important to find important information in the previous discourse and properly imitate the relationship of the discourse to ensure continuity of the conversation.So the difficulty of multi-turn dialogue is: how to clarify the key information of the context,how to simulate the relationship between multiple rounds of dialogue in the context.This paper designs and implements a multi-turn dialogue system based on Transformer and memory network.The specific work is as follows:Firstly,in view of the poor performance of the current memory network in complex Q&A tasks,this paper proposes a new memory network based on the gated attention mechanism which applies the attention mechanism of the end-to-end memory network to the gated mechanism,improving the memory network to solve complex reasoning based on the end-to-end memory network and achieving better results in complex Q&A tasks.Second,for the performance problems of the Transformer and Seq2 Seq models in solving the dialog generation problem,this paper designed experiments to verify that the Transformer model performance using Self Attention is better than the Seq2 Seq model using RNN and Attention.And this paper use a more accurate BERT sentence vector in the sentence encoding,so that the model can get the information of the sentence better.Third,the original Transformer considers the context information to be relatively simple.The attention mechanism only acquires information at a single sentence level,but does not introduce historical information.In this paper designs and implements a new multi-round dialogue model based on the needs of multiple rounds of dialogue,the memory network stores historical information based on the original Transformer model,and it can obtain important information in historical dialogue better.The Beam Search algorithm is used in the generated dialogue to improve the diversity of dialogue generation.