Generative Dialogue System Based On Transformer

In natural language processing tasks,the dialogue system is one of its most promising key application directions.Retrieval-based and generation-based are the mainstream methods for the implementation of existing dialogue systems.The retrieval method is to score candidate responses and select responses to provide relatively fixed and patterned responses.Search-based systems rely heavily on the size and quality of the corpus.As a result,candidate responses may lack important information in the context and are based on retrieval.The method limits the richness of the generated responses,and the output results are relatively blunt.Therefore,in order to allow the dialogue system to better complete tasks such as information expression and emotional expression,a generative-based method is used to construct the dialogue system,so that the responses generated by the dialogue system are more natural and lively and conform to the contextcritical information.Based on the existing dialogue model,this paper studies how to enhance the meaning and semantic understanding of the dialogue model and the phenomenon of information weakness under the condition of multiple rounds of dialogue.The important content and contributions of this article are as follows:First,compared with the current Chinese dialogue system,the performance of the existing dialogue system is not ideal because the Chinese information is more complicated and the boundary division of words is biased.In view of the shortcomings of existing research,this paper designs a Chinese generative dialogue system based on Transformer,which uses only a multi-layer Transformer decoder structure to build the system,and uses an incomplete Mask mask design to achieve one-way language generation,that is,question sentences.It can perceive contextual information in two directions,while the reply sentence can only be outputted in one-way autoregression.Output one word at a time through autoregressive method,after generating a new word,add the word to the input text sequence,the new text sequence will become the next input of the model,and so on until the end identifier is generated.The process of this system makes the task of unidirectionally generated dialogue more logical and reasonable,and its performance is better than that of the traditional dialogue system.Second,in order to enhance the ability of the dialogue system to understand the meaning and semantics of words,this research proposes an Embedding optimization method for word fusion,which introduces the word embedding matrix of the pre-trained language model to fuse the word vectors in the dialogue task,and uses sparseness Softmax to avoid overfitting problems.It is verified through experiments that the optimization method of word fusion Embedding further improves the performance of the Chinese generative dialogue system,and verifies the feasibility of the optimization method.Third,through analysis,it is found that absolute position coding will cause remote information weakness,so it is proposed to use relative position coding to improve,modify the calculation formula of Self-Attention in the Transformer module,and replace the absolute position coding of the position embedding layer by adding relative position information.In order to enhance long-distance attention.The experimental results show that the use of relative position coding can reduce the phenomenon of remote weakening of information.