Improved Attentional Seq2seq With Maxout Fusion Layer For Dialogue Generation

Chat robot is one of the core research directions which has broad application prospect and important commercial value.Based on the progress of big data and natural language pro-cessing technology,achieving more natural chat robot is getting closer to reality.Internet advancement makes people more likely than ever to obtain big data for human dialogue on the internet.The advancement of deep learning also provides technical support.Chat robot technology based on deep learning can use neural networks to extract features,understand dialogue information,and learn language skills to express.The goal of non-tasking chat robot is to achieve natural communication between human and machine.Good chat bot should understand the dialogue information and give a variety of responses.This thesis mainly focuses on generating more diverse and more relevant response sentences.Recently,researchers have begun to use seq2seq generation models to replace the original template-based and retrieval-based methods.This thesis introduces the attention mechanism and beam search algorithm on the basis of traditional seq2seq and combines the maxout fusion layer to improve the traditional seq2seq structure.Then this thesis introduces a penalty factor to improve the traditional beam search.The main work of this thesis is as follows.This thesis first selects a multilayer bidirectional LSTM as the encoder to better represent the context information on the basic seq2seq,then combines the residual connection and the maxout network layer to improve the structure of the output part of the decoder so as to alle-viate the problem of network degradation in deep learning.In addition,this thesis combines beam search algorithm and improves the traditional beam search by introducing penalty fac-tor.Finally,this thesis uses diverse beam search to further improve the diversity of generated sentences.In the model comparison experiment,this thesis selects the Transformer model to compare with the improved seq2seq model.This thesis uses the NLPCC2018 multi-round dialogue dataset.First,this thesis decom-poses the multi-round dialogue into a single round of dialogue,and compares the model effects based on automatic evaluation indicators.The experimental results show that us-ing Perplexity,BLEU and ROUGE-L indicators,the improved seq2seq has advantages over other models in various indicators.Compared with the baseline models seq2seq and Trans-former,the improved seq2seq has better performance in sentence fluency and diversity which proves the effectiveness of introducing the maxout fusion layer on the basic seq2seq and the improvement on the traditional beam search.