Research On Topically Driven Open Domain Dialogue System

Conversation systems,also known as dialogue systems and sometimes chatbots,aim at generating relevant and fluent responses in human-like natural language,which is a challenging task in AI and NLP communities.Conversation systems can be divided into goal-driven systems and open-domain chatbots.The former such as technical support services intends to help people to complete a specific task while the latter focuses on talking like the human in the open domains such as gossip or interaction between computer game characters.Previous researches on conversation systems focused on goal-driven systems.Recently,with the large amount of conversation data available on the Internet,open-domain chatbots are drawing more and more attention in both academia and industry.The current open-domain conversation system is driven by deep learning methods.Specifically,sequence to sequence(Seq2Seq)model is widely applied to generating response.However,in practice,neural conversation model based on Seq2 Seq model has two major problems.First,end-to-end generation model is based on maximum likelihood estimation and tends to generate trivial or noncommittal responses,such as "I don't know","Well",etc.Second,the basic Seq2 Seq model can only learn the local information in conversation corpus and there is no memory mechanism for the conversation context,for the reason that it is easy to generate inconsistent and contextfree responses.In this paper,we study the response generation problem via incorporating topic information into the sequence-to-sequence framework,which aims to use the topic information to help the dialogue system to generate more interesting,diverse and informative responses.Our work can be divided into three parts:Firstly,we propose a method to extract conversation text topics by convolutional neural networks.For the characteristics of dialogue data,the convolutional neural network is appropriate to model the dialogue text.Secondly,based on convolutional neural network and multi-layer perceptron,we propose a response topic prediction method.Since the assumption that the topics in the message context and in the response should be consistent,or slightly different,we add the multi-layer perceptron into the convolutional neural network-based topic model to simulate this difference.Finally,we propose a topically driven conversation model(TDCM).Our model exploits the convolutional neural network and the multi-layer perceptron to extract the message topic and predict the response topic.Then,the topics are combined with the bidirectional RNN-Seq2 Seq framework by an attention mechanism through a gate and the mixed probabilistic model.In addition,to solve the problem of the inconsistence between the training and the testing process,we improve the process of beam search during the test stage,which enhances the performance with better efficiency.TDCM introduces the topic information into the Seq2 Seq model.The topic which contains the context information reduces the generation probabilities of the inconsistent and context-free responses.Moreover,TDCM increases the probability that the topic word appears in the responses,making the responses more informative and reducing the generation probabilities of trivial or noncommittal responses.The experiments conducted on the Cornell Movie Dialog Corpus demonstrates that the proposed TDCM outperforms pervious state-of-the-art dialogue systems in terms of lexical richness and semantic coherence.This proves the promotional effects of the topic information for the dialogue response generation.