Research On Key Technologies Of Open-domain Generation-based Dialogue Systems

Dialogue systems intend to converse with humans with a coherent structure.They have been widely used in real-world applications,including customer service systems,personal assistants,and chatbots.Early dialogue systems are often built using the rulebased method,which is usually labor-intensive and difficult to scale up.Recently,with the rise of social networking,conversational data have accumulated to a considerable scale.This promoted the development of data-driven dialogue systems.On one hand,data-driven dialogue systems can be divided into task-specific dialogue systems and open-domain dialogue systems.The content of the former is limited in taskrelated domains.The latter does not have such limitations,therefore,it involves more diverse topics and expressions.On the other hand,data-driven dialogue systems can be also divided into retrieval-based dialogue systems and generation-based dialogue systems according to the way that responses are produced.Retrieval-based dialogue systems reply to users by searching and re-ranking candidate responses from a pre-constructed response set.Mainly written by humans,these responses are always diverse and informative,but may be inappropriate to dialogue history due to their being prepared in advance and thus incapable of being customized.In contrast,generation-based methods can produce responses tailored to dialogue history.In this paper,we mainly study open-domain generation-based dialogue systems.Currently,the research focuses on two aspects: coherence and diversity.The coherence requires the response to be fluent and relevant to the dialogue history and is the basic aspect of dialogue systems’ performance.Based on that,dialogue systems are also expected to reply with diverse topics and flexible expressions.However,from the aspect of the coherence,existing systems tend to generate generic responses,which are less informative and thus less coherent to the dialogue history;Meanwhile,improving the informativeness by introducing external resources(i.e.,keywords)may lead to a decrease of relevance due to the lack of planning of the content.From the aspect of diversity,dialogue generation is essentially a “one to many” task,where the same dialogue history has many valid responses.However,the number of responses covered by the training set is limited,and the model lacks the exploration of potential responses,which limits the possibility of learning a diverse dialogue policy.In addition,the existing training approach that optimizes the likelihood of a single ground-truth response oversimplifies the goal of dialogue generation.Despite the model explores potential responses,the generation process will still be punished by the existing objective,which further limits the promotion of diversity.To this end,we propose the following research contents:Research on the Startup Mechanism of Dialogue Generation: Sequence to sequence based dialogue systems tend to generate generic responses that are of less information.This paper analyzes a possible cause of the problem: the existing startup mechanism tends to predict less informative high-frequency words as the first word,which affects the subsequent generation process in cascade.Based on that,this paper proposes a novel startup mechanism,which can significantly improve the accuracy and the informativeness of the first word and subsequently improve the informativeness of the complete response.Experimental results on multiple sequence to sequence based dialogue generation models show that it can significantly reduce the probability of generating generic responses.Multi-Keyword based Content Planning in Dialogue Generation: To address the relevance problem caused by the lack of content planning mechanism in information-rich dialogue generation,this paper proposes a content planning approach under the paradigm of multi-keyword dialogue generation.Given dialogue history,it can automatically reason the most suitable order of keywords and generate a response containing the keywords in the order.Counterfactual Reasoning based Training for Dialogue Generation: To address the issue of lacking the exploration of diverse potential responses,this paper proposes a counterfactual reasoning based training approach.It can automatically reason the scenario of a given ground-truth response and subsequently predict a potential response as an alternative,which is a counterfactual response.Experimental results show that the exploration of counterfactual responses can significantly improve the diversity of generated responses.Retrieval-Enhanced Adversarial Dialogue Generation Model: Existing models overly simplifies the goal of dialogue generation,making it hard to recognize diverse responses.To this end,this paper proposes a retrieval-enhanced adversarial dialogue generation model.It simulates the process by which humans evaluate the quality of responses based on similar dialogue data in experience.It introduces an N-best candidate response based discriminator to better identify diverse responses that are different from ground-truth,thereby encouraging the generation of diverse responses.In general,this paper is committed to research the two aspects of open-domain generation-based dialogue systems,including coherence and diversity.For the coherence problem,this paper discusses the way of improving informativeness,and the way of improving relevance with sufficient information.For the diversity problem,this paper studies the way of better exploring and recognizing potential responses in the training process to improve diversity.Our research can significantly improve the coherence and diversity of responses,making our responses more human-like and improving the user experience in real-world applications of the dialogue systems.