Research On Open Dialogue Algorithm Integrating Context Modeling And Maximum Mutual Information

In recent years,with the progress and development of artificial intelligence technology,service robots have gradually entered thousands of households,especially the wide application of chatbots in the open field in daily life,which gives the "temperature" to the escort robots,satisfies the communication needs of users,and provides an important means to solve the escort problems of empty nesters,left-behind children,etc.Therefore,it is of great practical significance to study open field chatbot dialog generation algorithm.With the rapid development of network social media platforms,the problem of insufficient data in natural language processing technology has been solved,and massive dialogue corpus has been built,laying a data foundation for the research of data-driven open domain dialogue algorithm.The pre-training model based on Transformer has been successfully applied to natural language processing tasks and gradually applied to open domain multi-round conversation generation tasks.Although the pre-training model has achieved better results than the traditional cyclic neural network,it still has some defects in accuracy,fluency and diversity.In order to improve the effect of the dialogue model,this thesis has optimized and improved the algorithm in three aspects:context relevance,response diversity and topic switching.The specific research content can be summarized as follows:The first is based on the GPT2 pre-training model and the construction principle of multiround dialogue,a Chinese multi-round dialogue model based on GPT2 is proposed.Top-p and Top-k sampling methods are used to replace greedy search and cluster search to optimize the decoding strategy and improve the model performance.In order to further improve the context relevance of dialogue statements,a dialogue history Keyword copy mechanism is introduced into the Chinese multi-round conversation model based on GPT2,and a Chinese multi-round conversation model based on KeywordCopy-GPT2 is designed to improve the probability of keyword copy generation.The fusion improved network achieves a good effect in balancing the relationship between replication and generation.Secondly,in order to reduce the probability of safe reply generated by the dialogue model and improve the diversity of reply statements,the maximum mutual information method is introduced to optimize the objective function on the basis of the Chinese multi-round conversation model of GPT2 based on the keyword copy mechanism,and a Chinese multiround conversation model based on MMI-KeywordCopy-GPT2 is designed.By calculating the dependency and correlation between input and output of maximum mutual information,the dialogue generation effect is improved.Thirdly,in order to solve the problem that the contents of the sentences replied by the dialogue model remain in the previous topic due to the topic switching in multiple rounds of dialogue,the dynamic information flow mechanism is introduced to guide the model to generate the reply statements through the semantic influence of the discourse of the historical information of dialogue,which can make the dialogue model better adapt to different topics and fields in the process of chat.Last,with Qiming service robot as the hardware platform and relevant communication mechanism,the chatbot system is built based on the optimized and improved Chinese multiround dialogue model in this thesis.The above parts explore how to better model the dialog generation task based on the advantages of the pre-training model and further improve the quality of the open domain dialog system.The experimental results show that the dialog model can generate smooth,informative and context-relevant response statements.Objective evaluation index and subjective evaluation index are used to evaluate each model proposed in this thesis.The dialogue model proposed in this thesis has achieved good evaluation results in both objective evaluation and subjective evaluation.