Research On Multi-turn Dialogue Reply Generation Technology For Chatbot

In recent years,with the substantial improvement of computer clusters ' computing power,algorithm researching and industry landing about Artificial Intelligence(AI)have entered the fast lane of development.Chatbot,as an important application of Natural Language Processing(NLP),which is an important branch of AI,has gradually become a research hotspot and a great industrial product direction in the academe and the industry.The multi-turn reply generation technology in the stack of chatbot technology has many advantages such as interest,diversity,robustness,etc.,and also has the challenges of data,algorithm,tuning and so on.This design mainly focuses on the related technologies about multi-turn reply generation in chatbot.Every sentence said by human beings during the chat process can be regarded as the semantic modeling of the current state of mind under the influence of past many rounds of historical dialogue.It will focus on some historical conversations in a full implicit way,and will also use implicit feedback in past rounds of dialogue to convey information,and will also omit or restore some information in the historical conversations in a explicit way.Based on the above thinking,we decided to explore multi-turn reply generation from multiple perspectives.The main research content includes:(1)Exploring the full implicit multi-turn dialogue attention modeling technology.We use the Seq2 Seq model(also known as the "encoder-decoder" framework)with attention mechanism.We try to model the historical dialogue statement and explore the influence of using different attention mechanism methods on multi-turn dialogue modeling and generation effect,to finally get a better attention mechanism multi-turn dialogue modeling method.We can use this modeling method to indirectly improve the multi-turn reply generation effect.(2)Explore semi-implicit and semi explicit multi-turn dialogue implicit feedback modeling technology.On the pragmatic level,we try to explore position,emotion,and stalemate implicit feedback.First,we use the Seq2 Seq model to pretrain multi-turn reply generation models,and use the techniques of position recognition and sentiment analysis to model these three implicit feedbacks and introduce them.In the process of parameter learning and tuning based on reinforcement learning,the implicit feedback is used to provide more information to the reply generation model in real time,achieving the goal of improving the reply effect.(3)Exploring the full explicit multi-turn dialogue omission recovery modeling technology.In the process of explicit reply generation,through modeling the omission phenomenon in the dialogue process,part of the content omitted in the dialogue process is restored,and the context information representation is complemented,thereby explicitly providing richer modeling information for the following reply generation.Finally our model can generate more relevant and consistent responses.We try to explore the multi-turn reply generation technology of chatbot from different angles above,and use the combination of objective evaluation and subjective evaluation to evaluate the effect of our model.Experiment results show that each modeling technology we explored imporve the effect of multi-turn reply generation models,which proves the effectiveness of our proposed methods.