Research On Algorithm Improvement Of Talking-robot Based On CVAE

With the rise of deep learning,all fields of natural language processing have undergone tremendous changes.As for the field of talking-robots,it is also different from the past: there are more and more generative talking robots rather than those based on match.They can satisfy users' needs of communication,improve users' stickiness,and also undertake some tasks-robots.The research of generative talking-robot has become one of the hot spots.In the human context,everyone has a fixed personality.People tend to reply to sentences with distinctive personal style and keep their personality consistent unconsciously.The traditional generative talking-robot based on the seq2 seq model has made great success in the past,but the model often generates responses lack of personality or performing inconsistently between the previous and the latter.In addition,the seq2 seq model also suffers from a variety of problems: security responses,poor diversity of responses,long training time……Based on these reasons,this paper will try to abandon the traditional seq2 seq model,and take the CVAE as the overall framework of the model,to make the responses personalized,keep the context personality consistent and improve the diversity of the responses.This paper will propose Trans-CVAE model based on the Per-CVAE model,and the main improvements are as follows:(1)The encoder part of Per-CVAE,RNN is replaced by Transformer.Transformer is one of the most advanced models in NLP field,and it is also the foundation of the pre-training model Bert.Compared with RNN,it can greatly reduce the training time,and the Transformer model performs better than RNN in many important tasks such as machine translation.With the use of Transformer,We can expect improving the encoding quality of the model.(2)Improve the storage and selection of persona,learn from Transformer's multi attention,and enrich the content of global personality vector.We can achieve more accurate personalized response generation by the technique of the storage of persona and decoding methods,and achieve the goal of keeping persona consistent in dialogue at the same time.In the part of selection of persona,Trans-CVAE don't just select the response with the highest score,but choose a response randomly in several best results which meet the threshold.It can improve the diversity of responses.(3)In this paper,Trans-CVAE integrate the solutions of the disappearance of KL divergence in variational autocoder,then analyzing and comparing KLA,word dropout and other methods,pointing out their limitations.Finally,Trans-CVAE choose the method of adding bag-of-word loss,adding another module relying on hidden variable Z to enhance the dependence of the whole model on Z,so as to avoid the disappearance of KL divergence.(4)Integrate the evaluation indicators suitable for the personal dialogue corpus,and improve the personal coverage index.Use TF-IDF value to screen the stop words,so as to prevent the generation of responses which are highly similar to the personal text from being wrongly evaluated as lacking personal coverage.