Research And Implementation Of Dialogue System Based On Knowledge Injection And Dialogue History Extraction

With the continuous development of deep learning and the study of language model matures,dialogue system offering a new approach to human-computer-interaction for the ordinary people and it is more convenient than traditional windows operation interface,but in the existing dialogue system,there are still several problems such as can not understand historical dialogue information,unable to process the complex information behind the proper noun,can only generate simple response,not competent for complex conversation.In order to enable the dialogue system to combine the historical context of the dialogue and understand the background information on proper nouns.The main research contents are as follows:(1)Propose a method of model compression.Through pre-training parameter initialization tasks and module alignment tasks,the bidirectional encoder representation of transformer model(Bert)is compressed into a distillation bidirectional encoder representation model(DistilBert),and the DistilBert is fine-tuned on different tasks.Model compression reduces the number of model parameters,speeds up the calculation of the model for various tasks,and achieves the purpose of improving model training efficiency and shortening system response time.(2)Improved the method of calculating semantic similarity.Use the Word Rotator's Distance(WRD)algorithm to assist the DistilBert model to calculate the semantic similarity between the current input and the historical dialogue.Then extract the historical dialogue based on the semantic similarity,so that the dialogue system could contact the context information in multiple rounds of dialogue,which enable dialogue system maintains the continuity,the purpose of improving the quality of the dialogue response is achieved.(3)Designed a dialogue system that based on knowledge injection.Through knowledge injection,the entity information of the dialogue content is extracted from the knowledge graph,which complements the background knowledge in the dialogue,and has achieved the effect of enhancing the richness of the dialogue content.The encoder-decoder structure solves the problem of the mismatch between knowledge injection and generative dialogue tasks.The encoder uses the DistilBert model for knowledge injection tasks,extracts knowledge information and then encodes the input data.The decoder uses the Generative Pre-Training Model version 2(GPT2),receives the output vector from the encoder,and then generates a response,and finally performs the response sorting through the reversed language model.(4)Designed and implemented a dialogue system based on knowledge injection and dialogue history extraction,introduced the system design,function realization,and described data processing,dialogue history extraction and knowledge injection realization methods.Finally,the system operation test was carried out,which proved that the application can recognize the entity information that appears in the dialogue based on the knowledge graph data,and extract similar sentences from the dialogue history to achieve a smooth and natural dialogue with the user.