Application Research On Deep Recurrent Neural Networks In Automatic Question-answer Under Specific Occasions

It has been the mainstream that applying deep learning methods to deal with nature language processing tasks.The essence of deep learning is to build neural networks,which can stimulate a general black-box function with outstanding generalization performance for identification and prediction.Researches show that system constructed by deep recurrent neural networks work well when dealing with sequential data.For instance,the application of sequence to sequence technics(seq2seq)based on recurrent neural networks in automatic question-answer.However,seq2 seq technics is insufficient to handle user's specialized requirement in specific occasions.First,classical recurrent neural networks unit ignores some state changes and loses some information in the hierarchical connection between input layer and output layer,resulting in incomplete interpretation of text.Second,a large amount of answers generated by seq2 seq are of less meaning.According to problems described above,this paper aims to build an automatic question-answer model that can sufficiently understand user's requirement and generate meaningful answers under specific occasions.In this research,the goal is divided into 2 progressive subtasks: text labeling and automatic question-answer model based on text labeling.The main research tasks of this paper are as follow:(1)Research on the structure of Grid Long Short-Term Memory.For the drawback of classical recurrent neural networks,an improved recurrent neural network unit is proposed.Although common recurrent neural network unit such as Long Short-Term Memory(LSTM)and Gated Recurrent Unit take into account sequential status changes of text,they have ignored status changes in the hierarchical connection between input layer and output layer.Therefore,Grid Long Short-Term Memory(GLSTM)has been brought in to solve the problem.Based on the theory,peephole connection in GLSTM has been proposed.To explore the performance of GLSTM and GLSTM with peephole connection,experiments are conducted to verify performance of kinds of recurrent neural network units in text labeling and automatic question-answer.(2)Research on the way of text labeling based on recurrent neural networks.For text labeling,deep neural networks has been applied in text labeling.To explore which kind of recurrent neural network unit has the best performance in text labeling,an experiment is conducted to compare text labeling model based on different types of recurrent neural network unit.(3)Research on semi-generate and semi-retrieval question-answer model based on text labeling.For automatic question-answer,a modified automatic question-answer model has been proposed.Combining the advantages of the generative model and retrieval model,an automatic question-answer method based on the semi-generate and semi-retrieval model with placeholders is proposed.The core idea is: firstly match the text labeling information with placeholders of the question-answer pair in knowledge base;secondly find out the question in knowledge base that is closest to user's question by calculating semantic similarity between sentences;thirdly put these sentences into the encoder and acquire the value of their similarity in order to determine whether to use the generative model or retrieval model to generate the answer.(4)Design and implement an automatic question-answer system in shopping assistant robot under occasion of shopping guide service.