Research On Answer Matching Techniques Based On The Attentive LSTM Network Model

In recent years,the spoken dialogue system based natural language understanding has become one of the hottest topics in the field of artificial intelligence.Many influential famous spoken dialogue systems(such as Apple Siri,IBM Watson,Amazon Echo,etc.)all include an intelligent question answering module.Question answering system is a complete system,which can be divided into three components of Query Analysis,Document Retrieval and Answer Matching.Answer Matching is a very important component for question answering system in that whether the answer is correct or not directly relates to the quality and performance of the whole system.As for the answer matching task,it mainly depends on feature engineering,linguistic tools or external resources to choose the correct answer.We extracted question and answer text features using human strategies,then matched them,so as to choose the given answer;sometimes linguistic tools(such as syntax tree,dependency tree,etc.)should be introduced to extract linguistic features of questions and answers.However,traditional approaches for answer matching have the following several problems:(1)human strategies for feature extraction has some subjectivity,which cannot fully understand the problem;(2)to get better effect,we need to keep adjusting and optimizing feature extraction strategies,making flexibility lower;(3)the introduction of linguistic tools maybe caused very high complexity of the system.With the development of deep learning technology in image recognition,machine translation and other fields,deep learning models have been proved to have great advantages in data preprocessing and feature extraction.Based on the general process of matching questions and answers and a series of challenges faced by deep learning models,a deep research on the application of deep learning models to solve QA matching problem has been conducted in this paper.The main research work of this paper is as follows:(1)for the problem that deep learning models are hard to be trained caused by high dimensional data representation in the field of natural language processing,Word Embedding mechanism,where each word is mapped into K dimensional vector space,was used in this paper.That is to say,each word is represented by a K-dimensional real-value vector rather than a 0 and 1 sequence of high dimension.Word embedding is used as the input to our model because words with similar meanings generally are close to each other in the vector space,which can improve the accuracy of question answer matching task.(2)this paper describes the challenges of deep learning in the field of natural language processing,such as long distance dependence issue and gradient disappearance issue.Linguistically,the dependency relation between core words in a sentence is a common phenomenon.Usually,they are not adjacent in a sentence but rather away from a certain distance.The gradient disappearance issue is a phenomenon in which the gradient would gradually decrease to 0 after n time steps in the process of the execution of back propagation algorithm.(3)Based on previous researches,an Attentive LSTM based answer matching model was designed to extract features of questions and answers.In the model,Attention mechanism is added in order to get the semantic encoding of inputs by calculating the attention probability distribution,which can avoid information loss and redundancy in the process of feature extraction,and highlights the influence of key words on the expression of feature vectors.