Question Classification Based On Deep Learning Model

Question classification is an important part of question answering system.It can effectively improve the performance of question answering system.Question classification is a kind of short text categorization.It can assign a label to each question,which represents the category of the answer to the question.In question answering system,problem classification can restrain and filter the results of question answering system and intelligent search.In recent years,the question answering system has attracted a large number of researchers.As one of the key technologies of question answering system,the question classification has attracted more and more attention.The early rule based classification method is not universal.Then,feature extraction based on feature extraction and machine learning need to make the extraction of feature strategy artificially.The process of feature extraction and classification is relatively independent,which leads to the accumulation of error in the classification process.Recently,deep learning has been widely applied in question classification.In recurrent neural network,the disappearance or explosion of the gradient in the recurrent layer is a hot topic at present.Although the long short term memory(LSTM)has been greatly improved compared with simple recurrent neural network(SRN)in solving long-term dependence problems.However,in practical applications,long sequence input with complex dependencies can not be effectively handled.The essence of this problem is that the error of the recurrent layer will be multiplied by a value greater than 1 or less than 1 when backpropagation.Aiming at this problem,we improved the structure of the long term memory model and got a new model Att-LSTM.In our model,continuous hidden states of previous time step are inputted into the current step.The novel architecture is capable of capturing local features and learning long-term dependencies.However,continuous hidden states cannot be inputted into the current time step because the dimensions don't match.Thus the attention mechanism is introduced to these hidden states.In addition,the information control method has also been changed accordingly.Experiments show that the performance of Att-LSTM improves significantly in learning long term dependence and capturing local features.In Adding problem,the MSE of Att-LSTM is basically unaffected as the sequence length increases.In pMNIST,the classification accuracy of LSTM is increased by 4%.In practical applications,it is difficult for a single neural unit to learn the data expression.Therefore,multilayer neural networks or multiple networks are often used to form complex networks.In this paper,we design a hybrid network framework CNN-AttLSTM,which extracts text features hierarchically.A sentence is consist of several words.A word is made up of some characters.Therefore,in this hybrid network framework,the text features are extracted from the words features,and the word feature are extracted from characters features.The convolution neural network(CNN)is used to extract the local features of words in the text,and the recurrent neural network Att-LSTM is used to learn the dependence between words and words in the text.The experimental results show that the method can effectively deal with the problem classification task without making any tedious feature rules.The classification accuracy of the method on TREC is increased by 1.6%,and 1.5%is increased by MSQC.