Research On Question Classification Based On Weak Supervision And Deep Learning

In the information society,the demand for fast and accurate access to valuable information is increasing rapidly.Intelligent answer System is a new technology to realize quick feedback of questions and answers,and the market demand of intelligent answer System is multiplied.Question classification is an important foundation of intelligent answering system processing,and it is the key to affect the system performance.In question classification,there are two main problems.The first is that the existing question corpus resources are insufficient,which can not meet the needs of open domain question answering in various scenarios.This problem tends to lead to biased research results and poor general applicability.The second problem is that it is difficult to extract the key features.because the question itself is short and semantically complex,which is easy to lead to low classification accuracy.In this thesis,the process of question classification is divided into two steps.The first step is to expand the corpus with the method of weak supervision to solve the first mentioned above.The second step is to extract the semantic features of the question with the deep network to solve the second mentioned above.Among the steps,the process of expanding the corpus as following: first,calculate the TF value of the keywords of each type of question set respectively to select the words,which can reflect the category semantics as the category keywords;then,using the synonyms in the synonym dictionary to replace the key words of the category in the original questions to get a large-scale high-quality corpus.Deep learning is used for semantic feature extraction,The concrete realization is to train a “long-and-short-term-memory-network” with attention mechanism.Finally,we get the abstract representation and semantic features of questions,and further complete the classification of questions.We select the appropriate model parameters to train the abovementioned deep network classification model.Then,using the same data set,the performance differences between different classification models are compared to verify the effectiveness of this method.The experimental results show that the extended corpus based on weak supervision and the deep learning model based on "the attention mechanism of the answer toward question" can achieve significant results in question classification.