Content-based Question Semantic Retrieval System

In recent years, with the rapid development of network technology,the Internet has penetrated into every aspect of human society, including the online education. For online education, the question resource is one of the most important resources, and the question retrieval is a strong demand. With the problem of high repetition rate, difficult access and non-standardization, the traditional information retrieval is difficult to filter the information from the vast amount of information to meet the needs of the question resource. Therefore, the design and implementation of a semantic retrieval system for question is very challenging to the scientific research and engineering applications. Based on the research of deep learning and word embedding in natural language processing, we design a content-based question semantic retrieval system. The main research contents are as follows:(1) Based on neural network language model, we focus on the research and implement of word embedding. We use a large-scale text corpus and word2vec tool to conduct training and word similarity computing experiments.(2) Based on word embedding and BLSTM model, we focus on the research of Chinese word segmentation. Firstly, we use a large-scale text corpus to train word embedding model, then propose to use word embedding and BLSTM for Chinese word segmentation. Experiment result shows that our approach gets state-of-the-art performance in Chinese word segmentation on standard datasets.(3) We propose to use word embedding and support vector machine model for Chinese synonym expansion. Based on semantic features of word, we use word embedding as feature to train SVM classifier and thereby expanding words that have the same context.(4) We realize a question semantic retrieval system based on Chinese word segmentation and synonym expansion technology. This system has highly engineering research value and application value.