Research On Key Technology Of Community Question Answering System Based On Deep Learning

With the continuous development of the Internet,the community question answering system has become an important platform for Internet users to obtain information and share knowledge.There are tens of thousands of new questions posted every day in community question answering sites such as Baidu Know,Sogou Ask and Zhihu.At present,hundreds of millions of existing questions and answers have been accumulated in the community question answering system.How to better reuse these existing knowledge has important research value for better serving Internet users and improving user experience.The main work and research contents of this paper are as follows:First,a sentence similarity calculation model based on deep learning is proposed.This model mainly considers semantic information and structure information in sentences.Syntactic analysis is used to improve the semantic representation of the word vector model.Based on the semantic vector,the syntactic information that the word undertakes in the sentence is added,and the similarity of the word form and the sentence similarity are added to the sentence presentation layer.Features,combined with a deep learning algorithm to calculate the semantic similarity between two sentences.Using web crawlers,a large number of community problem pairs were collected and annotated by the Baidu short text similarity API.Finally,a model that can be used to calculate the similarity of two sentences was trained.Second,a method of question type judgment based on template mining is proposed.This method uses the standard words in the synonym word forest to replace the words in the sentences,then uses the entity recognition method to identify the entities in the sentences,and uses the entity concept dictionary to replace the entities in the sentences with the corresponding concepts to generate a semantic template.Finally,the template is filtered by information entropy,and a high-quality question type template is excavated.At the same time,using the question and answer training set provided by NLPCC,an automatic labeling method was proposed.Thirdly,on the basis of researching the key technology of community questions and answers,a set of open-field community-based question and answer system based on deep learning was designed and implemented.The system can find answers to similar questions in the community question and answer corpus and return them to the user based on user input,which can significantly improve the user's knowledge acquisition efficiency.In addition,since the method proposed in this paper does not limit the application field,it has good versatility and is not limited by the field.The question and answer system can be answered through more and more question and answer corpora so that the question answering system can answer more questions and serve the user better.