A Key Techniques Research On Natural Language Oriented Knowledge Search

With a remarkable increase in the number of information on the Internet, people are having an even higher demand on information retrieval. How to retrieve what the users require out of the great mass of structure diversified information precisely and rapidly has become a problem which needs to be solved urgently. Traditional search engines, with Google as the representative, involve an information retrieval technique based on key words matching. With such a technique, traditional search engines manage to serve the users by collecting, decoding and categorizing information with certain strategies. However, traditional search engines still have some demerits like unfriendly API, complex searching process, return information with too large amount but little pertinence. With the awareness of the existence of such demerits, this research is to introduce an user-friendly natural language oriented knowledge search system. This system would offer different knowledge search services corresponding to different questions.For factual questions, the system would offer knowledge search service based on structured knowledge graph data. Problems exist within such a search model, for example, the search grammar and data structure are both rather complicated. Therefore, how to map unstructured natural sentences to structured queries comes to be the core of this model. A layered approach and a relation patterns mining algorithm is proposed in this research to map natural mentions to entities and relations in knowledge graph respectively. What’s more, this research also proposes algorithm based on manual defined templates and algorithm based on semantic relation extraction to map unstructured natural sentences to structured queries.For non-factual questions, the system would offer knowledge search service based on community question answering. This research solves the problem from two aspects: algorithm based on similarity between questions and algorithm based on similarity between question and answer. An improved WMD algorithm is proposed to measure the similarity between questions. And kinds of deep learning architectures are proposed to measure the semantic similarity between question and answer in common methods. Based on methods which only measure semantic similarity, this research also considers the co-occurrence feature to improve the situation when answers are too short and there are huge difference between question and answer.