Research On Heterogeneous Knowledge Fusion Methods In Big Data Environment

Data characteristics and realistic demands have changed because of the large-scale data's links and crossover. The data, which has main features of large scale, cross domain, cross language, cross media, generalization, dynamic evolution and multi-source heterogeneous, is playing an important role. And the corresponding data storage, analysis and understanding are also facing a huge challenge. The immediate problem to be solved is how to use the data association, cross and integration to achieve the maximization of the value of big data. In the big data environment, as the data has different structure, and wide data sources, low density value and the data updates in real-time, knowledge service are also facing a great challenge. Knowledge fusing provides a very effective means that we can use to acquire knowledge and organization knowledge in the big data environment. The value of knowledge is that knowledge fusing can dig out the implicit and valuable knowledge(such as rules, methods, models, constraints, experience, etc.) from many dispersed and heterogeneous data sources and knowledge source. Knowledge Fusion still has many shortcomings. In this paper, the main research work can be generalized as follows:The construction of multi-source heterogeneous knowledge base:This paper proposes a construction model of multi-source heterogeneous knowledge base, and in this paper, the construction method of knowledge base were elaborated. This paper studies the knowledge representation of Resource Description Framework. Knowledge extraction for different knowledge sources takes a structured knowledge extraction methods, knowledge extraction methods of semi-structured and unstructured knowledge extraction methods to acquire knowledge. The knowledge of knowledge base is stored in the form of RDF/XML document.Multi-source heterogeneous knowledge fusion:It has not yet formed an efficient algorithm for knowledge fusing. The purpose of Knowledge fusion is to provide better information services through a series of processing and to obtain high reliability knowledge and discover the implicit knowledge. This paper proposes a knowledge fusion algorithm based on data fusion algorithm. We also propose some corresponding improved methods.The experiment analysis of multi-source heterogeneous knowledge fusion algorithm:We use Hadoop to build our experimental platform and we use the MapReduce framework. And we obtain true probability of RDF triples by using the algorithm of multi-source heterogeneous knowledge fusion. We make the sources of knowledge have higher quality by using the improved algorithm in this paper. The results of the data has been plotted by using MATLAB. After analyzing the results, we know the improved method can effectively improve the performance of the multi-source heterogeneous knowledge fusion algorithm.