Research On RDF Stream Data Partitioning Algorithm Based On Graph Model

In recent years, large graph data，in particular multi-billion vertexes scale graph dataappears in large numbers, which is a new challenge to the graph data management field. Thestorage of graph data directly determines the efficiency of data access, data query and datamining.How to divide and process large graph database, to improve the system throughput andoverall performance becomes the urgent need to solve the key issues in the field of large-scalegraph computing.In the present situation, the basic framework of the big graph storage is distributed storage.Graph partitioning theory and method provides an effective way to solve the problem. But theincreasing complexity of parallel computing, existing graph partitioning methods and modelsstill have many deficiencies. This paper studies the problems in graph partitioning algorithm，andproposed a graph data stream partitioning algorithm based on heuristic, whick can get aacceptable result in the limit time consumption.The main work are as follows:First of all, this paper introduced and analyzed the development of the theories and methodsof the graph partition, and classified and summarized the classic graph partition method,including: grid partition method, heuristic method, the spectral clustering method, and multilevelgraph partition method and so on.Secondly, this paper introduced the graph data stream concepts and defined the model ofstreaming graph partitioning and heuristic partition function strategies. Then this paper proposedthe graph data stream partitioning algorithm and implementation process, which focused on thegraph file and RDF file.Finally, this paper verified the validity of the graph data stream partitioning algorithmthrough the experiments of the several real RDF datasets, and comparing with the METIS(aMultilevel Graph Partitioning Algorithm) method and the hash partitioning method. The resultshows that the graph data stream heuristic partition algorithms reduced the number of cuttingedge and improved the performance comparing with hash partition method; reduced the timeconsumption of the partition processing function comparing with METIS method,and thealgorithms can be better suited in large graph databases and incremental graph database...