Design And Implementation Of Target Relation Graph System Based On BigData

With the advent of the “big data” era,we are being surrounded by data,and traditional methods of data processing and analysis are no longer able to meet the exponential growth of data and the processing of a variety of mutually dependent data.The mining and application of Big Data has penetrated into every field of today.With the growth of various relational networks,the research of network graphs has become the focus of attention of all walks of life.This thesis aims at the particularity of the relation graph of military communication networks.The graph relationship data in military communication networks is different from the graphs of social relationships.It has features such as falsification,real-time,and concealment.The relationship between military objectives is also more complex and depends on traditional network graphs.The PageRank method is extremely difficult to accurately characterize the importance of the goal.In order to better identify the importance of different nodes in the military communications network,aiming at features such as false information and covert information in the relations between the various levels of military communications networks,the MilitaryRank method for military communications networks is proposed.At the same time,the traditional community detection algorithm that depends on the expansion of important nodes have issues such as community turbulence,so a community discovery method for elastic expanding clusters of centers is proposed.Build a target relation graph mining system framework based on Hadoop platform,and implement network graph node sorting and community detection.The research content and innovations of this paper are as follows:1.The overall architecture of the Hadoop platform was studied to understand the connections between the data storage layer,the resource manager layer,and the scheduling layer.At the same time,it collects and summarizes various military communications network rules,understands the association rules of military communications networks,and some common false covert information transmission methods.Build a framework of target relationship graph mining system based on big data platform,including a series of processes from data storage to final visualization.2.Combining the PageRank algorithm of traditional web page sorting and the characteristics of military communication networks,aiming at the existence of false and hidden characteristics of communication nodes in military communication networks and the difficulty of communication time,a MilitaryRank algorithm for ranking the importance of military communication network nodes is proposed.Conditional factors such as time,position change,and in-degree,theoretically studied the rationality of the algorithm and the complexity of the algorithm,and conducted experiments through simulation data.Simulation experiments show that the algorithm is better than the traditional web page sorting algorithm.The use of time,access and other conditions to achieve the effect of quickly and accurately tap the important goals of the network map and sort them.3.Combining the PR removal algorithm proposed by Baumes,the Lancichinetti node fitness function,and the modularity function proposed by Newman,and Aiming at the shortcomings of strong community randomness and frequent monster communities in traditional community detection algorithms,a community detection algorithm based on an elastic center expansion subgroup is proposed,which fully considers the influence of different core nodes on community size.Real experiments and simulation experiments show that compared with the traditional community discovery algorithm,this algorithm has better robustness in dividing the number of communities and has better processing results when dealing with highly overlapping communities.There is a lack of military network community data in the experiment,but experimental results are expected to be used in military cyber community discovery.