Design And Implementation Of The Open Source Intelligence Extraction System Based On Site Relationship Network

In recent years,underground industries have become more mature.In this industry chain,communication platform existing in the form of sites provides theories and technological supports for channels to privacy data,click farming ways and emptying plans.This information can be easily detrimental to the business and brand images of internet companies.In order to understand and grasp the information of these sites,and provide supports for digging the work of underground stations,designing and realizing the site-level intelligence system has theoretical and practical meaning.Targeting this demand,this paper makes link relationship as entry point and starts from known underground sites,seeks related sites,evaluates the content and behavior,designes and realizes the open source extraction system based on site relationship network.In order to formulate a reasonable open source intelligence extraction sche me,this paper analyzes and studies the concept and general process of open source intelligence extraction and extracts three core links of the data acquisition,processing and analysis.In this paper,an open source intelligence extraction model is design ed based on site relation network by citing the theory of relation network analysis.First of all,the network data is abstracted as the graph structure,the graph structure analysis method is introduced and the community is extracted.Then the influence m odel is established for each node in the community and the influence time series trend is formed according to the history.Finally,the intelligence output rule is worked out and the sites that may have threat are output.In order to meet the clustering re quirements of theme and correlation,this paper proposes a community detection algorithm based on the similarity of site theme and the network topology on the basis of the modularity algorithm and comprehensively weighs the theme tendency and correlation o f nodes in combination of the greedy algorithm to detect the community structure in the network.On the basis of the above research,this paper defines the system framework logic and the data abstraction to design and realize the open source information extraction system based on the site relation network,including the core function modules such as data acquisition,theme feature extraction,relation network construction,site influence trend analysis,etc.The system firstly collects the related basic data in the network,extracts the keywords that can describe the theme attributes in the data,then constructs the relation network from the angle of correlation and tendency,divides the relation network into the community,helps analyze the characteristics of the site,finally establishes the time series model fitting the data change in relation network to reach the purpose of outputting the site-level intelligence.In conclusion,with the deep analysis of site relationship network,the open source intelligence extraction system has been designed and realized.Test and application results show that the system can help dig the underground stations from the open source information,with high level of feasibility.It can satisfy demands of enterprises and reach the expected values.