Research And Application Of Bidding Data Mining Based On Graph Clustering

Human society of information data is growing at an unprecedented speed in the age of big data. Most of the data is the format of web pages. Because the Web information resources has a semi-structured, real-time, heterogeneity and the characteristic of discreteness and so on.How to dig and analysis the Web resources to extract the specific topic information of people needed, has become an important research topic. This article selects the online data as the research object in the field of bidding.As a common data structure, graph is the composition of node and the connections between these nodes. It has become modeling tools for a variety of complex objects and the link of these objects. Online bidding information contains rich data, the data can be used to build the complex relation of the tenderer and the bidder network diagram. Graph clustering analyzes those clusters which are internal closely and external loosely by the clustering technology. Graph clustering has been widely used in various areas such as the discovery of a community in the social network and the detection of the complexes in the protein. In the tenderer’s relationship with the bidder network diagram, graph clustering method can be use to dig up the valuable information.This article first to the graph clustering and data mining technology were studied, on the basis of the combination of graph clustering and data mining method is applied to the field of bidding. Set up the bidding data mining method based on graph clustering system and model which adopted the Louvain algorithm as the bidding data mining algorithm. And then, apply the research results to the bidding system design and implementation of data mining. In the thesis is mainly around the graph clustering analysis method, data mining technology,information extraction method and design and implementation of bidding data mining system.The main research results are as following:(1) A method is presented for a kind of web information extraction especially the form data. This method is introduced in this paper through artificial keyword as a tag, and then learning the samples of web pages.Rules of web pages data extraction is obtained through by summed up the rules of key words in the path of the web labels. Compared with previousDOM information extraction method, more able to adapt to the non-normatively of web data structure.(2) Through the bidding application binary graph clustering analysis method in data mining can found some clusters. All these clusters have features of industry background. Also learned the bidding network conditions required for the community to form. The tenderer and the bidder is regarded as the vertices in the bipartite graph, The relationship between the tenderee and the bidder through the undirected weighted graph, With the figure of the edge right on behalf of the degree of close relationship between them, use of vertices between rights can construct a similarity matrix, thus to do clustering analysis.(3) On the basis of the above research contents, for specified in the bidding website bidding data mining system was designed and implemented. The functions of system adopts modular design, can better maintenance and extension, meet the requirement of web crawl tasks And the requirement of the bidding data mining.(4) The system function module and data mining algorithm respectively experimental verification. the experiment results show that the system is able to crawl the valid data from the specified website. After compared with other algorithms, prove Louvain algorithm is suitable for bidding data mining algorithm.