Web-based Data-mining Enterprise Competitive Intelligence System Design And Realization

With the diversity of information in the Internet, the massive Internet information provides a rich source of intelligence information, so the intelligence workers pay more attention to the open source intelligence. This article aims to construct an Enterprise Competitive Intelligence System (ECIS) which assemble intelligence automatic collection, intelligence analysis and intelligence service to one whole, according to open source intelligence collect technology and analysis theory research.This thesis expounds Enterprise Competitive intelligence theory systematically, puts forward an ECIS construction based on web open source data mining and divides the ECIS to three parts: Intelligence collection subsystem, Intelligence analysis subsystem and Intelligence service subsystem.Then, The thesis gives the subsystem design solutions and the key technology to develop the System. In the intelligence collection subsystem, the thesis takes the theme focused crawler method and meta-search data collecting technology to ensure the collected data relative to given theme. And then create a crawler to traverse web links and download data automatically. With the support of web pages content extraction, PDF or Office documents text conversion technology, the collected semi-structure data is converted to structural data. The thesis takes"I-Match"algorithm to detect and remove duplicate documents, then index and store the data. To sum up, this segment builds a automatic data collecting and processing platform for the ECIS. In the intelligence analysis subsystem, the thesis use Chinese text information processing and mining technology to do some research as intelligence classification analysis, then constructs a Enterprise Competitive Intelligence product set with the analysis results .The Enterprise Competitive Intelligence System, based on Internet open source data, can facilitate the enterprise to make the most of the massive internet information warehouse and discover competitive intelligence. It makes a great importance of building enterprise competitive strategy and improving their core competitiveness.