Research And Apply On The System Of Open Source Intelligence Mining Oriented Special Application

Intelligence is a process that analyzes the existing knowledge to obtain the useful information. The development of information technology gives intelligence work new opportunities and challenges, which has changed the ways and methods to obtain information from fundamental, and the opensource intelligence has already been the primary way to acquire the intelligence.In this paper, we introduced the related technologies and tools in OSINT mining,which include web crawler, text processing and database technology, and these technologies are the hot reseach fild in big data at present. How to apply these technologies to analyze OSINT is not only a simple application of existing tools, but also boosting the related filed. In text data mining, the traditional vector space model method cannot take advantage of semantic information, it shows its shortcomings in some scenes. In this paper, a better model for the main title text feature representation, and through the experimental comparison and analysis show that the method in text feature dimension reduction and text clustering effect. Intelligence element is the most important component of intelligence, which contains main components of intelligence. In this paper, two methods to extract intelligence element are applied, and the simulation results indicates the advantage and disadvantage of the two methods, and we also propose some advanced strategies which is satisfactory. The clustering for intelligence text is anthor topic for OSINT research. The purpose of clustering is to make the intelligence have the same topic into the same category, and make the intelligence has difference topic into the difference category. Most OSINT is from Internet, so online clustering for OSINT is necessary.This paper has imporved the Single-Pass algorithm and the result is content according to the result of simulation. At last, this paper shows the application of OSINT in military through the actual programming to build a prototyping system, andsom key parts of the implementation are introduced, which include the method to implement web crawler,deal with defferent kinds of web page and design database. And this paper shows some results for OSINT that use the system. Through the above work, this paper indicate that OSINT has important value in practical application.