Reserarch And Implementation Of Security Ststem Based On WEB Log Mining And Vulnerability Discovering

The security problem of Internet application has been the focus of people’s attention, vulnerabilities exist in any website,and it brings a great threat to the site and users. To the web site developers, to make the Web website system as much as possible to reduce the vulnerability become a top priority.What’s more ,web application log is an important guarantee for the response of the whole system. From the web log can get the system running state, performance index, the user access behavior and various statistical analysis of the data. Through the web log mining,analysis of log response of user behavior, pumping from the user preferences and that the data can be used in such as search optimization,recommendation system, web structured analysis and so on. In the research of log mining, it is difficult to adapt to the large amount of log data on a single computer. This paper focuses on the study of multi-source log format, and completed in Hadoop platform, the web log log pre processing, cluster analysis and user behavior set should be used for digging holes, the main research contents are as follows:First, the analysis of the vulnerability discovering and intrusion inspection system of domestic and foreign research present situation,combining with the theory of log mining, put forward to Web Log Mining Based on user behavior, and use it as the basis for mining site loopholes to prevent intrusion of feasibility.Second, did a lot of research on the web log system, analysis the different log format, and the role of the log has carried on the detailed elaboration, of log mining and log mining steps and algorithm are described. For system with analysis and processing capacity of massive log, this paper combines the Hadoop distributed framework for the in-depth study and practice, and web log off-line analysis and delay,expounds the advantages of MapReduce programming framework and ELK stack in parallel for log processing.Third, to the mining system clustering analysis algorithm are described, analyzed the advantages and disadvantages of various clustering. Combined with the machine learning framework Mahout provided by Hadoop, the K-means clustering algorithm provided by Mahout is realized. The log files are processed and the results are verified.In the end, in this paper, the Hadoop ecosystem is given based on the prototype of the vulnerability mining system based on log mining.