Research And Implementation Of Intrusiondetection Based On Host Log

With the increasing of enterprise information construction, host-based log analysis becomes an important techniques to troubleshoot enterprise security issues, which can monitor enterprise security and find irregularities or illegal activities timely, also can minimize enterprise security risk effectively. However, with the advent of the era of big data, how to cope with the huge log analysis effectively is becoming a priority design problem of log analysis system. As such characteristics of log as exponential growth, dynamic transformation, diversity, and heterogeneity etc, which makes traditional centralized log processing based on a single node is already unable to satisfy the requirement of the massive amounts of log analysis. And how to design an efficient and scalable platform for log analysis has become an inevitable choice for enterprise development.This paper designs and implements a massive log analysis platform, which is based on the above actual project background and further study of log characteristics and existing distributed platform. Firstly, the platform collects logs in a free agent method, and which is based on the actual needs of enterprises.Then, analyzing logs, which including internal multiple hosts of corporate customers, network devices and applications, with keyword and correlation techniques. Lastly, according to the analysis results give system managers alarm information and security events orders which made for security operations staff analysis,and provides an alarm report. This paper firstly introduces the research background and the development trend of related fields, studied the host-based instrusion detection system related techniques, which focuses on the free agent remote acquisition technology including the commonly used wmi, smb, ssh, telnet, syslog and other technology. Then, this paper describes the application of data mining in log analysis, mainly studied the advantages and disadvantages of the common customs rule mining algorithm Apriori and Fp-growth, and on this basis, this paper proposed an improved association rule mining algorithm according to the characteristics of huge amounts of data. Also this paper introduces the application of the messaging middleware rabbitmq in distributed cluster system, and in order to improve the processing speed of rabbitmq massive log analysis mainly studied the rabbitmq confirm mechanism and the rabbitmq respectively internal state transition process. Finally, this paper designs and implements the whole system which is base on all above, and has a detail describe of the log preprocessing, decoding, partial correlation analysis.