| The rapid development of Internet technology to Web applications become the most widely used information systems,but hackers use technical means to obtain Web server ROOT authority,stealing business important data.These problems make more people began to pay attention to enterprise application security issues.Most threats have fixed characteristics,but some threats vary depending on the architecture and programming language of the system.Traditional defense tools such as application firewalls are based on known vulnerabilities for security testing.But for the dynamic web page attacks are generally no fixed rules.So the need to establish a user rules can analyze the security audit system to strengthen the protection of Web applications.Large Web applications produce a lot of logs every day,we need to dig out in these massive log of enterprise security valuable content.Now a single node data mining system has been unable to meet the requirements of mining massive Web log.The use of distributed framework analysis and storage log can solve this problem.Hadoop Yarn is a general resource management system,so that the cluster in terms of utilization and resource management has a higher efficiency,so it is in log analysis has a better application.And its file system has high throughput and high fault tolerance and other characteristics,it is suitable for storing large amounts of log files.It provides a distributed programming model for large data processing Map/Reduce can also be convenient to prepare the processing of massive log file code.This paper studies the problems in log processing and analysis,and implements a secure audit system using Map/Reduce,Spark Streaming and PFP-Growth based on Hadoop Yarn.Mainly including the collection and storage of logs,the establishment of user behavior rules and audit interface display,the system than the previous audit program has greatly improved to solve the massive log mining process of space and time efficiency problems,improve work efficiency. |
PDF Full Text Request