Research Of Technology Of Network Security Protection Based On Campus Traffic And Hadoop Analysis

With the continuous development of the campus network,security issues of campus network have become increasingly prominent.Attack event has occurred all the time,which bring about a great threat to the information security of the school.So it is important to raise the level of the intrusion protection of campus network.The college use security precautions and security audits as the primary means,and constantly strengthen the capacity of security monitoring and security warning.Firewall,WAF,anti-virus software and other security mechanisms is currently used by school,but there is still some distance from the national information security level protection requirements.On the one hand,the ability of the school to collect and store the web logs is not enough.Logs are distributed on many servers and need to be cleaned up when the number of logs is large.Hackers will delete the logs when he invaded the server successfully.At the same time,the shcool has not analyzed and mine the logs information,the security value of the huge log can not be reflected.On the other hand,Although the campus network have used the Web intrusion detection system(WAF),through observation of the operating conditions of various sites of campus network,attacks still occur from time to time.The web log of the campus network is quickly generated and distributed on many servers.The huge log records all users' requests for access to the campus network.The amount of information is very rich,not only contains the normal user's request,but also contains hacker's malicious request.A thorough analysis of the log can detect security incidents and discover security vulnerabilities and the attacked sites of campus network.At the same time,the complete web log provides an important clue to analysis the attacker's behavior.Therefore,it is of great significance to improve the security of campus network by collecting and mining the complete web log.Through the research and analysis of the actual network environment and in view of the lack of the capacity of log collection and storage,this paper proposed a new method of traffic collection,and built hadoop cluster to store the log,which provides the basis for the analysis of security events and solves the problem that the storage capacity of a single server is insufficient,and avoids the risk that the hackers delete the log.At the same time,this paper uses Hive technology to map the data,so it's possible to query the log in a short period of time.In view of the situation that the school has not analyze and mine the logs,this paper proposed three kinds of security event mining methods based on the research of attack mode.Based on the above study,this paper proposed the scheme of the security event mining system based on the school web logs,The implementation of the scheme fully utilized the safe value of the logs and provided the support for the campus to enhance the ability of network security protection.The Web Intrusion Detection System(WAF)currently used by the campus network is rule-based.One of the serious drawbacks of the rule base is that it can not recognize unknown attacks and only recognizes known attacks,so the school need improve the ability of security precaution.Based on the research of the Uniform Resource Locator(URL)and the machine learning algorithm,and taking the characteristics of the campus network HTTP log into account,this paper proposed an anomaly detection model of URL request resource based on One-Class SVM algorithm and an anomaly detection model of URL request parameter value based on clustering algorithm.By training the URL request resource field and URL request parameter value field which extract from HTTP log,the normal user behavior profile is built and can be used to judge whether the new request is abnormal.The model has the ability to detect unknown attacks and made up for the shortcomings of the rule base.Based on that,this paper proposes an improved scheme of web intrusion detection on campus network.Finally,In order to verify the effectiveness of the campus security event mining system based on Web log and the Web anomaly detection model based on machine learning,this paper proposed the testing scheme and analyzes the test results.The result proved that security event mining system based on web log can effectively collect and store the log and can mine a large number of security events,it also proved that the anomaly detection model based on the machine learning algorithm has a good detection effect.On this basis,this paper proposed an improved scheme of campus network security protection.