Research On Network Abnormal Traffic Detection Based On Machine Learning

Nowadays,with more and more diverse network use scenarios,it is very important for social order,life and economy to ensure the safety of network system.This thesis mainly detects the network situation and attack identification by detecting the abnormal network traffic.After studying the system log and URL in the network,this thesis chooses CNN algorithm to train the detection model,and designs a platform to provide detection model training internally and detection task scheduling externally.The main contents of this thesis are as follows:1.This thesis designs and implements the network abnormal traffic detection platform,which provides the training retraining model,task management,data upload,model deployment and other functions for the browser login users,and provides the exception detection of log class and URL class for the external access system.At the same time,the model retraining function provided by the platform can mix the new types of abnormal data detected into the training samples,so as to train a more adaptive model to the changes of the environment,which provides the basis for the external system to automatically adapt to the new abnormal.In particular,the platform limits the access of users to the detection task,thus providing a guarantee for the data security of the external system.2.This thesis introduces advanced technologies such as zookeeper and NAS service,and will carry out training task and detection task deployed on different machines,which reduces the load of system operation,and zookeeper can effectively manage the status of each node.NAS service can centrally manage the data distributed on each machine,which makes the network abnormal traffic detection platform an efficient,monitorable,effective storage platform with large storage capacity.3.In this thesis,CNN text algorithm is applied to the log class file detection task.The algorithm does not care about the context content of the log but only about the key features,which is consistent with the characteristics of the log exception information only related to the constant string,and can effectively detect the log exception information.4.In this thesis,SA-CNN algorithm is applied to URL detection task.Based on token partition and LSTM language model,the algorithm retains the word order and semantic information of URL well.At the same time,the algorithm also introduces the attention mechanism,which gives high weight to the malicious code part of the URL,and detects the attack of Web specific character specific number combination.5.Complete system function test,external interface test and algorithm model index test.The platform can be used smoothly and the external interface can be called smoothly.Compare CNN text with CRT to verify the superiority of CNN text in log detection.CNN text is used to detect the data sets of two different systems.It is verified that CNN text provides a log detection function with system universality for the network abnormal traffic detection platform.By comparing SA-CNN with LSTM,the superiority of SA-CNN in URL based attack detection is verified.