Research Of Network Intrusion Detection And Network Transaction Anomaly Detection

With the development of internet-related technology,all kinds of industries flourish,which presents some challenges to the network security problems,and brings a crisis to the intrusion detection for the attacking network security problems such as the virus intrusion,these are problem of bottom layer and upstream,while for the downstream,under the protection of upstream network security,there are some problems,such as the payment link of online transaction,which is not used by the user but embezzled,which brings a challenge to the transaction anomaly detection.Specifically,on the one hand,network intrusion behavior is more and more diverse,and the data scale is larger and larger,which brings the challenge to intrusion detection,on the other hand,relying on the Internet for online transactions is becoming more and more popular,and the problem of transaction fraud is increasing,which brings losses to e-commerce,banks and individuals.So far,many machine learning methods have been used in intrusion detection,but the traditional machine learning methods tend to solve a certain scale of classification problems of intrusion data.In recent years,deep learning methods have been developed,and deep learning can solve large-scale data problems,so how to apply deep learning to intrusion detection can be discussed.In the aspect of abnormal transaction detection,traditional methods such as misuse detection rely too much on negative sample set,and the cost of obtaining negative sample is too high.In the present method,the Markov chain model in anomaly detection is the main one.This method can summarize the user portrait well,but the periodic change of the user's transaction behavior leads to the increase of the false alarm rate,Therefore,it is not entirely suitable,in view of this,this aspect still has the research value.The main research work of this paper includes:(1)Introduce the background and research status of network intrusion detection and abnormal transaction detection,and mainly summarize the intrusion detection system,includes its classification and related technologies,then,concept of abnormal transaction and the relevant theory of risk control system for detecting transaction are introduced.(2)To improve the detection rate of less data category and reduce the training time,reduction of feature dimension can solve the problem that the training time is too long when the data scale is large.Therefore,the sparse auto-encoding network(SAE)is studied by using NSL-KDD dataset,Adam function is used as an optimizer to reduce the dimension of sparse auto-encoding network,and the structure of binary tree is constructed by integrating with Light GBM algorithm,it is used for classification.Finally,analyze the compared algorithm and do a conclusion by experiment..(3)Because over time,user behavior will change periodically,which will lead to problem of concept drift and increase false alarm rate.To solve this problem,research and put forward a method of user's order behavior profiles and expand the diversity of users in future transactions,to avoid the problem that user behavior is influenced by historical transaction data to a certian extent.finally,analyze result and get a conclusion by compared experiment with other algorithms.The innovations of this paper are as follows:(1)An intrusion detection method consrtucted by structure of binary tree based on stack sparse auto-encoding(SSAE-Light GBM)is proposed.Firstly,the data is divided into five categories,and then two sampling methods are combined to solve the problem of unbalanced data distribution,especially for the category of data of small amount of data,the above pre-processing method divides the large-scale data into several parts and train their respective models,and then uses sparse auto-encoder network to reduce the feature dimension,this method of network structure can extract more deep features from the original data and save the time of dimensionality reduction at the same time.Finally,the training model is integrated with Light GBM(lightweight gradient lifting model),and the Light GBM model has more advantages in training time than other models that can achieve the same performance.The experiment uses NSL-KDD dataset to verify thatthe accuracy,recall and F1 value,the final results are 87.42% ? 98.20% and 91.31%,respectively,which are better than the compared algorithm and save computing time.(2)An method of improved diverse anomaly transaction detection based on order relation is proposed,which is based on the directed logic graph of defined user behavior portrait,and combines information entropy and cosine similarity to increase the diversity of users' transaction behavior.The diversity coefficient can overcome the concept drift problem(reduce the false alarm rate)to a certain extent.Finally,through the experiment,we measure the stability level of consumption amount,which is classified into three categories: HS(High Stability),MS(Mid Stability),LS(Low Stability),their accuracy,recall,area under ROC and running time were measured,and the results were 91.75% ?94.07% ?83.50% and 1800 s,respectively.Although the aspect of the running time has a little short,the overall performance was better than the compared algorithm.