Estimation And Recommendation Algorithm For Network Traffic Data

Network monitoring data plays a vital role in various network engineering tasks.We usually use two-dimensional matrices or higher-dimensional tensors to record this type of data.Due to the limitation of objective factors such as the monitoring and transmission cost of network data,the data of the tensor model we built is often incomplete.Tensor completion is used to solve such problems.By using a small part of the observed data to fill the missing data in the tensor model.At present,tensor completion has been widely used in capacity planning,load balancing and other network engineering.The traditional tensor completion algorithm has a good effect on the data following the normal distribution,but has poor performance when the network monitoring data are highly skewed with heavy tails.In network engineering such as anomaly detection,researchers are more eager to accurately identify whether the top-k data are more likely to be abnormal traffic.Therefore,this paper focuses on two major issues,and the main work and innovation points of this paper as follows:Aiming at the problem that conventional tensor completion algorithm has a smaller recovery value for BIG-data in a dataset.This paper proposes a new non-negative tensor completion algorithm,which uses expectile regression and applies it to the loss function to replace conventional symmetric least square,and a corresponding update rule is proposed to update the factor matrix,and finally use these factor matrices to form a complete tensor.The experimental results show that this new algorithm has a significant effect on the accurate recovery of BIG-data.In viewing of the anomaly detection and so on practical network engineering scenario,this paper presents a method to recommend the top-k traffic in a network.Qualified these traffic data into 4 levels and introduce the method of ordinal classification,reconstruct the data in the origin tensor in the using of the generated sub-tensor,finally recommend abnormal traffic in each time-slot.Experimental results show that this new top-k recommendation method has obvious advantages over the method of recommending the traffic after the tensor is completed with the traditional tensor completion algorithm.