The Key Techniques Of Sketch-based Network Measurement

With the rapid expansion of the Internet,network measurement,as an important means of monitoring,recognizing and mastering network behavior,is one of the research hotspots in the field of computer network.Network measurement can provide support for congestion control,traffic accounting,capacity planning,anomaly detection and other applications,which is very important for daily network operation and network management.At present,the commonly used network traffic measurement technology mainly includes sampling based and data flow based.In data flow based technology,network measurement based on sketch has become the focus of the industry because of its high computation efficiency and accuracy.However,sketch is a compact probabilistic data structure,and the measurement based on sketch still faces how to get smaller estimation error in a given memory space.In order to solve this problem,more and more complex sketch structures have been proposed,but there are often some parameters that affect the estimation error in the complex structure that need to be set.The existing solutions only rely on simple rules or human experience to set the parameters,so it is difficult to achieve the optimal measurement.In order to solve the above problems,the main work of this paper is as follows:(1)Inspired by the multi-layer sketch structure,a new network measurement method based on three-layer sketch is proposed.The sketch structure consists of three parts: accurate statistics layer,buffer layer and approximate statistics layer.It can provide smaller estimation error than the existing solutions in a given memory space.(2)This paper proposes a method of memory allocation between sketch layers based on reinforcement learning.The neural network model can adjust the proportion of memory occupation between different layers of the sketch and allocate the memory between different layers of the sketch reasonably,so that the network measurement results can achieve higher accuracy.In this paper,real Internet datasets are used to verify the proposed three-layer sketch method and the memory allocation method between sketch layers.The experimental results show that,compared with the existing measurement methods,the network measurement method based on three-layer sketch and the memory allocation method based on reinforcement learning can significantly improve the accuracy of network measurement.