Design And Realization Of Scalable Platform For Network Flow Analysis

With the rapid widespread of network, our life has become much more convenient. Network is everywhere whatever we do and wherever we go indeed. Just as the coin has two sides, it is not so secure for us to use network sometime. It is not strange for us to know that our information on the internet may be stolen, modified and even falsified illegally.The network also brings convenience and benefits for some organizations and departments. But some of them are anxious about the security of their private data which includes their user information and financial records because of the existence of hackers and attackers. As the reasons listed above, network security is becoming more and more important. When we talk about the network security, some may think of the firewall. But as we know that the firewall can't check the hostile attacks hidden in the codes, though it can shield the unexpected access. So to avoid this defect, it's necessary for us to do research on IDS (Intrusion Detection System).The network has become an indispensable part of our life. Our dependency on the internet is growing no matter whoever we are and whatever we do. It's much more necessary for us to manage the network and know the status of our network. As an important factor of the network management, the network monitoring becomes research hotspots. However, the detection of the network flow is an effective way for network monitoring. Though there are some comparative mature tools such as sniffer, they are not so convenient for normal users for different application. So research on the extensible platform for the detection of the network flow is also much more important. Users can develop different application tools on the extensible platform for different application backgrounds. In this process we can also reduce the cost of the system. We can use this platform for the normal detection, such as some important parameters of the network, the status of the network, the dubious attackers and so on. In a word, we can use it for different applications. So, it has a new and deeper meaning for the maintenance of the network security and the improvement of the quality of services provided by the network.The main goal of our paper is the research and realization of the extensible platform for network flow measurement. Our research is mainly on the basic existed network monitoring system, and then do the research and realization of the extensible platform for network flow measurement. The users can apply new application plugin to this platform, and this platform is based on libnids which provides API for the network monitoring application developers. This platform can be used to measuring the rate of the accessing flow and detecting the dubious flow items. We also can use this system to record the size of the flow packets, classify the packets by size and type of IP or TCP protocol. Users also can know the status of the network by user interface browser. We also propose an algorithm called CHHFR (Cashing Heavy-hiiter Flows with Replacement) to identifying heavy-hitter flows fast and accurately. It demonstrated that the algorithm can work well and can achieve the expected effects.