An Adaptive Network Data Collection System In Software-Defined-Network

With the development of Internet,IoT(Internet of Things)and Mobile Internet,network services and facilities have become a vital part of infrastructure services,which greatly convenient our lives.However,when we enjoy the benefits and convenience of Internet,a great number of network attacks,malicious traffic and intrusions may cause huge loss.As one of the most convenient way to learn the network environment,network data collection provides data basis for real-time network monitoring,network performance evaluation,network attack detection,traffic analysis,filtering and billing.However,with the growth of 5G and network data scale,traditional network data collection methods,such as NetFlow and sFlow,usually depend on traditional IP network architecture.As the result,it is difficult to formulate accurate and effective strategies for network data collection.Thus,it's hard to adapt to emerging network needs and challenges.The evolution of SDN(Software Defined Network)network structure brings new changes for the improvement of network data collection.Based on the quality of the network awareness and programmability,a lot of methodologies are proposed to achieve real-time network data collection.However,current methods still cannot recover network traffic,which is vital for network administrators and researchers to execute network monitoring,traffic analysis and network measurement.At the same time,these schemes also collect too many redundant data,which bring great pressure to data collection equipment.In the sight of the above problems,we propose an adaptive network data collection system.We aim at drawing an efficient,intelligent and adaptive network data collection mechanism,which can not only ensure the recovery capability of network situations,but collect the traffic data as less as possible.Specifically,we first evaluate the availability of nodes and links according to SDN's perception of network topology and traffic.Then we propose a node selection module to find suitable network nodes for network data collection.By this way,we can realize an adaptive and intelligent distributed network data scheme,which is quite different from traditional centralized and distributed schemes.Finally,with the consideration of the redundancy of elephant flow,we propose a sampling method to collect less packets in elephant flow,which is required to guarantee the recovery capability of it.The sampling method is of great significance in many DDoS scenarios and the real network which are dominated by elephant flow.In order to verify the effectiveness of our method,we implemented a prototype of network data collection,which is based on an SDN controller,Floodlight.Then we simulated the corresponding network and traffic scenarios by the topology generation tool Mininet,and the traffic simulation tool scapy.In our test experiments,we proved that our prototype has good performance in terms of operation efficiency,CPU and memory consumption,storage resource consumption,stream size recovery and threat perception.Finally,we can verify the effectiveness of our scheme.