Network Measurement Data Storage Management System

With the rapid increase of network scale,network measurement has become an important prerequisite for understanding user rules,discovering network faults,allocating network resources and other network management behaviors.It is more urgent to store and manage the large amount of network measurement data.Based on the design to achieve a used in network measurement data storage management system as the research subject,analyzes the common network measurement data storage management system structure,combined with distributed database technology,design a network measurement data storage management system based on the fast greedy modularity maximization algorithm.Its main work is divided into four parts.(1)This thesis studies the commonly used network measurement data storage management system,and analyzes the shortcomings of this system,such as large storage space requirements of data storage nodes,high processing capacity requirements,and large single point failure losses.Based on this,a network measurement data storage and management system with low requirements for storage space and processing capacity and easy recovery of failure is designed.Combined with the characteristics of distributed database,this thesis uses distributed structure,scatter the nodes of cached data in the network,reduce the requirement of storage and processing capacity of single point,enhance the robustness of the system,and improve the query efficiency of the system as much as possible.Based on the above research,this thesis designs a network measurement data storage management system based on the fast greedy modularity maximization algorithm,and introduces the structure of the system and the functions of each module.(2)Under the designed system architecture,the selection of system cache nodes is analyzed,and based on the characteristics of various clustering algorithms,the fast greedy modularity maximization algorithm is used to solve the problem.We first perform a clustering process on the network,and then use Floyd algorithm to calculate and select the cache nodes in each cluster class.The selection method of the cache node is simulated,and it is verified that the system using the algorithm has great advantages in query efficiency and synchronization cost compared with the system using Modulo algorithm,the system using centralized storage and the system not using the cache node storage.(3)Under the designed system architecture,the system query cost is optimized.The application of each recommendation algorithm is analyzed and an improved user-based collaborative filtering algorithm is used for each cache node to recommend data to each other.We simulate the recommendation algorithm and verify that the system using the recommendation algorithm has certain advantages in query cost compared with the system of traditional user-based collaborative filtering algorithm and the system not using it.(4)Based on the designed system framework and simulation results,a network measurement data storage management system based on the fast greedy modularity maximization algorithm is implemented by using Java language,and the realization method and processing process of each module are introduced in detail.The system was deployed in a real network,and the availability of the system's cache node selection function,data query function and data recommendation function was verified.