Reasearch And Implementation Of Key Technologies In Distributed System For Log Data Acquisition

In network management, how to improve the quality of service and guarantee the safe operation of the network is an important research topic. Network management needs the support of network log data, thus the log data acquisition technology has become a key research content. This paper mainly presents some explorations on the network monitoring point deployment algorithm and log data acquisition algorithm in log data collection.For the problem of network monitoring point deployments, in order to resolve the difficulty in changing the monitoring points of the original network after extending the network topology structure, an incremental selection algorithm for incremental network monitoring points was proposed. The greedy algorithm was optimized by the proposed algorithm to obtain the approximate solution of fewer vertices, which used the degree of vertices in the network as the greedy-choice strategy for the weak vertex cover of the graph. While calculating incremental network monitoring point sets, only the extended network topology was used to obtain the corresponding monitoring point sets of the new network. The obtained incremental monitoring points could be directly added to the collection of the original network monitoring points for the new whole network monitoring point set and the cost of ranging the whole network monitoring points could be reduced. The experiment results show that the number of vertices of the whole monitoring point set obtained by the proposed Incremental selection algorithm is basically the same with that of a new monitoring point set generated by recalculating the whole network topology structure in the new network. The proposed algorithm can be effectively applied to the deployment of actual network monitoring points.In case of network monitoring points have been deployed, in order to reduce the network bandwidth occupied by transmission of network monitoring data, a data acquisition algorithm is proposed. The algorithm analysis and preprocessing of data collected on network monitoring terminal for network monitoring data acquisition, according to the collected data changing situation, the transmission time interval of the monitoring data is adjusted dynamically, then get better balance between the network bandwidth occupied by the data transmission and the amount of data transmitted. The experiment results show that the algorithm reduces the number of network monitoring data transmission and the amount of information, the result of data fitting can be a good fit to the changes in network status gentle or severe on network management terminal.The research into the key technology of log data collection in this paper makes a contribution into the study of network monitoring deployment and data collection and provides support for network management.