The Technology Of Aggregation And Feature Analysis Of Diversified Network Data With Tags

Nowadays,the Internet is rich in information and content,and people can use a variety of network services on the Internet.However,the ever-changing network poses a challenge to network supervision.There are various types of labeled data in the network,and their acquisition methods and forms of existence are different.It is difficult to conduct unified management and analysis.At present,almost all research work or mature platform products on network data aggregation are aimed at collecting specific types of network data.To solve this problem,this article will design a platform based on Docker and Kubernetes that can obtain corresponding data from different targets in the network,and can store,statistics and analyze these data.Kubernetes makes containers the carrier of data aggregation,storage,and analysis processes,and these containers are uniformly scheduled by Kubernetes.However,Kubernetes cannot meet the requirements of data aggregation.According to the above problems,this paper designs corresponding solutions and implements a complete system.The specific work of this paper is as follows:(1)A cluster bandwidth resource management strategy is proposed and a scheduling process is designed for data aggregation tasks,and the Kubernetes task scheduling method is optimized,solving the problem that tasks that need to be completed in time when the cluster bandwidth resources are insufficient cannot be executed smoothly.(2)A storage process is designed to be suitable for multi-source heterogeneous data,improving the container platform's processing capability for multi-source heterogeneous data,solving the problems of data backlog due to data peaks and file data loss due to the statelessness of containers.(3)A scalable and highly usable multivariate network tag data aggregation and characterization platform is designed and implemented,which meets the needs for collecting and characterizing Internet service information and Internet device information,and also carries out actual data aggregation,statistics and characterization..