A Stream-processing Traffic Matrix Filling Approach Base On Contrastive Predictive Coding

The scale of the Internet is expanding,and the massive data generated by it has brought great pressure to normal network operation and maintenance.To carry out network operation and maintenance tasks such as anomaly detection,anomaly root cause analysis,and traffic prediction,we have to get complete monitoring data.Assuming that the number of nodes in the network is9),the cost of a whole network measurement is(9)~2),but it is not acceptable for large-scale networks.Some surveys have shown that network data has low rank property.By using the low rank property,we can infer the remaining data from some node data.This kind of method using"sampling and filling"architecture has gradually become the mainstream research direction in the industry and has accumulated some achievements.The current mainstream network filling algorithms are mainly divided into two categories:mathematical modeling methods based on matrix filling or tensor completion,and deep learning methods based on neural networks.Although the above two types of methods have high filling accuracy,there are still some contradictions that have not been resolved,such as the correlation of network data is dynamically changing,the characterization learning speed is slow,the use of conventional loss functions has gradient imbalance problems,and existing methods Stream processing cannot be performed or the accuracy of stream processing is not high.In response to the above challenges,this article proposes the following related solutions:1.Aiming at the problems of dynamic changes in correlation and slow characterization learning speed,contrast predictive coding is used for training.Contrast predictive coding learns the correlation in network data by taking the data to be filled as positive samples and irrelevant data as negative samples.At the same time,since the contrast predictive coding module includes an autoregressive model,it can also capture the dynamic changes of network monitoring data.Experiments show that since no additional auxiliary network is required to reconstruct the input,the learning speed of contrast predictive coding is faster than previous methods,and the Spatio-temporal correlation of changes can be captured more quickly.2.Aiming at the problem of gradient imbalance,a composite loss function using weight transfer is proposed to guide data generation.When the commonly used MSE(mean square error)is used for network data,it will cause gradient imbalance.Aiming at the characteristic that network monitoring data obeys heavy-tailed distribution,a robust relative error with high accuracy is proposed.Based on this,it is further proposed to train the generative model by using the compound loss function of weight transfer.The composite function combines the advantages of rapid improvement of the accuracy of the mean square error in the initial training stage and high robust relative error accuracy.The generated model can quickly and stably maintain the filling accuracy.3.Aiming at stream processing requirements,the architecture proposed in this paper has high filling accuracy in stream processing mode.Since the comparative predictive coding model can capture the Spatio-temporal correlation in the historical sequence,it also benefits from the improvement of model convergence speed.The data set Abilene shows that the method can run in stream processing mode,and can capture dynamically changing data relevance,and has high filling accuracy.