Memory-Assisted Redundancy Reduction In Networks Packets

With the explosive growth of the Internet traffic,data compression can be a pow-erful technique to improve the efficiency of data transfer in networks and consequently reduce the cost associated with the transmission of such data.The very high amount of data traffic produced and transmitted daily around the world call for new techniques to reduce the considerable amount of redundancy in the traffic.Recent studies confir-m that most of this redundancy is present at the packet level.In other words,packets generated by the same or different sources and destined to the same or different clients contain significant cross-packet correlation.In this dissertation,we analyze the minimax redundancy of universal compres-sion,which is the redundancy measure of data compression redundancy.From theory aspect,we analyze the redundancy of data compression for mixture data source mod-el based on the redundancy analysis of single source model.We proposed a novel clustered memory-assisted data compression strategy to reduce the redundancy of da-ta compression.To prove the theoretical analysis,we implement the memory-assisted data compression model on theoretical data.In the simulation,we propose a memory-assisted compression framework.We imply theoretical mixture source model on this compression model and prove the validity of this assumption.In network traffic,we utilizes the memory-assisted data compression framework on the packet-level mem-orized context to reduce the inevitable redundancy in the universal compression of the payloads of the short-length network packets Then,we investigate the constructed data and real traces with data visualization and the practical aspects of implementing cluster-based memory-assisted compression,such as the assistance of side-information and the proper size the training set.We analyze the drawbacks of previous K-means training selection algorithm and propose a non-parametric training selection algorithm for training packet selection.This improvement is verified with 3%?8%more re-dundancy reduction.In the final simulation of memory-assisted data compression on real traces,we demonstrate that,when compression speed is not an issue,our proposed non-parametric clustering algorithm with Lite PAQ compression algorithm can achieve nearly 70%traffic reduction on real data gathered from Internet traffic compared with ASCII encoding.We also explore the trade-off between the memory-assisted com-pression speed and performance using different clustering algorithms and compression methods.This dissertation presents the first attempt to suppress network traffic data at the packet level using universal compression,and hence,several directions exist for the continuation of this work.In the following,a list of the more important future research directions are presented.On one hand,the cascade of deduplication and data compres-sion is a promising direction to reduce both the redundancy in transmitted chunks and bit sequences.On the other hand,we can also investigate the complicated real traces under real-time scenario in wired networks and wireless networks,which larger scaled clustering analysis is needed to simulate.Apparently,the packet-level network traffic redundancy reduction is significant in future networking development.