Research And Implementation Of Data Cube Techniques For OLAP Analysis Of Network Security

Internet has become the key information facility of our country. With the rapid development of Internet technology, vicious attacks against network information system tend to be distributed, complicated, indirect and scalable. Thus it's impendingly required to research for new technology to accurately acquire, monitor and analyze the security situation of large scale network system in real time. Figuring out methods to acquire and interpret current security state of the network and disclose the underlying changes to grasp the general security situation is where our study begins.OnLine Analytical Processing (OLAP) is an important technology to do integrated analysis on the massive and complicated network monitoring data. By rapid, consistent and interactive access of information from various possible viewpoints, OLAP allows the analysts to observe data in depth, providing greate flexibility.Efficient computation of data cube is the key to support OLAP analysis. To get OLAP capability, we have to precompute the whole or at least partial data cube in order to reduce the query response time. The core problem of our study is to find out scalable techniques to compute partial data cube under restraints of storage space and computation power to get a balance between data cube's computation&storage cost and query response time.Since the acquirement, monitoring and analysis of network security situation is often required to be done in real time, we proceed to the study of OnLine Analytical Processing on rapid changing streams. With streams, the data cube computation has a more rigorous restrict on computation time and storage space. Studying partial materialization techniques of stream cube under restraints is the emphases of our work. We summarize our work as follow.First, basic concepts of data cube are introduced with discussion of its implementation schemes followed.Second, the characteristics of OnLine Analitical Processing on data streams, and the design requirements of stream cube are analyzed. Then a hierarchical tilted window model, which decreases the size of stream cube to adapt to the computation and storage constraints, is proposed.Third, a new method for partial materialization of stream cube, a Dwarf-based stream cube framework called StreamDwarf, is proposed. The corresponding computation algorithms, including incremental update algorithm and query algorithm, are developed. Then the algorithms are implemented and testing results are presented.At last, a prototype for network security situation analysis, which is based on StarOLAP platform and is capable of multi-dimensional, multi-level and integrated analysis on the massive network monitoring data in real time, is developed.