The Design And Implementation Of IP Traffic Activity Database

With the development of network technology.Large-scale network security issues have become increasingly prominent.These malicious acts undermined the order of Internet threat to information and property security of Internet user,even has a serious threat to the national security.Faced with new security threats,the traditional security technologies seem slow and inefficient.Technology in the field of big data has been used frequently in network management and network security situation awareness.Off-line network traffic analysis based on big data storage is widely used.Before network traffic off-line calculation this type of technology will store the huge network traffic data.IP traffic activity database is a massive database stores netflow.It is necessary to choose the appropriate database.This thesis studies the mainstream NoSQL database.Netflow generated by CERNET backbone is massive and durative.In order to choose underlying database.This thesis designs the database performance test scheme and chooses HBase as the underlying database system finally.After choose the underlying database.System functional design is required.In this thesis,IP traffic activity database can be divided into four parts: data writing module,data querying module,data lifecycle management module and Operation module.Data writing module is the foundation of establish active database.In order to satisfy the requirement of writing netflow record into database in real time.This thesis studies the data writing mechanism of HBase,optimizes the data writing performance of HBase using caching technology and parallel technology.Finally implement the writing module of activity database.The active database need to provide real-time multi-dimensional data query service,but the query performance of column data in HBase is very poor,this thesis designed and implemented the secondary index scheme,solved the performance problem of querying data from HBase effectively.Afterward,according to the requirement of the application system,activity database implements the data query method in a variety of conditions.In order to facility the users using activity database,database provides a common data query interface and a friendly web inquires page.The persistence of the data source and the limited storage space of the activity database are contradictory.In order to ensure the normal operation of the activity database,this thesis designs The data lifecycle management scheme to manage the storage space.It includes the design of data lifecycle management model,the design of storage value evaluation model,the periodic data aging processing scheme and the periodic data delete scheme.The active database will operate in the real network environment,in order to make activity database working regularly,activity database designed and implemented the operational function module.This module has the following four functions: host State monitoring,process monitoring,exception management,log management.In order to display the communication behavior characteristic of target IP address.this thesis designed and implemented the visual module analysis the communication behavior of IP address from time dimension and space dimension.As an important application system of active database,IP address characterizing tests the query performance and system stability of active database.