Design And Realization Of IP Actvity Table Based On A Distributed Infrastructure

Contents of this thesis belong to the project of CERNET backbone upgrade in the 3rd phase of 211 program. To store CERNET backbone netflow, the thesis establish IP activity table after researching data management model, inserting mechanism, query mechanism and space management. Consequently, users could acquire IP records generated by CERNET backbone and analyze IP activity based on these IP records. The primary task of this thesis is to design and realize data management model, data inserting algorithm, real-time query algorithm, periodical maintenance algorithm and system functions of IP activity table.Data management includes researching data life cycle and data management. CERNET netflow is persistent and massive while IP activity table storage space is limited. The thesis researches the life cycle of netflow in IP activity table, and designs life cycle model based the research of netflow life phase transformation. The thesis also designs data management model based on life cycle model, realizing netflow dynamic life cycle in IP activity table.The data inserting algorithm of IP activity table includes designing how to analyze CERNET backbone netflow and how to insert the flow. As CERNET backbone netflow uses dynamic templates, the thesis designs a method to analyze the flow. In addition, considering the massive size and persistence of CERNET backbone netflow, ElasticSearch is selected as the database to store inserting data.The real-time querying algorithm of IP activity table includes designing user interfaces, query methods and real-time query. To ensure users to interactively analyze IP activity table, The thesis designs and realizes simple and clear user interfaces. In addition, thesis designs single, multiple and massive query functions to satisfy users’ complex query. Finally, considering that IP activity table is distributed and CERNET backbone netflow is massive, The thesis designs real-time query method to support users’ real time-query.The Periodic maintenance algorithm includes periodical merging, periodical compression and periodical deletion. Considering that CERNET netflow is persistent and massive while IP activity table storage space is limited and considering dynamic data cycle model, the thesis designs periodical merging method, periodical compression method, periodical deleting method, which completes data life cycle transformation and storage space compression.The functions of IP activity table system are inserting function, query function, periodic maintenance function, monitoring function and exception management function. The system realized first three functions through referring the research results of data inserting, real-time query and periodic maintenance. As there may be some instabilities and exceptions when running, the system also designed and realized monitoring function and exception management function.