The Research And Implementation Of Mobile Communication Network Big Data Management

In order to help network optimization staff quickly grasp the whole mobile communication network status, a data management platform for mobile communication network is in urgently demand. Facing massive data of mobile communication network, a data warehouse for the mobile communication network in Zhuhai is established in this thesis. In the process, the main contribution of this thesis is as follows:(1) Increasing the transmission speed of massive data. A data transmission method named binary serialization data compression is designed, which increases the speed of data transmission by reducing the size of data. First, the data to be transmitted is converted into a two-dimensional table structure, and then changed to be a binary sequence with Run Length Coding. At last, it is compressed. The experiments show that this method significantly increases the speed of data transmission. Compared to transferring data file directly, this method takes 98% less transmission time. Meanwhile, in the data receiving end, relative data fusion method is designed to automatically merge the same type of two-dimensional tables, which reduces the size of data to be stored further.(2) Designing the storage of massive data in the condition of limited local hardware resources. First, a time-tree database storage structure in which each database stores the data of one day is built, then a two-stage storage structure including local and remote devices is designed, and a time-period based database auto-scheduling algorithm is proposed. Combined with the data access periodicity of users, the databases which are most accessed by users are taken to local storage, so that users can access data quickly.(3) Enhancing query performance of traffic statistics. For query process of traffic statistics, a dynamic programming based query optimization method is proposed. In order to speed up index query, a calculation graph is built, which using some index calculation results to simplify the whole process of index calculation. The experiments show that there is more than 90% improvement with the usage of this method. The traffic statistics query performance is improved significantly.