The Design And Implementation Of Distributed Call Detail Record Query System Based On Hadoop Framework

In the telecommunication industry, monthly details of call record are typically a kind of big data. These data even can reach billions level which often cause severe test for storage space and data analysis. Meanwhile, details of call record, as a direct channel of communication between the telecommunication service operator and the customer, have so great significance that it is always a hot topic on how to provide convenient, flexible and reliable call details for all stakeholders. The Hadoop architecture is an newly cloud computing technology, which has many advantages such as open source, cross-platform, high fault tolerance and so on. It can be easily used to build a distributed cloud computing platform which can achieve full utilization of computing cluster, storage capacity and deep analysis of big data. Based on the Hadoop distributed architecture, this paper analyses, designs and implement the call detailed record management system. The system test and pilot show that query reliability, efficiency and scalability of system have improved significantly.Firstly, this paper investigates the main problems in telecommunication industry, analyses the advantages of hadoop framework in resolving these problems, and then the paper make clear the functional and non-functional requirements.After requirement analysis, the paper furtherly discusses the general architecture. The paper puts forward system’s object, and describles business process, technical architecture, functional architecture, data architecture and system interface. This system includes four subsystem in functional architecture, which are data storage, data pre-processing, data querying and CRM interface. There are seven kinds of data tables in its data architecture, and three external system are getting into this call record management system from interface.Thirdly, this paper makes a particular explanation about the detailed design of the four subsystems from data process, business process and sequential chart, also this paper make a explicit report about the key techonlogy such as the single point of failure in distributed clusters, high availability of redis cache, data compress model and pre-process abnormal handle mechanism. Finally this paper design the deployment solution to arrange different subsystem on different node, which can help to improve the data avaibility and query efficiency.Finally, the paper elaborates how this call detailed record system is developed, and gives some examples including call record download, monthly call record query, data mergement and so on. Also this paper introduces the test and pilot of this system, and the test result shows that both the query reliability, efficiency and extensibility are improved significantly, while the waiting time of starting the Map/Reduce in the pre-processing of data still need to be optimized furtherly...