The Design And Implementation Of Communication Data ETL Service Based On WCF

Based on de analysis of mobile network data, network optimization is important to work on the existing network equipment to be adjusted so that it can serve the mobile phone users better. The network equipment and service providers need a data analysis platform to analysis daily log files generated by the base station, to find the direction the network needs to be adjusted to and the existing problems of the network.The research on commonly used mobile data in network optimization and needs of staffs working on network optimization found that the network optimization work is still done manually, and considerable part of the daily work of network optimization staffs is spend in dealing with a wide variety of log files. Data processing program is not perfect, a lot of common analysis functions are done manually, which cost a lot of time and repetitive work. Based on the demand of network optimization staffs, summarizes the data processing requirements in the mobile communication field, the design of the communication data based on ETL WCF service comes. The system open outward the call interface by using WCF with windows service as its host, which will be convenient for client to refer data services and call the system functions. By using the data storage based on table schema and horizontal segmentation, numerous communication data in different formats could be represented and stored in SQL Server database. On the basis of detailed analysis on demand, data import progress is divided into four procedure:list procedure, download procedure, parse procedure and save procedure through which the import process will be detailed tested and the exceptions be detected. Developers are allowed to write plug-ins for secondary development approach, increase the data processing functions, extend the system directly without modifying it. In order to facilitate data reading, meet the data transparency, some attempt is made on the restrictions of query and the flexibility of search function. In the case of system scalability considerations and reserve computing expansion interface, two different data read mode are provided, based on which provide some common computing functions for mobile network data. In the design and implementation of the system, fully considering the needs of the client, the efficiency of the system is optimized through pipeline, multithreading, delay loading and so on so that the response efficiency is ensured and the data processing efficiency is improved.So far, after test the system in Dalian China Mobile, mobile network data ETL service has been submitted to Ericsson for internal use as the network data processing support project, assist network optimization engineers to analyze network data and has been fully affirmed by the relevant staffs, has practical value.