Massive Qos File Processing And Data Analysis

After several years of development, Shanghai Telecom IPTV users have increased from 50,000 in 2008 to 1.6 million registered users with nearly one million active users now. Because of such a huge number of users, operation and maintenance department receives thousands of failure appeals every day, every day they are under enormous pressure. If you want to achieve the desired control effect, we are required to accept the mass users quality data in real time (hereinafter referred to QOS data), and Make a timely response. The current network management system cannot do this. Meanwhile with the triple-network fusing, the development blue ocean appeared, but suffer from "mass data, lack of information," the received data cannot be effective analysis, a lot of meaningful data are left on the shelf, if we can do effective analysis on the mass QOS data, we will bring great convenience to future operation and maintenance work. Therefore, the research and development of a platform which can do effective analysis on the mass QOS data in a short time and do some Multidimensional analysis work will have a certain practical value and theoretical significance.This paper studies the design of a kind of unified data processing and analysis system, called IPTV quality of service system (IPTV QOSS). The goals of IPTV QOSS.It receives all of Shanghai Telecom IPTV STB reported massive QOS user data.it can be process 500000 QOS data file within 30 seconds. After the completion of processing storage, the system uses pre-created index model to judge the mass QOS data, each user QOS indicators obtained result is normal or not. Finally after we got a completed user index data, the system can carry out multiple dimensions analysis on these data. This paper analyzes three key part of the system design:massive data processing and storage, the user index model creation, multidimensional data analysis. Finally this paper talks about the future development of IPTV QOSS:more intelligent threshold adjustment and more direction data analysis...