Research And Implementation Of Arrange Process System Cater To Massive Data Process

With the development of information technology, as a new data form, data stream is used widely in many fields. The scale of data stream in many problems becomes more greately. With existing conditions, the data stream manage systems can hardly meet the needs of massive data process. So, study and improve the existing systems for the target of massive data stream processing becomes a pressing project.Base on the existing TelegraphCQ system, when the scale of imput data exceeds the bottleneck of CPU, we set up hierarchy framework of data stream processing. Let many TelegraphCQs process the imput data stream at the same time. Achieve the aim of effective process to massive data stream.The main content of the thesis are as follows.1. On the basis of analyzing and intergrating the research status of data stream and the principle and structure of TelegraphCQ, expould the bottleneck of CPU which massive data stream processing meet, put forward the idea that set up hierarchy framework of data stream processing base on existing system.2. Analyze the basic principle, function and design idea of hierarchy framework of data stream processing in details, and explained the work circuit of system.3. Study the main algorithms of system, and analyzed their advantages and disadvantages in details.4. The paper illustrates the design and realization of main function parts, and analyzes the procedure of their work together. Give the detail execute steps of processing the massive data stream.5. Make a test to the system on functions, and give simple illustrate about test results.The study of the paper illustrates, with existing condition the hierarchy framework based on TelegraphCQ can realize the massive data stream processing, and return the results to user. This solution relieves the bottleneck of CPU which massive data stream processing faced.