Research On Real Time Computing Consistent For Massive Data

Concept of big data and cloud computing as well as the related technologies has developed rapidly along with the advancement of social and the industry need. In the field of sensor networks, weather analysis, key areas of satellite monitoring, nuclear and electromagnetic simulation calculation, the generation of data and processing requirements has been into the mass level, and making full use of these data can greatly improve the productivity and competitiveness of enterprises, institutions and nations.Nowadays the research of massive data still fundamentally belongs to the big data processing technology research category. Although the domestic and foreign enterprises and the research institutions have certain research on distributed storage and computing, when face to the problems of massive data distributed storage and computing, especially the less study of data consistency in the data center collaborative computing environment, they can not adapt to some features which are heterogeneity stronger, more computing and complexing in the process interaction relationship when multi-data center collaborative real-time computing.This paper makes some research on consistency of supporting massive data real-time computing requirements, and the main work is as follows:Through the studies of supporting massive data distributed storage architecture, the copy of the massive data backup management strategy and the data consistency mechanism of multi-data center for collaborative computing, we can analyze the characteristics of huge amounts of data real-time computing environment and the data consistency requirements;Aimed at the requirement of data consistency issues in massive data real-time computing and based on the research of data center network technology, we can put forward the platform architecture of collaborative computing which can support the massive data real-time computing, and also make a planning of collaborative computing platform processes;Through the analysis and comparison of the consistency strategies in mainstream computing model, combining with the reality of multi-data center collaborative computing platform, basing on the Paxos algorithm, it builds the data consistency architecture program for input data, intermediate and output data, and solves the problem of data storage in massive data consistency real-time computing environment, it’s backup and the using.On the basis of the above studies, it establishes the multi-data center collaborative computing platform by using the OpenStack, and verifies the effectiveness and correctness of the data consistency architecture plan when facing to the massive data real-time calculation in this paper by basing on the platform.Results of this study can be applied to massive amounts of data in real-time computing environments, and provide support for related industries.