The Research On Data-object Oriented Page Based The Big Data

Nowdays, Big Data is more and more popular in the world. There are many re-searches in collection, management, calculation and display of Big Data. A big data processing application always has its own features, especially the feature of data reuse in accessing memory. However, the original page replacement algorithm are too inflexible to find differ in unique applications to adjust its replacement algorithm. Furthermore, the memory thrashing phenomenon is more serious as data amount rises.There are twe kinds of ways to improve the gap between the original page re-placement algorithm and the memory access feature in big data application. One way is reserving some original kinds of page replacement algorithms in the system, and choose different algorithm for the specific application. But this method can not change the es-sential problem in original page replacement algorithms. The other way is building a new data processing model according to the memory access feature. Although it always needs to static analysis, it is strong in flexibility and versatility. Base on the second method, we have done three aspects of main work as followed:At first, we propose Data-object Oriented Page(DOOP). Based on original page management, we add a new page management layer to the system in order to sense the data reuse in big data application. According to the concept of object-orinted, DOOP builds the data-oriented object for the data structures which have features of data reuse. At the same time, we design the data-oriented object page replacement algorithm and its unified interface.At second, we create a new kind of page replacement algorithm based on DOOP. As original page replacement algorithms couldn' adapt to big data applications and da-ta-oriented object page very well, we design a user-directed non-fixed dynamical page replacement algorithm and supply the usage.At third, we demonstrate the working process of DOOP model and its page re-placement algorithm on k-means application. The simulation result proves DOOP is ef-fective in page faults compared to LRU page replacement algorithm.