Research And Implementation Of Distributed Persistent Caching System

With the development of information technology, Applications is having higherrequirements for performance of background server and the amount of data is alsogrowing bigger, traditional relational database has been unable to meet today’s massivedata-scale applications. In the era of big data, people always hope that there is aKey-value storage mechanism, just like HashMap in memory handling a large numberof Key-value pairs, improve data search, modify speed, So NoSQL technology hasmade considerable progress.So far, NoSQL has been mainly used in two aspects. The first aspect is storing thedata by column as a storage server for database. The second aspect is storing all the datain memory as the front-end cache of database, which can be used to cache query resultof database reducing the number of database access and improving the response speedof dynamic Web applications. Most of the traditional caching systems are based onmemory storage in order to achieve higher performance, and their data persistence is notperfect. So these systems may be limited to memory capacity. Also they will lose all thedata and be impossible to restore when systems break down. To solve the aboveproblems, this paper proposes the idea of persistent storage of cache.In this paper, after analyzing and comparing the traditional caching system, we firstpoint out the shortcomings and deficiencies of traditional caching system and applyLSM-Tree theory to improve persistence of caching data by giving up random readingability and accomplish sequential writing ability to achieve efficient persistent storageof data. Secondly, this paper introduce the scalability of distributed systems and relatedkey technologies such as data consistency in distributed systems, data synchronizationmechanism, data segmentation rules etc., and implements a distributed Key-Valuepersistent caching system SSDB (Sorted Set DB) by referencing the Stand-alonepersistent storage system LevelDB of Google and Amazon’s highly available distributeddynamo architecture. At last, this paper uses Bloom-Filter with other techniques tooptimize SSDB’s performance and test the SSDB comparing with Redis by showingtheir performance result in Highcharts plugin. The results show that SSDB has just aslight decrease in reading and writing performance comparing with Redis.