Research And Application Of Distributed Caching System Based On Redis

Entrusted by the government and enterprises of Wulingshan District,the construction and promotion of a new agricultural digital platform will be carried out.As the new agricultural digital platform was put into operation in many enterprises in the Wulingshan area,the platform has begun to face great challenges due to the rapidly increase in the number of enterprise users and consumers.Frequent operations by a large number of users can cause tremendous pressure on the database,which will bring about long response time of system and even system crashes in serious cases.How to improve the platform's high concurrent processing capability is an important research issue.The current common solution is to adopt the Redis distributed cache system.In order to improve the response speed of the platform in a high concurrent scenario,the hotspot agricultural data needs to be distributed to each cache node of the distributed cache system.Secondly,considering the promotion of the platform along the Wuling Mountain area,the distributed cache system developed by the laboratory will inevitably face expansion in the future.How to reduce the cost of data migration by each cache node when the system is expanded is also important.Therefore,the paper analyzes the data distribution of distributed caching system and the research status of data migration,and improves the existing data distribution and data migration algorithms.Mainly completed the following work:(1)The following two improvements are made to the virtual node-based consistency hash algorithm.Firstly,the WF_CH algorithm is proposed for the problem that the data distribution of each node is not balanced by using the virtual node-based consistency hash algorithm on uneven performance of laboratory nodes.By designing a weight factor strategy,the weight factor is used to caculate when assigning virtual node mapping to physical nodes.The better performing nodes cache more data by assigning more virtual node mappings,which solves the problem of uneven data distribution due to uneven server performance.Secondly,the DM_CH algorithm is proposed for the problem that the hit rate of the system cannot be guaranteed when the server node is down or the cache is invalid.By designing a double mapping strategy,the data is cached on two physical nodes by mapping to two virtual nodes,which solves the problem of a drop in cache hit rate when a physical node crashes or the cache is invalid.(2)The existing method of randomly selecting nodes for data migration is prone to a large migration cost.The cost-sensitive data migration algorithm selects the node with the least migration cost for data migration.However,the algorithm does not consider the size of the data migration,and cannot guarantee that the node with the lowest migration cost has the lowest average migration cost.Aiming at this problem,this thesis proposes an improved data migration algorithm.When the migration occurs in the cluster and selects the migration object,the average migration benefit of each cache node in the cluster is calculated,and the migration objects are selected in order,which reduces the data migration process.After experimental testing on the Redis cluster,the improved data migration algorithm has improved both the migration time and throughput.(3)The original architecture of the new agricultural digital platform was improved,and the distributed caching mechanism was applied to the platform module.Finally,testing for the performance of the improved architecture in the platform.The experimental results show that the improved architecture of the platform has a better improvement in system response time and TPS.