| In recent years,with the application of cloud computing,big data and artificial intelligence technologies,the Internet business has continued to expand,and the amount of data has shown an explosive growth trend.The limitations of traditional relational databases such as scalability,data model,and computational storage coupling make them unable to adapt to the storage requirements of massive heterogeneous data.Therefore,No SQL databases have been widely used in recent years.As the cornerstone of data storage,distributed key-value storage system is an important part of supporting No SQL databases with elastic expansion,high availability,and data security.Therefore,after in-depth analysis of the research status of distributed storage systems and their security,a distributed key-value security storage system based on the Rocks DB storage engine and Raft data consistency algorithm is designed and implemets.The main work are as follows.(1)To tackle with the scalability requirements of distributed key-value storage systems in the background of massive data storage,a load balancing method is designed.Firstly,in response to the problem of unbalanced utilization of cluster storage resources caused by uneven data distribution in distributed key-value storage systems,a data storage load balancing method based on Range partition is designed,which can dynamically migrate data according to the load quantifucation of different storage devices,so as to achieve the purpose of balancing the resource utilization of each storage device.Secondly,in response to the Raft algorithm only performs data query through the leader,which leads to the problem of unbalanced utilization of cluster computing resources,a data query routing method based on cost evaluation is designed.This method can comprehensively consider the time cost of data query of each node in the raft cluster,and select the appropriate node to complete the data query operation,whick can achieve the purpose of sharing the query load of the leader node.Finally,the effectiveness of the load balancing method is verified by experiments.(2)According to the security requirements of distributed key-value storage systems,user registration and authentication,multi-user data isolation and node communication encryption methods are designed.Firstly,in response to the problem of illegal users accessing the system maliciously,a user identity registration and authentication method is designed to ensure that only visitors with legitimate user identities can use the system normally;Secondly,for the multi-user data security problem,a method based on The multi-user strategy of user ID ensures the logical isolation of user data;Then,for the problem of data being monitored due to the plaintext communication of nodes in the distributed key-value storage system,a node certificate manage method is designed,which can ensure that only nodes with valid certificates can join the cluster and realize the function of encrypted data transmission between nodes.(3)Based on the above research,a distributed key-value security storage system is designed and implemented.Based on the computing and storage separation architecture,the system is divided into computing layer,storage layer,and scheduling layer according to functional requirements.Then,the system implementation is completed based on the architecture design and module division.Through the test,the system supports Redis data instructions,user identity registration and authentication for multi-user data access,node elastic expansion and load balancing,data consistency and high availability.Besides,the numerical result of performance test show this system is in the same order of magnitude as the same type of open source storage system Ti KV,which meets the expectations of the system design in this paper. |
PDF Full Text Request