Research And Application About NoSQL Storage Systems

Traditional relational database has good scalability,high availability,high stability,and has been widely used.But,after all,the relational database is a universal database which is lack of flexibility and in the case of large amounts of data,it exists slow response and other issues,and thus can not be fully applicable to the needs of Internet application requirements.In view of this,researchers have developed a variety of database technology to make up the shortcomings of relational database.NoSQL,the represent of new database which responses faster and more flexible,and the strong scalability advantage can get more and more attention of people.But many existing system is based on the relational database,how to introduce NoSQL from existing relational database system,make the relational database and NoSQL respectively play its excellent performance is still the current urgently needs to solve the problem.This article relies on the actual development of the project-Campus 58 classified information platform.the article bring NoSQL database in the relational database,and research both of the depth of integration of key technologies,including the establishment of a relational database-based library,from the library to Redis heterogeneous separate read and write programs,data synchronization technology,client cluster fragmentation equalization technology.This article is consist of the following main aspects.(1)Because the single storage architecture of original MySQL system,the new system introduce a new Redis NoSQL databases which improved storage system architecture programme.MySQL stores all data,provides persistence support,Redis cache the simple and high frequency of reading data.When the server receives a write request,it write data directly to MySQL and synchronize the hot data to the Redis.When the server receives a read request,it first reads data in the Redis,the read fails then read from the MySQL.This method achieves a storage system to read and write,and enhances the efficiency of the system to access large amounts of data at high concurrency.(2)According to the needs of large data storage migration,this article analyses and summaries existing storage structures,research data migration technology,design a suitable data storage structure and use Redis pipeline mechanism to improve the data migration plan;At the same time,we use hash structure and simplified key to save memory and merge multithreaded operations to reduce network overhead and improve system efficiency.(3)With the increasing number of users and data,a single database server can not meet the growing demand for the amount of data,we need to implement a distributed database cluster in the system.According to the existing shortcomings of Redis distributed cluster load balancing technology,this article proposed the Consistency hash algorithm which contains redundant data to optimize the existing algorithm and to improve system stability and availability.Finally,the research results are applied to the Campus 58 classified information platform.The results show that the system can run stably for a long time and the individual server failure will not affect the system operation.The data migration rate increased significantly,under the condition of high concurrency,The storage system overall performance improved higher than the original storage system.