Research And Application Of Distributed Smart Community Storage Technology

With the rapid development of the mobile Internet era,the scale of the industry's application systems has rapidly expanded,causing the amount of data generated by it to explode.Massive data poses a severe test on database storage capacity.Therefore,how to deal with Massive data storage,high concurrent access by users,and high availability of the system has become a problem for enterprises.Traditional databases have gradually been unable to meet the requirements for storage and performance of big data due to their performance bottlenecks and difficulties in scaling.The emergence of distributed databases effectively solves these problems.Distributed databases and computer networks logically connect multiple database servers together to build a distributed cluster,thereby dispersing the pressure of a single database.In view of the massive data storage and high concurrency problems encountered by smart community projects,this thesis uses Mycat database middleware as the key technology for building distributed databases.Combined with the actual needs of the smart community project,the original centralized storage solution in the project was improved,and a distributed storage solution based on Mycat was proposed.The main work of this thesis is as follows:(1)Using Mycat middleware as the key technology to achieve database read-write separation,data fragmentation and failover,thereby improving system's read-write performance,scalability,and high availability.(2)Optimize the data sorting in the data integration process,and improve the heap sorting algorithm used in it.By reducing the number of comparisons in the sorting process,the time overhead of the algorithm is effectively reduced.In the end,the running time of Mycat in the data sorting process is reduced by more than 50%,and the data sorting efficiency is improved.(3)In order to avoid the unavailability of the system due to a single Mycat failure,the HAProxy + Keepalived technology is further introduced on the basis of the above scheme to achieve load balancing,improve the high availability of the system,and make the entire system run more efficiently and stably.(4)The above design scheme is deployed for testing in a real environment,and compared with the original storage scheme of the smart community project.In the environment of massive data and high concurrency,the error rate of the original storagesolution when processing data requests rose to more than 50%,while the error rate of the Mycat distributed database always remained at 0%.It is proved that this scheme can relieve the reading and writing pressure of the system and improve the stability of the system.