Storage Engine Opitimation Based On Internet Application

This article first introduces the challenges encountered in a traditional relational database technology in the Internet environment, then analyzes the feature of Internet applications (especially blog-based social networking application):a huge amount of data, there are significant data hotspot; huge amount of concurrent requests, but read more write less, and many of which are key-value queries (such as find by primary key accounted for a large proportion of the queries). Then introduced the bottleneck traditional relational database face under the Internet applications. Followed by analysis of the studies about this issue taken by academia and industry, including nosql technology, read and write separation, sharding and distributed database. Analyzing the inadequacies of these methods, and then leads to the work this paper is going to do and also the significance of the work.this paper relys on a internship the author participated in developing a custom storage engine of mysql.Based on the feature of web apps:obvious hot spots of Internet applications, read more and write less, many queries just key-value type, design and develop a record-level cache specifically for cache hotspots records.this paper analyzes the design ideas, the overall architecture and the main process. Row-level cache design makes the custon storage engine performance much better compare with mysql innodb storage engine.Although due to the presence of a row-level cache, making custom storage engine perform significantly better than innodb, but custom storage engine has problems about stability and disk IO in performance testing, the engine transaction throughput and disk IO requests have problem of shock and jitter phenomenon, on the other hand, there is also disk IO request fragmentation phenomenon and IO merging is not ideal. To solve this problem, the author use a self-developed a true analog Internet social application (blog) performance benchmarking tool for performance analysis and optimization. Mainly focused on disk IO, including adjustments to write dirty block algorithm and strategy as well as asynchronous IO optimization. After optimization, performance benchmarking tools for realistic simulation of internet social class (blog) test, the custom engine performance excellent transaction throughput more than five times better than innodb storage engine, and the performance is smooth, there is no obvious shock.