Research And Implementation Of CBase Database Memory Management Technology

The CBase database is a distributed database independently developed by the Bank of Communications.The whole system is implemented in C/C++ programming language.When processing large batches of business requests,the technology of high concurrency and multithreading is used to reduce the maximum response time of the system,the mode of multi-point updating data is used to improve the system throughput,and the advantages of high availability,scalability,and high concurrency are used to increase the number of business requests processed in seconds.However,due to the continuous expansion of upperlevel business functions,the memory management of CBase has become one of the bottlenecks of current system performance that various problems will occur during memory usage,including but not limited to:(1)Developers are required to manage memory allocation and deallocation for themselves,there will be the problem of forgetting to release the memory,which will cause memory of the CBase system to increase continuously;(2)Because it does not implement a perfect memory detection mechanism,some problems occurred during the programs of CBase running,such as memory out of bounds,memory fragmentation,memory leaks,etc.(3)The system allocation function such as new/malloc is used in the business request process,which prolongs the transaction response time;(4)Due to the unreasonable design of some memory pool,the CBase database will happen memory exhausted for a long time.In response to the above problems,the thesis starts from the perspective of memory management in the context of improving database performance: firstly,there is introduces design framework of the current mainstream application software memory management,and the thesis elaborates the implementation methods and usage scenarios of the memory pool and memory allocator in the key technologies of memory management;next,there is a brief introduction to the underlying design and implementation of the C++ operator new/delete and the C language library function malloc/free;then,an efficient and universal memory management unit is designed based on the distributed database CBase system architecture and business scenario characteristics and the requirement analysis of the memory usage of the CBase database.The memory management unit uses memory pool and thread local cache technologies to divide large blocks of memory according to different granularities and manage them uniformly,which can effectively improve the overall performance of the system,reduce memory errors,and improve memory usage efficiency;and based on the idea of changing space for time,the batch cache of memory blocks in multi-threaded high concurrency scenarios is realized,which improves the access speed of CBase database memory requests and reduces memory fragmentation;in addition,all levels of cache pools can dynamically expand and reduce memory according to their own memory usage,which effectively alleviate the downtime risk of the database due to insufficient memory;finally,the thesis implements memory information statistics and memory detection functions based on the memory management unit of a variety of memory statistical monitoring technologies,and can detect memory out of bounds and repeated release,memory leaks and other issues,shorten the database development and testing cycle,making the CBase database more stable and safer during operation.The above implementation was tested on the distributed database CBase.The experimental results show that the memory management unit shows better efficiency in the small memory allocation and release scenario,and can effectively improve the overall performance of the CBase database and reduce memory waste.