Design And Implementation Of Memory Allocator For Deep Learning Platform

Deep learning has achieved great success in the fields of image recognition,speech recognition,Machine Translation and so on,bringing huge social and economic benefits.In the aspect of memory management,deep learning applications are still using traditional memory management,which includes only the management of CPU host memory without the support of GPU device memory management,resulting in management dispersion and increase of usage cost.The traditional memory allocator,in order to give consideration to various applications,has poor performance in deep learning application.and there is no way to achieve self-control by technology secrecy.So a high performance heterogeneous allocator is in great need.This paper designed and implemented a memory allocator for managing heterogeneous memory,named MADL(Memory Allocator for Deep Learning).It includes the management of CPU host memory and GPU devices memory.MADL provides a suit of unified interfaces to manage different memory,shields the differences in different memory,makes it much easier to use for those who do not know much underlying technology details.At the same time,Combined with the actual memory usage features of the deep learning application,MADL mainly focuses on large size memory,and uses memory pool technology to manage memory in the user mode,which greatly improves the efficiency of memory allocation.The first-fit method,which uses linear search to find free memory blocks,is used to manage memory.MADL builds index to reduce the searching time and improves the allocation performance.MADL has strong portability and can run on Windows,Unix-like operating system,Android and other operating system.It designed and implemented a new heterogeneous memory management architecture,which has good stability and maintainability.The test approved that,compared to the traditional memory manager,the MADL respectively increased 15~20% and 17~30% on CPU and GPU memory allocating speed.Meanwhile,it also respectively increased 2~5% and 10~20% on CPU and GPU usage rate.Overall,MADL has much better performance than traditional allocator,and achieves the design goal.