Research On Data Prefetch For Deep Learning In Hybrid Memory Environment

In deep learning(DL)training process,the transfer path of training data is typically from disk to DRAM and then to GPU memory.The efficiency of this path is largely limited by the performance of the disk I/O.Non-volatile random access memory(NVRAM)provides a new solution to the problem.NVRAM has a large capacity and fast read speed,but relatively slow writing speed,so it is often combined with DRAM to form a hybrid memory system to improve system performance.However,the vast majority of DL systems have not yet tapped the potential of this hybrid memory system to alleviate I/O bottlenecks.To address the data bottleneck problem above,a training data prefetching method(Daphne)for DL systems in a hybrid-memory environment is proposed,Daphne uses a sliding NVRAM cache(SNC)strategy: a cache window is maintained in NVRAM;the training data in the hard disk is mapped to the cache window;the data in the window is constantly updated;and the computation module reads the data from the cache window.For some situations with extreme high throughput,a fixed NVRAM cache(FNC)strategy is designed in order to improve the hit rate of the data: only a portion of the training data is mapped to NVRAM and the data is not updated during the training process.The DL system first fetches data from NVRAM and then reads it from the hard disk.The above two strategies take full advantage of the large capacity and fast read speed of NVRAM to improve the performance of the DL system to read data,while hiding the NVRAM writing access as much as possible during the normal training process to avoid the performance impact of relatively slow write speed.In addition,a new sub-dataset read pattern is proposed that improves the temporal locality of data in the NVRAM cache and helps it achieve better performance with SNC and FNC.The method described above is implemented on Caffe and its some derived systems,and some classic datasets such as Image Net are used for evaluations in emulated NVRAM and real NVRAM hardware environments.The experimental results show that Daphne can improve the training speed by more than 30%.