Research On NVM-oriented Lustre Persistent Cache On Client

High performance computing(HPC)cluster usually uses hierarchical storage management to manage the data in multiple storage layers.Burst buffer,which is a fast storage layer between computing node and backend storage layer(managed by Lustre or GPFS,etc.),is used to improve the performance of the HPC cluster.However,burst buffer and the distributed file system which manages the backend storage layer do not maintain a global namespace.Data migration between different layers still requires the involvement of administrator,which increases the management complexity and cost.On the other hand,data and metadata are stored on dedicated servers in distributed file systems(such as Lustre),and applications on client nodes(i.e.,computing nodes)access data and metadata over the network,which incurs network overhead and shared resource contention among client applications.Client node is usually equipped with high-performance storage devices,such as Flash-based SSD or 3D XPoint-based SSD.The utilization ratio of the local storage device is low because large amounts of data are stored in the backend storage layer.Exploiting the high-performance storage devices of client node to implement efficient and flexible cache is worthy studying.This thesis proposes non-volatile memory-oriented Lustre persistent cache on client(NVM-LPCC).By utilizing the high-performance NVM of Lustre client,NVM-LPCC implements two cache modes,i.e.,read-write cache and read-only cache.For read-write cache,files are cached by a single client to accelerate the read and write performance of applications.For read-only cache,files can be cached by multiple clients to accelerate read performance of applications.NVM-LPCC uses Lustre's hierarchical storage management and distributed lock manager to synchronize data between the client cache and the backend storage layer,and maintain the consistency and validity of data in a global namespace.The experimental results show that NVM-LPCC can increase the average read/write bandwidth by 35.82X/9.65 X compared with the original Lustre file system.NVM-LPCC also shows good scalability.In addition,using NVM as client cache device increases the average read/write bandwidth by 10.40X/4.41 X compared with the normal SSD.