Research On Storage Cache Performance Optimization Of Lustre Cluster System

The cluster storage system is the core component of High-Performance Computing(HPC)system,which carries the heavy task of data access.The diversified requirements of new applications continuously improve the performance requirements of the clustered storage system,such as read-write bandwidth and IOPS(Input/Output Operations Per Second).Giving full play to the performance advantages of the new memory devices and designing a reasonable cache structure can effectively improve the data reading and writing performance of the cluster storage system,such as the Burst Buffer of the data cache scheme of the computing node,which is widely used in the HPC field.However,the existing cache management is usually independent of the back-end cluster storage system,which makes the efficiency of storage resource management low.The existing data caching technology of computing nodes does not fully tap the potential of hardware heterogeneity in the field of HPC,which makes it difficult to give full play to the performance advantages of data caching.There are too many lock contention and remote procedure call burden in most metadata write-through caching methods in distributed file systems,which leads to the bottleneck of metadata access performance.This paper focuses on Lustre,which is the most widely used cluster storage system in the HPC field and aims to meet the growing performance requirements.Aiming at cache technology,which is an important performance optimization technology of cluster storage systems,combined with new storage devices.The research contents and main contributions of this thesis are summarized as follows:In the data cache design of existing HPC computing nodes,the front-end computing node cache layer and the back-end parallel file system layer are usually managed independently and have their independent namespaces,which leads to difficulties in data sharing,complex cache management,inefficient storage resource management and so on.To solve the above problems,a hierarchical persistence client cache architecture(PCC)based on a global aggregation namespace is proposed.PCC connects the client persistent cache layer with the back-end storage layer to form a global aggregation namespace,which eliminates the difficulty of data sharing and simplifies the complexity of file data access and management.PCC proposes a data management method based on a hierarchical storage management mechanism and a cache prefetching method based on the Lustre Change Log.By comprehensively considering self-defined rules and application load characteristics,it reduces the overhead of cache data management and realizes efficient data on-demand intertier migration.PCC has two caching modes: single-client read-write cache mode and multiclient read-only cache mode.Experiments show that in the single client read/write mode,PCC has nearly 50% improvement in write bandwidth and IOPs compared with Lustre;In the multi-client read-only mode,the average read bandwidth of PCC increases by nearly1.82 times as the number of clients’ doubles.The solid-state disk and persistent memory coexist in the computing node to form heterogeneous storage.The existing data cache technology of the computing node is difficult to consider the characteristics of different cache devices,and there are problems such as complex cache device management,uncoordinated software and hardware,and unreasonable utilization of storage resources,which makes it difficult to give full play to the performance advantages of the data cache of the computing node.To solve the above problems,a heterogeneous persistent client cache(HPCC)based on hardware/software cooperation is proposed.HPCC embeds different local file systems through the lightweight configurable client local heterogeneous fusion cache resource management module(HFMM)to efficiently manage different types of cache devices and form a unified cache resource pool,to simplify cache device management.At the same time,software and hardware cooperate to ensure the data cache performance of computing nodes.To match different read/write I/O paths and diverse application load requirements,HPCC proposes a storage medium aware data placement policy engine scheme based on user-defined policies and extended cache prefetching method,which can select appropriate cache resources as required to meet different application quality of service.Experiments show that in the single client read/write mode,HPCC improves the average write throughput by 9.4 times,4.68 times,and 1.7 times respectively compared with Lustre,solid disk persistent client cache,and persistent memory client cache without HFMM management;In the multi-client readonly mode,the average read throughput of HPCC is 28.1 times and 8.03 times higher than that of Lustre and solid-state disk persistent client cache,respectively.Most distributed file system metadata uses write-through caching to simplify the consistency overhead,but metadata write-through caching has lock contention and the burden of remote procedure calls.With the expansion of distributed file systems and the increase of workload,metadata access performance bottlenecks have become increasingly prominent.To solve the above problems,metadata write-back caching(Meta WBC)method is proposed.Metawbc caches metadata by embedding a POSIX-compatible cache management module(Cache MM)in the Lustre client,delaying the time when metadata is written back to the metadata server,reducing the number of remote procedure calls between the client and the metadata server,and alleviating lock contention.At the same time,Meta WBC uses refresh sorting during writeback to solve the specific refresh dependency problem in the metadata write-back cache and determines metadata-related operations through the designed state machine and cache state transition strategy to provide highperformance services based on ensuring metadata consistency.The test shows that the IOPS performance of Meta WBC increases linearly with the increase of nodes.Under the Filebench load,the average IOPS of Meta WBC scheme are 5.09 times,10.03 times,and1.38 times higher than that of Lustre,NFS,and ext4,respectively.