Research On Key Technologies Of Cloud Storage System Configurations For High Performance Applications

As the cloud platform becomes a promising alternative to traditional HPC (HighPerformance Computing) centers or in-house clusters, the I/O bottleneck problem is high?lighted in this new environment, typically with top-of-the-line compute instances but sub-par communication and I/O facilities. It has been observed that changing the cloud I/Osystem configurations, such as choices of file systems, number of I/O servers and theirplacement strategies, etc., will lead to a considerable variation in the performance and costeiffciency of I/O intensive HPC applications. This paper exploits the cloud I/O systemconfigurability and sysmatically defines the exploration space.However, storage system configuration is tedious and error-prone to do manually,even for expert users, leading to solutions that are grossly over-provisioned (low cost inef?ficiency), substantially under-performing (poor performance) or, in the worst case, both.This paper proposes a system which automatically searches for optimized I/O systemconfigurations from many candidates for each individual application running on a givencloud platform. The model takes advantage of machine learning models to perform perfor?mance/cost predictions. To tackle the high-dimensional parameter exploration space, thispaper enables affordable, reusable, and incremental training on cloud platforms, guided bythe Plackett and Burman Matrices for expeirment design. The evaluation results with fourrepresentative HPC applications indicate that the model consistently identifies optimal ornear-optimal configurations among a large group of candidate settings.To fast evaluate cloud I/O system for specific applications, this paper proposes aframework to automatically generate I/O benchmarks by program slicing. Given an ap?plication source code, it obtains a program slice through static analysis, and generates acompilable and human-readable benchmark from it. Results show that both the code sizeand the execution time of the application can be dramatically reduced, while keeping theoriginal I/O behaviors.