Research On Storage System Experimental Datasets Construction Method

With the applications of Big Data used more and more widely, the demand ofinformation storage is constantly increasing. Furthermore, with the development ofstorage technology, storage system architecture and the storage system software havebecome complex and diverse. In addition, in the application environments, the stored dataare also increasingly complex and various; Thus, the management and experiment of thestorage system, performance optimization become extremely difficult and important. Theresearches on performance evaluation of the storage systems and distributed file systemmetadata management require a lot of workload data, but these workload data sources aremostly the previous trace datasets. The information of these datasets are often incomplete,and their various operations are confused. Furthermore, current public trace datasets haveslight relevant file system image information. In addition, there are no available trace datafor Big Data applications.In order to address the above problems, this paper proposes a method usingself-similarity models and based on statistical analysis to construct trace data. This methodcan not only generate accurate re-constructed file system image, but also construct thelarge-scale and customized file system dynamic workload datasets. File system traceworkload system obtains the dynamic properties of the workload model and evaluatesthem by statistical analysis to the actual trace datasets. File system generation enginegenerates file system image, and dynamic workload generation engine generates dynamiccustomizable workload datasets using self-similarity modeling methods.Experimental results show that the proposed method can accurately generate filesystem workload. The arrival pattern of trace datasets has very high self similarity, and theself-similarity parameter is almost close to1; Furthermore, the access pattern of syntheticI/O workload has a good burstiness and temporal-spatial locality. In addition, this methodcan synthesize self-adaptive dynamic trace datasets.