Research And Design On Benchmark Of Distributed File System

Recently data are exploding as the cloud computing is evolving and the cloud platform is becoming much more complex and diversified.Demands for higher performance of storage are growing rapidly.As a core component of cloud platform,distributed file system is a prevailing system which offers efficient and reliable data access and processing.Because tens of thousands of applications use it to access data,its performance has a great influence on the whole cloud platform.Realistic benchmark for distributed file systems is very helpful to its performance evaluation,performance debugging,as well as capacity planning.To the best of our knowledge,there's no benchmark designed for distributed file systems currently using the way of probability simulation to mimic its I/O workload.Moreover,the heterogeneity of I/O workload and complexity of distributed file system make it difficult to conduct benchmark.There are two important factors which have a great impact on evaluation.One is the testbed,which refers to the way of pre-populating data for operations.Features of these metadata are determinants of performance of metadata server's operations.In this paper,we characterize number of replication,depth of directories,number of files,file types and file sizes and state their simulation.Besides,validation for data simulation is listed in the experiments.The another is request arrival pattern which is a very important component in benchmark.Traffic model is necessary for realistic workloads because it decides when and how the I/O requests arrive.Many similar benchmarks simply regard it as constant rate or Poisson process which can't seize its heterogeneity.To address this problem,we categorize it into four aspects.(1)Request arrival rate.(2)Inter-arrival time.(3)Periodicity.(4)Request data.In this paper,we show these features' detailed characteristics and how they work together to generator complex and realistic workload.In addition,based on all of these characteristics,we develop a flexible,scalable benchmark framework.And in the following,its design and implementation are demonstrated.