I/O Performance Evaluation On The Hierarchical Storage System

In HPC field,I/O performance has already been an important bottleneck.In order to improve the I/O performance,the hierarchical storage system was developed.While the common I/O benchmarks,eg.S3 D I/O,BTIO,MADbench2,FLASH I/O,etc,are widely used,they are not capable to evaluate the hierarchical storage system,cause of their less of spatial and hierarchical locality,leading to the fact that there is still lacking of a benchmark aiming at hierarchical storage.This paper make up the IDX I/O bench,an I/O benchmark rightly for it.The main works of this paper are laid below:Firsr of all,this paper take an analysis on the structure of the hierarchical storage system and the its performance trend,and set models for performance evaluation.For hierarchical storage system,its I/O performance keeps dropping while the datasets growing up.Models for common used benchmarks which evaluate a vary data amount are not suitable caused of this character.To overcome this,we set I/O performance model and I/O efficiency model to describe the performance of hierarchical storage system through both the comprehensive I/O performance and the percentage change in properties.For the next,we take an analysis on IDX data format,hunting for the character suitable for hierarchical storage system.Because application's access to IDX format represent the superior locality,it fits the hierarchical storage well.Meanwhile,beneath the IDX data format,its granularity and stride keeps on changing with the resolution climbing high,so as to make differences on I/O patterns.So,to develope a specific I/O benchmark on IDX data format could realize the evaluation for hierarchical storage system and various I/O patterns.Further more,we designed and actually finished an I/O benchmark for hierarchical storage system.This benchmark holds 4 characterisics: hierarchical access,both random and sequence accesses,coexist of different I/O granularities,different greed of lock conflicting during I/O.According to the chacters above,we set the computing structures and relative parameters as global dimension,local dimension,nproc,variables and resolution,to control the I/O patterns.At the last,we used the IDX I/O bench evaluated the very file system.,compared with the I/O benchmark,and under the limitation of the various parameters.By this,we got the statistics of the experiment,laid a foundation for future work.