Research On Object Storage Optimization Based On QoS

With the explosive growth of the data and the importance of the information, more and more challenges on capacity, I/O performance, availability, reliability, security of the storage system have been put forward. The solution to this issue is to establish the hierarchical storage system with standard interface so as to integrate abundant software and hardware resource and achieve the user application requrements. However, the access interfaces that exist in the different hierarchies of the traditional storage system have covered the detailed information from the top user between application, host and device and the abundant semantic information can not be obtained by the storage system, which decreases the manageability and the ability on data organization in network storage system. The computing ability of the storage device can not be fully utilized for the improvement of system performance. The object based storage and attribute management hold great potential expectation for the solutions of the above problems.Moreover, the intelligent storage devices in object based storage framework are able to aware the access patterns from diverse user applacations, which is avalaible for the QoSS (Quality of Storage Service) gurantee. Quality of storage service is a general metric for the system prformance evaluation, which provides service partition and guaranteed performance for the users. QoSS can be quantitatively described as a series of parameters, such as capability, availability I/O throughput, scalability, service cost and etc. Service partition aims to provide different guaranteed quality storage access in accordance with the different application requirements; while guaranteed performance means to solve the problems of performance parameters, such as bandwidth, delay and etc. In some sense, many techniques and ideas which involved with the network storage are mainly focused on the issues of optimization and manageability. Enforcing QoSS efficiently can decrease the complexities and difficulties in the large scale storage.Firstly, based on the object storage technology and attribute management, learning the QoS control and management mechanism from the network communication system and considering the character in storage system, the storage resource management system based on QoS is eatablished in accordance with the QoS methodlogy, meanwhile, the attribute based QoSS description and relative enforcing framework are improved, including the QoSS providing mechanism as well as the QoSS control and management mechanism, which constructs the base for implementation and optimization of the attributes based storage. A QoSS framework based on object storage and relative QoSS mechanism are introduced and a QoSS optimization approach based on request breakdown is analyzed, which summarizes the typical optimization strategies about the QoSS. The beneficial research on TCP delay performance model has been carried out theoretically, which is meaningful for the further research.Secondly, considering the phenomenon of replicating batch small files always represents poor performance in systems, especially in the distributed system. A novel method on QoS optimization of batch small files is proposed. Parallel, consecutive, aggregating and other polices have been implemented in the study and optimization of the replication and emigration process for batch small files on ext3 file system. The experiment shows that the algorithm of consecutive reading source files and parallel writing target files have the best performance in local replication, and aggregating algorithm also do in network replication. Some relevant data about metadata operation have also achieved in the experiment, which will be helpful for the further optimizing file system performance.Lastly, in order to demonstrate and evaluate the proposed strategy, an attribute-managed storage prototype system with guaranteed QoSS called AMSS is designed and implemented in accordance with the QoSS control and namagement mechanism. The storage strategy based on delay and bandwidth attributes is proposed, which is the base for the attribute transmission mechanism. The object based access interface based on extension of OSD and iSCSI protocols is implemented. The experiment result shows that the performance of aggregate output bandwidth in the system QoSS optimization strategies increased by 28～38% than the general system without QoSS optimization strategies. Compared with the iSCSI based storage system, AMSS can maintain more client number. By effective QoSS control and management mechanism, AMSS can achieve the better performance and end to end delay guaranteed QoSS than general system.