Research On Hotspot Data In Object-based Storage Systems

With the rapid development of computer network and the increasing data requirements by users, it brings more opportunities to advance network storage technology, while dis-tributed storage systems have to face more and more challenges. In these challenges, how to improve the I/O performance of storage systems and how to manage large-scale complex storage systems, which are two major problems placed in front of many companies and re-search institutes. On one hand, hotspot data represent the users'behaviors, which are the key to solve the problem on I/O performance of storage systems. However, hotspot data are not made maximum use in storage systems nowadays, which are not given a comprehensive and systematic analysis, either. These reasons makes the overall performance of storage system haven't been improved totally. On the other hand, with the increasing complexity of storage systems, the traditional manual management mode doesn't meet the requirements of modern storage technology any longer. How to transfer the users'demand to the storage systems, and then make large-scale storage systems self-adapt management is also a significant issue.Object-based storage and attribute management technologies have been appeared, which have the potential to solve these two problems. In an object-based storage system, objects are instead of typical files, which have richer semantic contents and can transfer more information for hotspot data. Combined with attribute management technique and through analyzing the users requirements and data access patterns, we can extract the at-tributes on hotspots, which can makes an implementation of the adaptive management and an improvements on I/O performance in storage systems.Therefore, first this paper presents a system-level solution based on hotspots. Accord-ing to the analysis of hotspot phenomenon and the access patterns of hotspot data in real applications, this paper first proposes the definition and classification of hotspots in object-based storage system. Combined with T10 OSD-3 standard, this paper also establishes a hotspot attribute page, and gives dynamic data organization schemes and Quality of Service (QoS) on hotspots based on different applications and various workloads.Second, to solve the problems on hotspot prediction and single level cache in storage systems, this paper proposes a new hotspot prediction method based on both the access pat- terns of hotspot data and Zipf-like law-Object Hotspot Prediction Model (OHPM). And in accordance with the characteristics of stage hotpots, this paper adds increasing rate of access frequencies to OHPM as supplementary, which can predict stage hotspots. Then based on the difference between the access patterns of permanent hotspots and the access patterns of stage hotspots, this paper gives a single level cache strategy-use dual cache stacks and re-alize adaptive management. This paper also gives some discussion on arguments in hotspot hit ratio and time granularity.Third, due to the initiator and target in object-based storage systems already constitute a system of multi-level cache, and address the hint problem in current multi-level cache algorithms, this paper gives a creative solution-a multi-level cache algorithm using K-step hints (Hint-K). Hint-K uses history information of demote and promote hints, and can simply evaluate the activeness of a data block by its K-step hint value. This paper also gives case studies of Hint-K approach when K is equal to different values.Finally, this paper gives the design and implementation of our prototype-Hotspot Attribute-managed Storage System (HASS). By using dynamic data organization and QoS of hotspots (HO), Object Hotspot Prediction (HP) and single level cache strategy (HC), the performance of HASS improves up to 62% and decrease up to 25% I/O operations. Though simulations of multi-level cache approaches in various workloads, Hint-K achieves better performance compared to other multi-level algorithms (such as MQ, DEMOTE and PRMOTE algorithms).