Research On Key Technologies Of Optimizations For Efficient And Reliable Storage Systems

The core of any computer system lies in the processing of data,which is only possible on the basis of data storage.In the era of big data,the ubiquitous data explosion poses higher demands on the current data storage systems.On the one hand,the ever-scaling size of a storage system inevitably leads to more and more frequent data unavailability as well as data loss,which further highlights the importance of data reliability of a system.On the other hand,the gigantic and skyrocketing data volume makes it particularly vital for existing storage systems to achieve better storage efficiency.As practically hopeful as it is,erasure coding has gradually replaced the triplica with relatively lower storage cost,and thus been applied widely to various storage systems as the mainstream data reliability solution.Meanwhile,data deduplication poses the capability to directly remove duplicated data contents,thereby significantly achieving more efficient data storage and attracting lots of practical attention.Based in general on a modern storage system,which incorporates deduplication or erasure coding or both technologies for data reliability as well as storage efficiency.This dissertation starts from the perspective of performance optimization,and proposes several optimizational work specifically in erasure code extension,data layout,data update,failure cache,etc.The main contributions and innovations are as follows:1.Proposed a RAID-6 based code extension methodology for fast data recoveryThe existing methods of extending RAID-6 focus on a solo purpose of improving device-level reliability,resulting in the fact that the newly added redundant device is barely used in various data reconstruction processes,which is a waste of resources.This paper proposes a RAID-6 code extending method RAID-6Plus for fast data recovery from the combined perspectives of enhancing reliability and performance.RAID-6Plus performs special encoding on the third redundant device with the purpose of multiplexing more overlapped elements,such that this newly added redundant device can assist various data recovery situations with more overlapped elements,thus reducing as well as possibly balancing out the total data reads in the processes.RAID-6Plus is explicitly applied to the RDP code and X-code to produce two new corresponding schemes named RDP+ and X-code+.Performance evaluations show that RAID-6Plus exhibits better fast reconstructio performance than RTP and STAR,which both adopt the traditional extending methods.Also,RAID-6Plus is proved to have a higher reliability than RAID-6 after the extension.2.Proposed a request-aware and globally-locally even deduplication data placementExisting storage systems with incorporated deduplication directly adopt as inheriated before the incorporation either a simple rotating placement or a random placement,both of which fail to capture the impact of the altered data sequence upon subsequent read accesses after deduplicate data contents are removed through deduplication.Therefore,the phenomenon in which system-wide read parallelism cannot be effectively utilized or underutilized is often seen in the system.This paper aims at a typical deduplicationequiped small-scaled storage system,which has abundant internal transmission bandwidth as well as good parallelism.A request-aware deduplication data placement GLE-Dedup is proposed to achieve the purpose of making globally and locally even placement of unique data in the system.In essence,GLE-Dedup is a coarse-grained group placement strategy,which enables the perception of the location of each data block inside a single request while maintaining the global storage balance.In this way,unique data can be placed with insightful guidance to avoid congested nodes and thus making better use of the systemwide read parallelism for better read performance.3.Proposed a deduplication-aware and erasure-coding-aware data placementIncorporating deduplication and erasure coding simultaneously is preferred to achieve both storage efficiency as well as economical data reliability for modern storage systems.However,these hybrid systems naively inherit the data placement before the incorporation,and lacks an adaptive adjustment of the added deduplication and erasure coding at the system level,thus causing the problem of ”read imbalance”,in which parallel data read is capped by throttled storage nodes and cannot be effectively utilized.Therefore,this paper proposes an even data placement with both deduplication awareness and erasure-coding awareness(DA,Dual Awareness).DA placement can simultaneously enable both effective awareness of deduplication and erasure coding,and place data with careful guidance adjustive to changes of the original data sequence caused by deduplication and erasure coding,thus making better exploitation of system-wide read parallelism in subsequent accesses.Extensive experiments using both real world traces and synthesized workloads,prove DA achieves a better read performance.4.Proposes a dual-level relay update scheme in erasure-coded systemsThe existing erasure-coded storage systems adopt for parity updates a naive star transmission structure,which overlooks the characteristics of the system network,thus causing excessive cross-rack traffic and suffering from potential single-node bottleneck problems.To this end,this paper proposes a dual-level relay update scheme in erasurecoded systems DR-Update.A mathematical formulation of the data update problem in an erasure-coded system and the detailed DR-Update design are both presented.Essentially,DR-Update aims to 1)minimize cross-rack traffic,2)alleviate the single-node bottleneck problem,and 3)further balance rack-level update traffic and node-level connections.In detail,DR-Update minimizes cross-rack traffic by introducing a rack set and intra-rack relay,and further mitigates the single-node bottleneck problem of the starting node by enabling cross-rack relay.To achieve both rack-level and node-level balance,a balanced linear transmission structure is constructed with respect to both the outbound degree of each involved node and update traffic across related rack.Furthermore,extensive tracedriven experiment results show that DR-Update not only manages to reduce excessive cross-rack traffic but also maintains balance of the system on both node level and rack level.5.Proposes a greedy failure cache considering failure recency and failure frequency for an erasure-coded storage systemExisting big data storage systems assume that temporary failure data will automatically come back alive after a certain period of time.Therefore,there is no mechanisms for preserving failed data and sharing failure information among multiple users in the system.Instead,a data reconstruction process is mechanically triggered to regenerate the failed data each time upon data unavailability and the newly recovered data is directly discarded after serving the original data access.Such disposal of failure data not only prevents the sharing of failure information among clients,but also leads to many unnecessary data recovery processes,(e.g.caused by either recurring unavailability of a data or multiple data failures in one stripe),thereby straining system performance.To this end,this paper proposes GFCache to cache corrupted data for the dual purposes of failure information sharing and eliminating unnecessary data recovery processes.GFCache employs a greedy caching approach of opportunism to promote not only the failed data,but also sequential failure-likely data in the same stripe.Additionally,GFCache includes a FARC(Failure ARC)catch replacement algorithm,which features a balanced consideration of failure recency,frequency to accommodate data corruption with good hit ratio.The stored data in GFCache is able to support fast read of the normal data access.Furthermore,since GFCache is a generic failure cache,it can be used anywhere erasure coding is deployed with any specific coding schemes and parameters.Evaluations show that GFCache achieves good hit ratio with our sophisticated caching algorithm and manages to significantly boost system performance by reducing unnecessary data recoveries with vulnerable data in the cache.