Research On Key Technology Of Reliability In Storage Systems

Ever since Jim Gray, a Turning Award owner, put forward an empirical law that the data generated every18months equal all data generated before in networking environment, people have realized that data are increasing in an explosive way. As the carrier of data, storage systems have been swiftly developed in information society. Abnormal factors often happen to storage systems, which causes data loss and damage. The issue of data storage reliability are becoming increasingly important, especially the research on critical theory and technology of storage systems reliability.Many kinds of reliability technologies are applied to storage systems. Firstly, redundant information can raise the reliability of storage systems. When a fault happens to the system, the redundant information is used to rebuild the data. However, in the current technical conditions, the system recovering time window is so long that it will seriously impede the system I/O performance. What is worse, the possibility of a second disk fault is big, which may result in all data damage. Therefore, to reduce the recovery, increase and predict disk reliability is an issue worth researching. Secondly, large amount of experiments show the highly relevance between high-utilization and high-loss in disks, which demonstrates that it is necessary to combine that relevance in the research on storage system reliability. Third, statistics show that popular data are the decisive factor in the reliability of storage systems. According to the popular data to predict and use it for performance optimization study, usage and prediction of popular data is an important issue to improve the reliability of storage systems. Finally, because the too high disk temperature can lead to disk failure, the main reason for the disk temperature is the increased workloads. Therefore, adjust and predict workloads to control disk temperature are a research direction to improve the reliability of storage systems. Based on the theoretical analysis and technology discussion of many research metioned above, the main works are as follows:In order to improve the reliability and predict disk, we propose disk-based SMART technology disk reliability model, which is based on the actual value of the disk SMART parameters, thresholds, and attribute values to establish parameter reliability model, through theoretical discussion and practical test and analysis to choose the weight of important parameters, according to the reliability of parameters and the parameter weights to re-create a disk reliability model, finally disk reliability early-warning threshold is determined by the analysis the SMART parameter actual value and parameter weights case, combined with the reliability of the disk and disk reliability early-warning threshold on disk reliability level classification. This technology eventually migrate the low-level data reliability disks to backup disks, thereby improving the reliability of the storage system.To utilize and predict popular data, given the data popularity model based on Zipf s Law, the model combines two known parameters:current time frequency ranking of all data access and data access frequency increase in the current period. We can predict the ranking of data access in the future time. At the same time, the a and C of Zipf s law, prediction of future time as well as popular data queue length parameters are estimated in detail and introduced a different approach. Then we use the classified popular data characteristics to provide discussion and analysis of prediction accuracy, and the combine different forecasting of the future time and the queue length of popular data to analyses and discuss of the forecast rate. Finally, we take data migration and other protective measures for popular data on different disk reliability and disk utilization of the current and future, thereby improving the reliability of the storage system.In order to control disk temperature and predict disk workload, we propose disk temperature control method based on the Hurst exponent. The method combines the close relationship between the disk temperature and workloads, and put forward a method to calculate I/O workload self-similarity (H) by using Hurst exponent, and use two method, variance-time and R/S analysis, to estimate the value of H. The calculated value of H is used to predict the I/O load in the next moment. Finally, by command current high-temperature disk and predicted future time high workloads disks to transfer loads or shut down, and other protective measures, it can be prevented that the disk temperature is too high lead to disk failure, thereby improving the reliability of the storage system.In order to verify the solution effect of the disk reliability, disk utilization, data popularity and disk temperature, the disk reliability model based on SMART technology, the data popularity model based on Zipf law and the disk temperature control method based on Hurst exponent are tested and analyzed. At the same time, two reliability optimization methods are proposed, one based on disk reliability, disk utilization, and data popularity, the other based on disk temperature and workload. Experimental results show that these key technologies and solutions can effectively improve the reliability of storage systems.