Research On Buffering Algorithms For Heterogeneous Storage Based On GBDT

Big data has higher requirements on the performance and capacity of the storage system.At this stage,the data center introduces more storage devices to meet the continuously increasing performance and capacity requirements.The performance difference between storage devices makes the storage device load and service capability mismatch,resulting in the overall system performance can not meet the demand.The current research work lacks the consideration of device performance differences in storage systems.The difference in the access speed of I/O requests on different storage devices causes performance short boards,which affects the overall system performance.Aiming at the problem of performance degradation caused by the mismatch between the equipment load and the service capability of the heterogeneous storage system,a cache management algorithm based on the access performance prediction(Heterogeneous Storage Device Cache Algorithm Based on GBDT).The memory is divided into multiple logical partitions.Each storage device has a corresponding logical partition.In order to reasonably set the size of the cache partition,the GBDT(Gradient Boosting Decison Tree)decision tree model based on machine learning is used to predict the performance of the I/O request on the storage device,and then the performance prediction results are combined with the performance prediction results.Section cache allocation size.This algorithm balances the load distribution between different storage devices by adjusting the buffer allocation,which makes the load on the storage device match its own service capability,thus reducing or even eliminating the overall performance bottleneck in the heterogeneous storage system and improving the overall performance of the system.Use real storage workloads to access heterogeneous storage systems for testing.Test results show that the GBDT decision tree can accurately predict the performance of I/O requests on storage devices.And under different types of load access,compared with Forney algorithm,G-Cache algorithm can significantly improve the performance of heterogeneous storage systems.When the cache size is fixed and the stripe length is changed,the performance improvement is 8.5% to 27.8%.When the stripe length is fixed,the cache size is changed,and the system upgrade rate is 6.6% to 25.5%.