Design And Implementation Of Object Storage Hotspot Prediction System Based On Integrated Learning

In the 21 st century,Internet technology has developed rapidly,and various smart devices are generating large amounts of unstructured data every day.Faced with such a huge amount of data,the traditional storage area network(SAN)and network-attached storage(NAS)are obviously difficult to meet people's needs.Therefore,object storage(OBS)has entered people's field of vision due to its excellent performance and low cost,and effectively meets the storage needs of a large number of users for large amounts of data,and provides a new solution for the problem of big data storage..However,at present,in the object storage system,the processing of hot data has not exerted its maximum effect,and the performance of the object storage system still has a lot of room for improvement.To this end,this paper designs and implements an object storage hotspot prediction system,which uses various attributes of the storage object to predict the access heat of the file,and then intelligently adjusts the storage method of the file to realize the prediction and management of hotspot files and improve the object The performance of the storage system optimizes the user's file access experience.In order to realize the core file heat prediction function of the system,the text designs a file heat prediction algorithm.The traditional prediction method mainly focuses on the analysis of the historical access records of the file,but ignores the internal relationship between the file name and the amount of access.Therefore,this article uses the integrated learning method to fuse file history access records and file name semantics,so as to obtain a model with better prediction effect.The implementation of this algorithm is mainly divided into three parts.First,considering that the historical access records are time series data,the LSTM model is used to model the historical access data of the file;then,for the semantic information contained in the file name,first through the access Divide the heat level by quantity,and then use the text classification method to obtain the file heat level.This part uses TF-IDF weighted word2 vec to extract text features and classify them by the SVM classifier.Then use the integrated learning method to fuse the first two models to obtain a prediction model with better prediction effect.Finally,through experimental verification,it is feasible to use the LSTM model for file heat prediction of the object storage system,and it shows better prediction results than other models.However,after the file name semantics are merged into the LSTM model,the prediction effect is improved.Further improvement.After completing the design and implementation of the core algorithm,this paper analyzes the functional requirements and non-functional requirements of the system.According to the results of the demand analysis,the overall architecture of the system is explained,the functional modules are divided,and the design The structure of the database and the interface of the system are followed by detailed design and implementation of each functional module of the system.Finally,the functions of the system are tested.After testing,the functions of the system implemented in this paper have met the business requirements and can be used normally.