| With the advancement of cloud computing technologies,serverless computing is gaining popularity as a new cloud computing model.Serverless computing is widely used for data analysis tasks because it can quickly and cheaply invoke hundreds of concurrent functions.Modules of data analysis applications typically rely on remote storage systems to exchange temporary data due to the restrictions of the serverless platform design.However,existing distributed storage systems are not built for dataintensive scenarios and cannot meet the rapidly fluctuating storage requirements of data analytics applications in serverless platforms,making the storage system frequently suffer from performance degradation or resource underutilization.Furthermore,temporary data in serverless scenarios have unique properties like unequal size,unbalanced access frequency,and various life cycles,resulting in frequent node load imbalances.To this end,we proposed a distributed storage system optimization solution for serverless computing.Through the adaptive cross-storage layer dynamic load balancing mechanism and the serverless function-based resource elastic scaling mechanism,this solution considerably improves the performance of data analysis applications by enhancing the distributed storage system’s ability to process temporary data.The main contributions of this thesis are as follows:1.To address the problem of node load imbalance when processing temporary data in existing systems,we proposed an adaptive cross-storage layer dynamic load balancing mechanism.This mechanism combines the power of the two choices algorithm with the consistent hashing algorithm to adaptively select servers with lower load through virtual nodes in multiple storage layers of the system,allowing temporary data to achieve relative stability in both static distribution and dynamic access.The AWS platform experiments show that this mechanism can effectively deal with the specific load characteristics of temporary data.Compared with the traditional load balancing mechanism,this approach can improve storage node load balancing by roughly 8.33 x,allowing data analysis applications to reduce their processing time by 7.60%-21.14%.2.To cope with the drastic changes in the storage requirements of data analysis applications,we proposed a serverless function-based resource elastic scaling mechanism.This mechanism exploits the separation of mechanism and policy to promote system scalability by offloading elastic scaling tasks to the serverless platform in the form of functions.This mechanism also includes a range of pluggable policies,such as reactive and predictive,to support various application circumstances.The AWS platform experiments reveal that this approach can efficiently elastically scale storage resources within 9.73 s at a low cost,allowing data analysis applications to boost their performance by 11.09%-17.75%. |
