Performance Optimization Of Cloud Storage File System Based On Fastdfs

With the development of cloud computing technology,Kodak's future foothold in the forefront of the printing industry is that efficient,safe,fast storage services provided by cloud storage and Kodak image technology are organically integrated into one,Kodak has developed Alpaca cloud storage system based on distributed file system FastDFS,which aims to enable Kodak global users to use file storage and resource sharing service through mobile client and realize the integration of Kodak customers' printing solutions.With the growing popularity of mobile Internet,Kodak users around the world have a tendency to access and use Alpaca cloud storage system through mobile internet.As cloud storage user groups are large and the network is complex,there is a growing demand for it.FastDFS distributed file system is the core underlying component of Alpaca cloud storage system,which was unable to meet the needs of global users for the performance of Alpaca cloud storage system.Since FastDFS is designed for traditional Internet,there are still a number of problems that need to be addressed when it is applied to mobile Internet:(1)Hot data reading performance:When a large number of mobile client users access those hot data,the system will appear too much resource consumption,high latency and slow data transmission and so on.(2)Load balance performance for server cluster:in the trend of cloud storage globalization,the storage system has put forward higher requirements for the performance of load balance algorithm,and traditional load balance algorithm can not meet the requirements of globalization application scenario of cloud storage system.(3)Large file storage performance:first of all,large file will cost cloud storage system a lot of time and traffic in the upload process.Then,large file backup will spend more CPU time.At last,these issues affect the performance of storage node.In order to solve above issues,the paper studies and designs Alpaca cloud storage system based on FastDFS,the main tasks are as follows:(1)For Low reading performance of FastDFS hot file,some cache management rules have been studied and formulated in the paper,such as:precondition of cache is used,file search process,cache elimination mechanism and cache update strategy.At last,a cache management and optimization module is designed and implemented in the paper.(2)FastDFS only has several simple static load balance algorithm,such as polling,selecting the largest residual space and designating server,which can not meet the needs of Alpaca cloud storage system.In the paper,some important load balance factors,such as CPU,memory,disk,network bandwidth,are applied to the existing dynamic load balance algorithm for the sake of load balance performance.At last,the paper designs and implements a load balance management and optimization module.(3)FastDFS does not consider data chunking technology and data deduplication technology for the sake of simplification.Accordingly,the paper introduces a new file access mechanism by applying the existing data chunking algorithm and data deduplication algorithm,and implements a large file storage management and optimization module.Alpaca cloud storage system based on FastDFS is implemented in Linux environment,and cache technology,load balance technology,data deduplication technology are used.Three modules,such as cache management and optimization module,load balance management module and large file storage management and optimization module,are implemented in the paper and integrated with Alpaca cloud storage system,it achieves the design goal of dealing with hot data,balancing server cluster traffic,and improving the efficiency of large file storage.