Desigh And Implementation Of Erasure Coding Based On CUDA

With the continuous development of the Internet,the need for storing and processing data grows rapidly.As a result,the technology of big data has gained a mushroom growth.In addition,how to store such massive data with ensuring the availability of the data has become a hot topic.In order to ensure the availability of data in the cloud storage systems,at present,most of the cloud storage systems adopt multiple replication technology.Although the implementation of the multiple replication technology is simple,its low storage efficiency significantly increases the storage cost of the cloud storage systems.In order to solve this problem,the erasure codes have been used more in practice.Compared with CPU,GPU has higher processing power and bigger memory bandwidth.In recent years,the continuous development of GPGPU(General-purpose computing on graphics processing units)technology using the CPU+GPU heterogeneous model,broadens the application field of GPU,with the complex logic processing and transaction managements dealed with by CPU and large-scale data parallel computing dealed with by GPU.And NVIDIA launched CUDA(Compute Unified Device Architecture)for developers to use the powerful performance of GPU effectively,and CUDA makes a significant improvement in GPU's framework.As a result,CUDA is more suitable for GPGPU.Today,CUDA's performance has been much better than the traditional GPGPU technology,and CUDA has been used in a wide range of applications in various fields.In this paper,in order to solve the problem of erasure codes in the time cost of encoding and decoding,the erasure code algorithm is optimized in both framework and CUDA.The main work can be included in the following aspects:1)In this paper,different methods used by storage systems to ensure the availability of data,are introduced,including RAID,multiple replication and erasure codes.2)The principles and characteristics of different erasure codes are introduced,as well as skills used in practice while using erasure codes.3)The related knowledge of CUDA,and how to use CUDA to accelerate a program is introduced.4)Some open-source libraries of erasure codes are introduced,especially Jerasure.Besides,the the decoding performance of reed_sol_r6 algorithm is optimized.5)CUDA is used to optimize the system,and the system performance is improved greatly.6)We summarize this paper's work,and put forward the research direction in the future.According to the final test result,the new system shows its ability to ensure the availability of data,and both encoding and decoding performances are improved greatly by using CUDA.The encoding speed is increased by up to 20 times and the decoding speed is increased by 5 to 50 times,which makes it possible for erasure codes to use in the cloud storage system which is real-time.