Analysis And Research On Distributed Symmetric Encryption Of Data Storage Based On Hadoop

With the rapid development of Internet technology and distributed data storage, the usage of the cloud platform is becoming more and more popular between the enterprises. At the same time, the interest of the data which are stored in the cloud platform has been raised to an extremely high level by the datas’ owner. People have higher requirements about the security of cloud platform data storage.Hadoop allows users to architect and use the distributed platform easily, and then process lots of massive information distributed. But the data storage based on Hadoop has big security problems currently and is lack of static data encryption. About the security part, it only focuses on the validation and backup of data, but pay no attention to the security of cloud files itself. So security issues frequently come out. The main content of this paper is to apply the encryption technology to Hadoop cloud platform, and achieve the goal of distributed and symmetric encrypting the data which are stored in the cloud platform, in order to improve the security of the data in the cloud.First of all, this paper introduces the present domestic and foreign research, situation and development trend of the cloud computing and distributed data storage. Make a detailed explanation about the two key technologies of Hadoop open resource platform: the Hadoop Distributed Filesystem(HDFS) and the MapReduce programming model.Secondly, through the comparison of the existing encryption algorithms and the analysis of the problems in traditional distributed data storage and HDFS, the paper combined with the programming model of Hadoop distributed clusters,and summarized a symmetrically encrypted data technology based on it. At the same time, give it a brief analysis.Then the paper designed each part of the new module, and introduced the entire process of using the MapReduce to implement the encryption AES(Advanced Encryption Standard), especially focuses on the realization of the relevant mapping and reducing function.Finally, build a real Hadoop cloud computing platform through the virtual machine in laboratory to achieve the process of encryption and decryption. And verify the whole process by function, expansion and efficiency test. By comparing the test data, the paper showed analysis of the simulation performance and put forward the optimization. At last, it gives the summary and outlook of the work.