Research And Implementation Of Secure Storage System Based On HDFS

With the rapid development of Internet technology,cloud computing,as an important research direction,has produced a wealth of cloud computing products at home and abroad,such as Alibaba Cloud,Huawei Cloud,and Amazon's AWS.The rapid development of cloud computing has made cloud storage an important research direction for current scientific research and corporate researchers.Cloud storage stores user data in a cloud server in a distributed manner.It has the advantages of virtualization,multi-user,and high scalability.HDFS file system is the underlying storage system of the open source framework Hadoop.Because of its high reliability,high scalability,and high fault tolerance,it is widely used as a file storage system solution for cloud computing platforms of major companies.However,with the increasing attention to data security issues,HDFS file systems still have some security risks that must be improved.Therefore,this paper relies on the national key R&D plan project to design and implement a HDFS-based secure storage system.This article focuses on HDFS-based secure storage systems.Firstly,it analyzes the security problems of the current HDFS file system,and then based on the above analysis,designs and implements a secure storage system based on the HDFS file system improvement based on permissions control security issues,cluster state security issues,and data storage security issues.This system implements the file management function on the basis of the HDFS file system,and supports users' visual operations on files.Aiming at the security problem of permission control,an authorization management component is designed to determine the authorization of user operations through a dynamic authorization management model based on roles and environments.Aiming at the problem of cluster status security,a monitoring and early warning component was designed to analyze the system performance by monitoring the status of each node of the system,and to issue an alarm when the system performance reached a threshold.Aiming at the problem of data storage security,especially the data leakage caused by data residues,a data destruction engine was designed.The engine applied a data destruction algorithm based on data block marking and ordering to achieve high-speed and reliable data destruction.A HDFS-based secure storage system designed and implemented in this article effectively manages user operations,promotes the solution of data privacy issues in an open network environment,and prevents the risk of data leakage caused by data residual issues.Provides a safe and reliable file storage system,which promotes the development and application of big data storage technology.