The Improvement Research Of HDFS File System

It is necessary to boost the performance of distributed file system and the security level and accountable capability of data stored in cloud.However,according to the default HDFS replica placement policy,the placement node selection is random,and then the following issues are uneven replica placement and big consumption in internal bandwidth due to the distance factor when the data needs to be restored.In the meantime,for the users who saved mass data on the cloud-based platforms,HDFS cannot provide sufficient security system to ensure data security.Therefore,it is very necessary to make researches on the default Hadoop replica placement policy and how to provide secure data storage and operation environment.Based on the research and analysis of HDFS,this thesis makes improvements from the perspectives of default replica placement policy and the security of data.This thesis proposes the measures to improve the deficiency of the HDFS default replica placement policy,which can take a series of factors including the distance between nodes,the current node loading situation,the I/O efficiency of the node disk and replica restoration times after optimization into consideration,calculate the matching degree for each node and select the node with the highest matching degree as an optimal node to place replica between the remote racks.The performance test result of the backup data placement policy after optimization indicates that not only load balance of the backup data placement between nodes has been realized but also the internal bandwidth during the data restoration process has been taken into account.As the times of invalid replica has been taken into account,the rapid restoration of the frequently invalid replica can be realized.This thesis designs a reliable third Kerberos accountability solution,which can ensure the data security and audit accountability for data security issues.The reliable third party Kerberos audit accountability solution can ensure security and audit accountability of data on cloud by the user authentication process,the data interaction and audit accountability mechanism.