Improvement Of Distributed File System TFS In Cloud Storage

Facing the current status of explosivly increasing data and that most of them are small files,the traditional storage technology has been unable to meet the needs of large data.Thus,the appearance of TFS(Taobao File System)solved the problem of massive small file access to a certain extent.However,on one hand,since the TFS file system uses the main block and the extension block to store data,the DataServer server space utilization is not high.On the other hand,as the TFS file system uses the Erasure Code algorithm instead of multiple copies of the data,it also has the problem of data failure and great recovery overhead.Therefore,it is of great significance to research the optimization of extension block and Erasure Code algorithm of TFS in order to solve the problem of massive small file access.Based on the in-depth analysis and research of TFS extension block optimization and Erasure Code algorithm,and aiming at polishing up Data Server server space utilization,this thesis breaks the main block and extend block boundaries and proposes a new method that in the phase of DataServer partition of the main block and the expansion,in which only the main block space is allocated,and then the main block is divided into a plurality of blocks for extended use just when the extension block need to be used.Aiming at reducing the overhead of the multiple backup data and the failure of data recovery,this thesis proposes an improved Erasure Code algorithm,in which the check block is divided into global and local check block to avoid the situation that the traditional data recovery need read the data by group the number of data block.By reading data blocks which are less than the number of data blocks to restore the failure data block,the improved algorithm can effectively reduce the data recovery overhead within network and read.Thefore,the storage performance of TFS file system is really improved.In this thesis,the performance of data access and simulation and the failure recovery process of improved TFS are tested,and the test results show that the extended block optimization can improve the utilization of DataServer server space,and the improved Erasure Code algorithm can reduce the data recovery within network and read,achieving the desired effect.