Design And Implement Of Distributed Storage System Based On Virtual Disk

With the upsurge of global network, network technology for personal applications is also soared. Traditional storage solutions have been unable to cope with the many challenges of massive data and the need from various applications. Large-scale distributed storage system, which has the advantages of high scalability, high availability, etc., has become the research and development direction in the storage areas and has been widely used in the major Internet companies and corporate. Combined with the current booming distributed storage technology, this paper presents a distributed storage system based on the virtual disk for the characters of massive data and client-specific kernel mode block data.The distributed storage architecture presented in this thesis, possessing a well control scheduling mechanism, hash algorithm positioning strategy and security authentication services, designs a Log-Structured storage system for the massive data based on the LogFS. The file system that relies on three memory storage medium, ssd and disks, can take full advantage of disk IO bandwidth, improve the efficiency of reading and writing.This storage system on on tertiary storage medium: memory, SSD and disk, can make full use of the disk I/O bandwidth, improve the efficiency of reading and writing.Data Storage System based on Log-Structure(Log-Structure Storage System, L3S) is deployed on each storage node of distributed storage cluster in this thesis. According to the data storage model based on LogFS, the user data are divided into fixed size of data fragmentations and then organize these fragments in a Log-Structureand which would been appended to the disk when it is filled. This method of batch writing make full use of the disk transmission bandwidth and reduce disk fragments. With the change of the data, the metadata of data fragmentations are also constantly updated. When the available disk storage space is not enough, the system would using a certain strategy to recycling the invalid data space. At the same time, the data storage system put all metadata and part of the data cache in the SSD, and keep a log structure in memory as a mid-tier cacheof the persistent medium and network cache.After the basic function tests and performance tests, it is confirmed that the system in this thesis can run well, and can efficiently complete the data service.