Research And Implementation Of Network Storage System Based On Web Service

The large-scale commercial network storage system supplies the service of storage for users over the world. Anyone can store local data in the internet anywhere, which makes people's daily life easy. The network storage developers not only need to solve the problems of heterogeneity, consistency, openness and security in the distributed system but also need to face the difficulty of expensive machine and economic cost of network bandwidth. Especially the high economic cost makes the development of network storage system very hard. With the development of free storage service (FSS) just like email, network hard disk and FTP, the limited and low-quality services temporarily satisfy the requirement of network storage system, but with the development of information technology, FSS will be not satisfactory enough.The network storage system ppStore solves the problems above efficiently. We realize that though the service quality of single FSS is limited, the quantity of FSS is huge and it is easy to get this FSS. The ppStore integrates the interface of different FSS to achieve the mechanism of data transfer between different FSS and unify multiple FSS, at the same time ppStore unifies the service of different FSS to supply service. Because we only possess simple right to use FFS but don't possess the complete right to control it, ppStore creates an adaptive mechanism to automatically collect the performance parameters, analyze the metadata intelligently, schedule the tasks and allocate the tasks. Besides, we use the technology of Erasure Codes to cut the file to pieces quickly and transfer the file, we also adopt the technology of P2P overlay and cache to establish a mechanism of three level data access to ensure the high performance of data. We propose a lazy backup strategy about deciding when to start the backup process according to the statistical analysis of DOS (Denial of Service) and metadata.The result of system test and simulation reveals that ppStore achieves high performance data transfer among client, cache server and FSS, we control and allocate the tasks of FSS reasonably to reduce the rate of DOS to 2% and make the speed of data transfer be 500KB/S. By the processing of instruction classification and clustering, not only cache hit ratio in our system is improved by two times higher comparing to the traditional cache system, but also the latency between clients and master server is reduced. Single master server could manipulate 13,000 users simultaneously as the test shows.