Research On Data Management In Peer-to-peer Storage System

P2P, which has been a lot of popularity and application in the real world, is one of the main technologys to build distributed architecture. As P2P main study contents Peer-to-Peer (P2P) storage system is a hot topic in recently year, which goal is to make full use of all possible edge node of wide area network environment for large-scale collaborative computing and storage sharing. However, data availability and security are two potential problems to users in the running of storage system because of openness, self organization and high dynamic characteristic of peers. Most P2P system problems can be attributed to the data management issues, such as data distribution and placement, data retrieval, etc. And these data management issues still exist in the emerging cloud computing, social networks, Cyber-Physical-System. In the cloud computing, due to the relative concentration of resources, it can achieve intensive management. While P2P system scattered manages and stores large number of files or data storage resource, it is difficult to protect the reliability of resources. Therefore, the study of storage system data management issues is the key for solving availability, reliability, scalability and security. Cloud storage is the product combined distributed storage with virtualization technology, and is the latest development of the distributed storage technology. P2P technology in this new computing model still has the vigor and vitality.In large-scale P2P storage systems, data management is the core issues of entire life cycle of system running. This paper tries to analyze and research on storage systems data management from the perspective of whole system, including the data itself and metadata management. The former is mainly related to data copy management, while the latter is to data distribution, resource search and find, namely metadata representation and organization. Finally, a semi-structured P2P storage prototype system, Scool, is designed and implemented. Based on systematically summarizing the relevant works on P2P storage technologies, this paper focuse on the related teehnologies of P2P storage data management and gains several achievements on sub domains and related technologies. The major contributions of this dissertation are as below:(1) Designed a data calculation model for the number of copies. Copy redundancy is key technologies to improve data availability and concurrent access efficiency. P2P storage system copies data management include data redundancy strategies, data distribution and data recovery and other issues. This paper designs the number of copies combining nodes online and resources access rate, which consider copy number according to access frequency of hot data resources, bypassing the calculation of non-deterministic way of system availability. To improve resource availability and increase the speed concurrent access to data resources, it reduces data replica consistency maintenance overhead. In terms of different role of node to adopt different repair strategies, uncertainty caused by high dynamic of P2P system node is resolved effectively, which save network bandwidth, reduce latency, improve data query performance.(2) A novel interest gathered storage systems metadata management mechanism is proposed. Research that aimed at P2P storage systems metadata management is still relatively scarce. Based on semi-structured P2P storage systems, Interest aggregated-based Bloom filter in Semi-structure P2P storage system (I-BSP2P) is addressed, which used interest aggregation and hierarchical Bloom Filters to locate and organize metadata, and routed the metadata query request to different metadata servers. Experiments showed that this method had good adaptability and scalability and significantly improved metadata query efficiency and access performance.(3) Designed and implemented a genre intertexuality P2P information retrieval model. Currently research against P2P information retrieval mostly concentrated in the resource search mechanism. While P2P storage system is multi-structured data storage system, which has multiple data sources, variety data representation. And different users may use different data to describe the same real world entity, which would induce the problem of semantic heterogeneity. This is the first to extend genre intertextuality concept of linguistics to task context of P2P information retrival based on the concept of genre intertextuality and address the problem of document genres automatic identification. Meanwhile, inquiry target and document target is denoted by using genre and the problem that traditional method is difficult to explicitly express inquiry target with genre is solved. This can provide certain reference to achieving and improving of P2P information retrieval technology and give new vision and methods to study on genre intertextuality theory of linguistics. Research focuses on three areas. Firstly, the basic concepts, nature and measurement of genre intertextuality in task context, expanding the concept of genre intertextuality to task context, researching on the nature of its quantitative, establishing intertextuality strength measurement methods. Secondly, distinguishing intertextuality document genre based, proposing concept of purpose genre according to the relationship theory, building algorithm for document genre automatic identification. Finally, predicting query intent using genre and genetic algorithm based on task context. Experiments show that good results is achieved in the design method in three metrics, the recall rate, the average accuracy rate of the mean, bpref, which are typical evaluation indicators of P2P information retrieval.