Key Technology For Peer-to-Peer Cloud

Peer-to-Peer network (P2P) succeeds in making network resources developed from a centralized manner to a distributed manner and making individual users participate in network applications and calculations. Therefore, it achieves an Internet environment with free interconnection. At the same time, along with the arrival of Big Data era and the emergence of User Generated content, there has been explosive growth in network data. The development of cloud computing and big data center effectively solves the massive information storage and processing problem in Big Data era. However, another serious problem is attendant:the energy and electricity consumption in data processing centers grows rapidly. One kind of solution is to deploy a class of lower power consumption hardware and/or software, but this will severely limit the service scope and scalability provided by operators. Another method is to place some service on peer nodes, which will contribute out their communication, storage and computing resources to exchange data and service, and data centers perform centralized management and authentication. The successful deployment of Wuala and Valancius demonstrates that the architecture based on Peer-to-Peer Cloud (P2P-Cloud) is not only able to ease the demand of computing and communication, but also can effectively reduce the power consumption. Its energy efficiency is more than five times of the C/S architecture. In addition, the P2P-Cloud platform can fully satisfy the demand of operation cost, system flexibility and scalability. Therefore, P2P-Cloud data center is increasingly becoming a trend to build and develop big data processing system.For P2P-Cloud there is not only a need to solve the old problems existing in P2P, but also to design new approaches for cloud performance optimization and job scheduling to satisfy the new demands for massive data processing of the cloud architecture system. For P2P routing topology discovery, it usually first obtains other network parameters, such as the delay correlation, and then treats it as the basis for topology inference. Therefore, the accuracy and deployment cost of delay correlation measurements will directly affect the accuracy and usefulness of the topology tomography. In order to solve the problem that it requires node collaboration for Round Trip Time Measurement (RTT) and the issue that it needs precise clock synchronization for One Way End-to-End Measurement (OTT), in this thesis a heuristic delay correlation measurement method named DCE is firstly proposed. It is not only able to require no complex node coordination but also avoid any clock synchronization. It greatly reduces the difficulty of deployment and improves tomography accuracy in practice. Secondly, in order to solve a series of problems in P2P-Cloud networks caused by churn, such as that how to make peers dynamically join and find a better transmission paths, etc, a passive network topology recovery method based on DCE delay correlation measurements in P2P-Cloud is developed. Finally, for the design and optimization of job scheduling in P2P-Cloud data centers, combined with the bandwidth control capabilities of OpenFlow, a novel task scheduling mechanism based on bandwidth aware with SDN named BASS is proposed. To our knowledge BASS is the first to exploit combining the SDN’s advantages with Hadoop system. By deploying BASS in real world Hadoop clusters, the efficiency of this scheduling mechanism is demonstrated.