Research On Optimal Deployment And Scheduling Policies Of Big Data Applications On Cloud Computing Platforms

Alongside with the immense proliferation of big data applications, cloud computing platform has become the most suitable platform for processing big data. The virtualization technology makes the resources in cloud platforms more flexible, which stratifies the dynamic resource requi rements of big data services. Moreover, the ―pay-as-you-go‖ manner of cloud services makes it practical for smaller enterprises and non-technical tenants to access big data applications. Correspondingly, as one of the key evaluation indicators, the performance of cloud-based big data applications has attracted increasing attention in both industry and academic communities. Aiming at improving the performance of cloud-based big data applications, this dissertation conducts a study on the deployment and scheduling policies of cloud-based big data applications. The main contents and contributions of this work include :1. At the infrastructure level, this dissertation proposes a novel mechanism TOMON, to optimize the topologies of virtual clusters. This dissertation analyzes the conflicts for optimizing the two performance factors(data transmission latency and data processing rate) in a virtual cluster, and then put forward a novel virtual cluster deployment mechanism TOMON to improve the overall performance of a virtual cluster. According to the ex perimental results, TOMON mechanism strikes the right balance between data transmission latency and data processing rate, and improves the overall execution latency of cloud-based big data applications.2. At the platform level, this dissertation proposes a novel mechanism RDMOC, to optimize the deployment of big data service replicas. This dissertation analyzes the impact of application features and device configurations to the installation latency of service replicas, and designs a performance model to formulate the overall image installation rate of a storage platform. By solving the performance optimization model, this dissertation designs a novel mechanism RDMOC, which determines the optimal replica number of each service based on their application char ateristics; and furthermore, optimizes the location of each service replica according t o the performance of the physical platform. Besides service execution latency, service installation latency is further optimized using RDMOC.3. At the application level, this dissertation studies the optimal collaboration of different ordering and dispatching policies, in the two-phase scheduling scenario of cloud tasks. Using the Stochastic Petri Net(SPN) model, this dissertation theoretically and consistently model s the execution processes of different ordering and dispatching policies, and analyzes the performance of a cloud service under different collaboration s of ordering and dispatching policies. Furthermore, this dissertation studies and evaluates the throughput optimal collaboration of ordering and dispatching policies based on the modeling results. Based on the deployment mechanisms proposed at the infrast ructure and platform layer, the proposed scheduling policy further optimizes the performance of multiple services at application level.