Research On Data Center Network Scheduling And Applications

The advent of cloud computing era has transformed the traditional resource-specific data center model into a new type of resource-reuse model.On the one hand,the structure of a data center network has developed from the traditional hierarchical tree structure to a flat symmetric structure.On the other hand,the network traffic has evolved from the North-South dominated traffic to that of the East-West.Therefore,the traditional scheduling mechanism for data center networks is not adaptable to the network structure and the traffic pattern of cloud computing data centers.On the other hand,the emergence of the resource reuse model makes the applications of the original data centers no longer suitable for cloud computing data centers.As a result,there are a large number of new applications emerging in cloud computing data centers.This thesis mainly focuses on the two main lines of the network scheduling and the study of new applications in cloud computing data centers.The following three aspects have been carried out in this thesis.1.Data center network Coflow schedulingIn the literature,the scheduling methods of data center networks are mainly based on flow granularity.They do not perform well in the scheduling of the data flows generated by parallel computing frameworks in the current data centers.The reason for this is that the scheduling methods ignore the semantic dependencies between the parallel data flows.The study on the scheduling methods with Coflow granularity is aimed at solving this kind of problems.This thesis designs a centralized real-time dynamic Coflow scheduling system,named Seagull++.The scheduling system targets at reducing the average Coflow completion time and improving the number of the Coflows that satisfy the deadline.It well implements the real-time dynamic scheduling of the Coflows and achieves the desired performance by implementing the modules of Coflow information acquisition and bottleneck detection and combining with a heuristic scheduling algorithm.Experiments on small-scale real test platforms and large-scale simulations have proved that the Seagull++ system can achieve good results in reducing the average Coflow completion time and increasing the number of deadline Coflows.2.Data Center Application-Multi-Tenant Remote Virtual SystemsThe rapid development of computer technology has accelerated the increase of the demand of the public for computing resources.People are constantly upgrading their personal computing devices to meet this demand,which in turn leads to high cost and waste of hardware resources.The birth of virtualization technology and the development and maturity of cloud computing have brought an effective solution to this problem via remote connection to the virtual resources in cloud computing data centers,sharing of the existing personal computing devices and reuse of hardware resources in the data center.However,there are a variety of personal computing devices in reality,including personal computers,smartphones,and wearable devices,etc.If a proprietary remote virtual system is designed and implemented for each type of devices,it will bring in many problems such as confusion to the standards,poor compatibility,and difficulties in maintenance.This thesis focuses on building a universal multi-tenant remote virtual system.It builds a necessary bridge between cloud computing data centers and personal computing devices.The main contributions of the system are as follows.On the one hand,since the proposed system is with a universal system framework,a specific virtual system can be implemented through modular resource adaptation.With the remote virtual system,a personal computing device only needs to install a simple application program to use the computing resources of remote data centers.Thus,it reduces the configuration requirements on the personal computing device,and protects the security of personal sensitive data.Moreover,it facilitates the centralized management and deployment.On the other hand,an algorithm for placement of virtual computing units has been designed in the system based on an improved ant colony algorithm to meet the needs of simultaneous use of multiple tenants.It can fully utilize the hardware resources of the data center and avoid unnecessary waste.To verify the versatility of the proposed system,this paper builds examples of applications for two types of personal computing devices,i.e.,personal computers and smartphones.Benchmark performance tests and user experience tests are performed in different scenarios with the two different types of devices.The test results show that the universal multi-tenant remote virtual system proposed in this thesis can,on the one hand,make full use of the computing capacity of data centers,avoid the waste of hardware resources,and has good scalability;On the other hand,the system is fully applicable to low-end client computing devices and it also brings a good user experience to end users,such as high-quality low-latency video display and low battery consumption.3.Data Center Application-Heterogeneous Distributed Deep Neural NetworksThe intelligent applications based on deep neural networks are usually deployed on highperformance servers in data centers.The related research focuses on how to improve the performance of deep neural networks.Since high-performance servers in data centers usually exist in clusters,distributed deep neural networks have become one of the most important methods for enhancing deep neural networks.However,the use of this type of distributed deep neural networks is limited to homogeneous servers within the data center,which ignores the data source collectors of the deep neural networks,i.e.,the terminal nodes equipped with various types of data sensors.This neglect has created a blind spot in the direction of research.There has been little research on hierarchical heterogeneous distributed deep neural networks.To this end,this paper proposes a new hierarchical heterogeneous distributed deep neural network(HDDNN)framework.It is an intelligent application that takes the data center as the core and combines the concepts of artificial intelligence,cloud computing,internet of things,and pervasive computing.The HDDNN framework forms a hierarchical architecture by linking cloud nodes,edge nodes,and terminal nodes.By combining the characteristics and capabilities of each type of nodes,it finally designs and achieves a new data center application with distributed computing node heterogeneity,distributed neural network heterogeneity,and distributed system tasks heterogeneity.