The Mechanisms Of Optimizing Resource Allocation In The Big Data Transfer Networks

With the growth of data,data-driven scientific computing leads to the fourth paradigm for science discovery.The emerging cloud and data center services intensify this bandwidth competition,and bring the “age of big data”.The emerging data center and cloud services carrying large blocks of data lead to an intensive bandwidth competition on today's Internet,especially during peak hours.Large flows co-exist with small flows and often consume a significant proportion of the network bandwidth.This imposes a challenge on network resource management.At the same time,the traffic in the bid data age is also showing new features.Taking advantage of these traffic characteristics will become an effective way to study the mechanisms of optimizing resource allocation for big data transfer networks.At present,many studies have studied the mechanisms of optimizing resource allocation for big data transfer network,but there are many problems and challenges that need to be solved:(1)the packet switched network has a low bandwidth utilization,and network has a high congestion degree in the peak hours as the elephant flows consume most of the bandwidth;(2)in the hybrid switching networks,there is still a need for means for design and dimensioning of resources in multiple switching planes and the understanding of the system behavior under different traffic patterns.Therefore,in order to solve these resource allocation problems and challenges in the big data era,this thesis focuses on:1.Research on the optimizing bandwidth resource allocation in the packet switched network when elephant flows and mouse flows coexist in the networkIn order to deal with the resource allocation problems in packet switched networks,we propose a mechanism of optimizing resource allocation that decrease the bandwidth consumed by elephant flows in the peak hours and increase the user experience of mouse flows.We use one simple yet effective congestion pricing scheme based on the number of flows,in single bottleneck networks carrying elastic traffic.During congested hours,data transmitters share bandwidth on the bottleneck and also are charged,according to the number of flows they use.By using this resource allocation mechanism,all users can achieve maximum utility and network capacity is efficiently used with increasing and strictly concave utility function.We investigate how the single congestion control parameter,the price index,affects network congestion degree.Through network simulation,we prove that with the proposed scheme,the big data transmitters sharply reduce their transfer during congested hours.2.Research on the resource allocation between different switching planes in the hybrid switching network when elephant flows and mouse flows are transmitted separatelyWe introduce a generic framework called the Blocking-LOss Curve(BLOC)to tackle the resource allocation problem in hybrid switching systems.The BLOC framework can identify all the possible combinations of resource allocation and traffic partitioning(which we refer to as the feasible region)that would satisfy both the packet loss and request blocking requirements.Using this feasible region,we can find an optimal resource allocation strategy that minimizes energy consumption.We also model a specific hybrid switching inter-datacenter network and a hybrid switching intra-datacenter network using the BLOC framework.The numerical results demonstrate how the flow size distribution,circuit reconfiguration delay,performance requirements and other system parameters may affect the area of feasible region and the optimal resource allocation.We also show how system parameters,such as flow arrival characteristics,may affect resource allocation and system costs.Our study reveals that a large amount of resource can be saved when the percentile of flow arrival rate decreases slightly.By defining the optimization objectives,the optimal resource allocation can also be identified.