Research On Optimization Of Bulk Data Transfers For Disaster Backup And Evacuation Among Multiple Datacenters

Geographically distributed(Geo-distributed)multiple datacenters are the in-frastructure of the national big data platform.For lacks of effective disaster-tolerance deployment and transmission mechanism,they are faced with high risk of data loss,underutilization of network transmission capability,heavy burden on critical paths and links,massive data traffic duplication in transmission process,and hysteresis of valued data evacuation.Therefore,we aim to establish disaster tolerance mechanism with reasonable backup deployment,and efficient data transmission among multiple datacenters by the use of Software Defined Networking(SDN)and network intelligence optimiza-tion.We develop integrated mechanism of backup datacenter location selection and evacuation prediction,improve the time-extended network method to con-struct progressive backup load distribution and transmission,provide customized proportion splitting and intelligent multi-path routing selection for multiple con-current data flows,construct capacity-constrained traffic aggregation mechanism for backup data transmission,and design load balancing and congestion con-trolling mechanism for critical transmission paths and forwarding nodes.We leverage intelligent optimization algorithm to solve the problems of backup dat-acenter placement with minimum expected disaster loss and evacuation latency simultaneously,fair distribution and fast transmission of backup load,progres-sively role-switching forwarding and transmission link load balance,cost-efficient backup transmission by capacity-constrained multicast,fast post-evacuation by bandwidth proportion scheduling according to data amount.The research con-tents are summarized as follows:·Backup datacenters provide massive data storage and access services,and their failure may result in huge economic losses.So their location selection requires low damage risk and high evacuation capability simultaneously.With global view of network resources in the SDN scenarios,we propose a new disaster-and-evacuation-aware backup datacenter placement strategy.To reduce backup loss risk and apply rapid post-disaster evacuation,we jointly consider expected disaster loss and evacuation latency,and formu-late a new Disaster-and-Evacuation-Aware Facility Location problem which is multi-objective.To obtain the solution according to disaster situation assessment,we propose a Disaster-and-Evacuation-Aware Multi-Objective Optimization algorithm.We optimize multiple objectives owning different coefficients in different disaster situations.We introduce location-output-capability,backup-evacuation-latency,Pareto-recommendation-degree and node-damage-loss to guide solution searching.We prune external set ac-cording to fitness-average-deviation to improve convergence speed and com-putation efficiency of the algorithm.Through extensive simulations we demonstrate that our algorithm is efficient and promising with less expect-ed disaster loss and higher evacuation capability simultaneously.·To prevent data losses and service interruptions caused by natural disasters or human misconduct,we need to leverage periodic disaster backup among geo-distributed multiple datacenters.Full considerations should be given to the backup redundancy,the limited receiving capacity of backup datacenter-s,and the fast completion of backup data transmission.We propose a new strategy to realize load-fair and bandwidth-efficient disaster backup under redundancy and capacity constraints using customized bandwidth alloca-tion and flexible flow scheduling in SDN.Based on many-to-many relation-ship in disaster backup,we formulate a new Redundancy-Guaranteed and Receiving-Constrained Capacitated Multi-Commodity Flow problem.By constructing Flow-Ratio-Constrained backup transmission model,we spec-ify flow allocation ratio among multiple backup datacenters with limited receiving capacity.To obtain higher performance in redundancy guarantee and enhance bandwidth allocation fairness among massive backup transfer-s,we propose a Fair-Rotating and Ratio-Aware Ant Colony Optimization(FRRA-ACO)algorithm.Especially,we use rotary routing search for mul-tiple concurrent flows based on backup requirement cloning to approximate the upper bound of bandwidth allocation,adjust ratio of bandwidth alloca-tion for multiple backup transfers with different requirements,and further improve flow rate according to the maximum link utilization on links if possible.Experiments demonstrate that FRRA-ACO outperforms state-of-the-art algorithms with less backup transmission time,fairer backup load distribution and higher network utilization.·The periodic disaster backup activity among geo-distributed multiple dat-acenters consumes huge network resources and therefore imposes a heavy burden on datacenters and transmission links.In this paper,we propose a new progressive forwarding disaster backup strategy in the SDN scenarios to mitigate forwarding burdens on source datacenters and balance backup loads on backup datacenters and transmission links.We construct a new Redundancy-Aware Time-Expanded Network model to divide time slots according to redundancy requirement,and propose role-switching method over time to utilize forwarding capability of backup datacenters.In every time slot,we leverage two-step optimization algorithm to realize capacity-constrained backup datacenter selection and fairer backup load distribution.Simulations results prove that our strategy achieves good performance in load balance under the condition of guaranteeing transmission completion and backup redundancy.·To guarantee backup redundancy requirement,previous works employ dis-joint unicast paths for bulk data transfers among multiple geo-distributed datacenters,causing massive unnecessary traffic duplication.This not only adds the overhead,but may result in severe network congestion.With flexi-ble network resource management in the SDN scenarios and powerful traffic aggregation capability of multicast,we propose Capacity-Constrained Mul-ticast to realize cost-efficient disaster backup.First,considering limited backup storage capacity and backup redundancy guarantee,we construct a new Capacity-Constrained Multicasting Backup model.Then we formulate the disaster backup problem as Capacity-Constrained Multiple Steiner Tree problem which is NP-Hard.To solve this problem,we design a new Multi-casting Backup Ant Colony Optimization algorithm based on requirement-aware-growth.It directly optimizes every disaster backup multicast tree from its root node to cover enough destination nodes guaranteeing sufficient redundancy,and then expands them into the forest under guidance of the multicast tree shared degree,the ratio of available storage capacity and the backup load distribution offset.We introduce unique edge fitness evaluation and pheromone for every disaster backup multicast tree to reduce mutual influences among multiple trees.Extensive simulations demonstrates that our strategy performs with less bandwidth consumption cost and relatively good backup load distribution fairness simultaneously.·Disaster evacuation assigns bulk endangered data to geo-distributed dat-acenters out of disaster zone within acceptable duration.For the band-width allocation proportion problem and multi-path routing problem among multiple concurrent evacuation transfers(especially with shared links),we leverage flexible traffic scheduling in the SDN scenarios,and propose a new optimal bandwidth proportion allocation strategy.To maximize disaster evacuation capability,we formulate the bandwidth allocation problem as a new Bandwidth-Proportion-Constrained Multi-Commodity Flow problem.To obtain optimal solution for practical networks of large scale,we pro-pose a Bandwidth-Proportion-Aware Ant Colony Optimization algorithm to achieve the maximum evacuation flow matching data amount proportion of concurrent evacuation transfers.We introduce available evacuation ca-pability,bandwidth proportion offset and link sharing degree to guide the optimal solution searching.We adjust bandwidth proportion by rearranging flows in shared links and alternate paths.Through extensive simulations we demonstrate that our algorithm has better performance with less total evacuation time and higher network utilization.