| Network Function Virtualization(NFV)makes the deployment of network services more convenient and flexible by decoupling network functions from dedicated hardware devices.With NFV technology,service providers(SPs)can deploy virtual network functions(VNFs)in data centers(DCs)(in different locations)to form various VNF service chains(VNF-SCs)for providing users in different regions with diversified network services.Meanwhile,the emerging services,i.e.,games and online conferencing,make the traffic going through VNF-SCs demand for large bandwidth and exhibit high burstiness and dynamics.Since the elastic optical network(EON)can easily accommodate high throughput and burst VNF-SC traffic with flexible spectrum slot allocation mechanism,inter-DCs EON(IDC-EON)is considered as a promising infrastructure for supporting VNF-SC deployment.In order to provide real-time and economical network services in IDC-EONs,SPs need to deploy the required VNFs within suitable DCs and establish lightpaths among them to form VNF-SCs.The whole process is called VNFSC orchestration and it mainly has the following difficulties:·Excessive VNF-SC setup latency.A VNF-SC deployment in an IDC-EON involves instantiating VNFs in DCs and setting up lightpaths in EON.Its total setup latency can easily reach the order of minutes,which would jeopardize the Quality of Service(QoS)of latency-sensitive service.·Lightpath establishment in VNF-SC orchestration.The lightpath establishment in IDC-EONs includes routing and spectrum allocation(RSA),which has been proved to be NP-hard problem.Since DCs are geographically distributed,an IDC-EON usually consists of multiple different carriers.Considering the domain heterogeneity and autonomy,inter-domain RSA problem would be more complicated.·VNF-SC orchestration in dynamic IDC-EONs.Since VNF-SC orchestration needs to jointly optimize VNF placement and RSA,the time-varying network state in the dynamic IDC-EON would make it more difficult.Regarding the above three challenges,this article takes advantage of intelligent algorithms(e.g.,deep learning(DL)and deep reinforcement learning(DRL))due to their excellent performances on high-dimension data prediction and complicated decision problems,and conduct researches on 1)pre-deployment based VNF-SC orchestration framework,2)multi-agent DRL for scalable inter-domain lightpath establishment,and 3)hierarchical DRL for dynamic VNF-SC orchestration in IDC-EON.The detailed research works are listed below:Pre-deployment based VNF-SC provisioning framework.In order to address long setup latency,this article proposes the pre-deployment based VNF-SC provisioning framework for the first time.It is based on cyclical VNF-SC request service process and each service cycle contains a pre-deployment phase and a provisioning phase.The pre-deployment phrase uses a DL based request prediction model to predict upcoming requests in current cycle and deploy resources accordingly.Then,in the provisioning phase,SPs only need to steer requests traffic through required VNF-SC to complete service.For realizing accurate VNF-SC request prediction,this article designs a Long Short-Term Memory based DL model and develops a loss function and a training scheme for it according to the high-dimension data features of VNF-SC requests.The simulations are conducted on real network data and the results demonstrate that LSTM based DL model achieves the lower prediction loss and request blocking probability than two benchmarks with the same framework.In addition,to further improve the adaptability of the proposed framework,the article introduces a DRL based monitoring model,namely DRL-Decision,to adjust the duration of each service cycle according to network load state.DRL-Decision is designed based on Actor-Critic,which contains an actor Neural Network(NN)for service cycle decision and a critic NN for evaluating and guiding the decision.The simulation results under different dynamic loads show that DRL-Decision can converge fast with the help of a few asynchronous training threads,and the improved framework with it can achieve trade-off among resource utilization,reconfiguration time and blocking probability.Multi-agent DRL for scalable inter-domain lightpath establishment.To cope with the high computational complexity and domain autonomy constrains of routing in multi-domain EONs,this article proposes a multi-agent and cooperative DRL based routing framework,namely DeepCoop,and designs a coordinate mechanism considering domain autonomy constrains to realize scalable cross-domain routing.Specifically,it deploys a DRL agent in each domain to optimize intra-domain routing,while a domain-level path calculation element is used for calculating the domain sequence to go through.The cooperation among DRL agents is realized by sharing limited states and cooperative reward calculation.In order to ensure universality and scalability,the actions of each DRL agent are designed as the choice of RSA heuristics.Numerous simulation results demonstrate that DeepCoop can accommodate different multi-domain EONs(e.g.,different topologies or traffic loads),and can always select the suitable RSA heuristic at different network states to achieve lower blocking probability.Moreover,this article verifies the scalability and universality of its distributed training scheme for multi-domain EONs with different scale topologies.Hierarchical DRL for dynamic VNF-SC orchestration in IDC-EONs.For accurately describing state transition process in dynamic IDC-EONs,this article models the VNF-SC orchestration in dynamic IDC-EONs as a Markov decision process(MDP),and proposes a graph NN(GNN)based hierarchical DRL(HRL).In order to ensure its universality and scalability,HRL’s policy NN is designed based on GNN.Because GNN based policy NN can directly extract features from graph-structured network states of IDC-EONs,it can be applied to any size or form of IDC-EON topology without any modification.Then,by analyzing the VNF-SC orchestration process with the goal of minimizing blocking probability,it is found that the dynamic IDC-EON is a sparse reward environment.For addressing the problem,this article designs the HRL with lowlevel and high-level DRL models to improve the training convergence.The low-level model aims to optimize the resource usage of each VNF-SC orchestration process,while the upper-level model coordinates the orchestration of all arrived VNF-SC requests to minimize the request blocking probability.Therefore,they can collaborate to optimize the VNF-SC orchestration in dynamic IDC-EON during the training process.The simulation results verify the universality and scalability of HRL,and demonstrate it can achieve better performance than existing VNF-SC orchestration algorithms. |
