Research On Migration Strategy Of Service Function Chain Based On Reinforcement Learning

As Network Function Virtualization(NFV)expands from the network center to the edge,Service Function Chaining(SFC)can be deployed more flexibly at the network edge to provide users with higher-quality services.However,the mobility of users might affect the quality of services and even cause service unavailability,so it is necessary to migrate the SFC in time to follow the user movement.The scenario where the user’s movement path is known and the arrival time is predictable is a very common mobile scenario in life,which is called the userpredicted movement scenario.In this scenario,combined with the predictability of user mobility,migrating SFC in advance for mobile users can provide better and more stable services for user experience.However,the user’s service downtime and the migration success rate of SFC are closely related to factors such as migration timing,migration path,and user-predicted arrival time.Currently,there are no research papers on the SFC migration timing and migration path decision problems in the user-predicted movement scenario.Therefore,in order to reduce service downtime and improve the migration success rate of SFC,this paper conducts research on the decision-making problems of SFC migration timing and migration path in the user-predicted movement scenario.Aiming at the decision-making problem of SFC migration timing in the user-predicted movement scenario,this paper establishes a prediction model of user arrival time and expresses the SFC migration process as a mathematical model,and analyzes the impact of migration timing on SFC service downtime and migration success rate.Then,the paper models the SFC migration process as a Markovian decision process,designs the state space,action space and reward function,and proposes a deep Qnetwork-based SFC migration timing decision(DQN-MTD)strategy.The DQN-MTD agent can deeply perceive the state changes of network resources,and combine SFC migration information to decide the appropriate migration timing for Virtual Network Functions(VNFs)migration.The experimental results show that,compared with existing algorithms,the DQN-MTD algorithm can reduce the average service downtime by about 14%,improve the migration success rate of SFC by about 20%,and reduce the average VNF migration time and migration memory when network load is low;and as the network load increases,the DQN-MTD algorithm can continuously adjust the migration scheme,giving priority to ensuring the success rate of SFC migration.Based on the above research,aiming at the VNF migration path decision-making problem in the SFC migration process,the paper introduces the link bandwidth cost model and calculates the unit bandwidth usage cost on the migration path according to the current link bandwidth usage rate.Then the paper models the VNF migration path decision process as a Markov decision process proposes a deep Q-network-based SFC migration path decision(DQN-MPD)strategy and designs an SFC migration timing and migration path decision(MTPD)algorithm that combines DQN-MTD agent and DQN-MPD agent.The MTPD algorithm uses the DQN-MTD agent to perceive and predict network resource status changes,combines the SFC migration information,intelligently decides the migration timing of the VNFs,and then uses the DQN-MPD agent to perceive the status of the underlying link resources,and combines the VNF migration bandwidth and other information to decide the migration path for the VNFs.The experimental results show that,compared with the DQN-MTD algorithm,the MTPD algorithm can increase the migration success rate by at most about 14.25%without increasing the service downtime;compared with other algorithms,the MTPD algorithm can reduce service downtime by at most about 12%,and increase the migration success rate of SFC by at most about 15.6%.