Delay Control Mechanism For Delay Sensitive Services

Due to novel Internet time sensitive services,the requirements of network services are switched from bandwidth sensitive model to delay sensitive model,such that delay control will play a more important role in future communication systems.Different sources of network indeterminacy lies in three layers:optical transport layer,IP layer and TCP layer.In optical transport layer,overlarge scale of network and services lead to poor coordination of performance factors;in IP layer,the enlarging area,increasing number of devices,available protocols of access network make full network observations hard;in layers above IP,network queues relate strongly to dynamic service distributions and couples TCP protocols.Solutions for ubiquitous,intelligent and coordinated network control schemes are greatly needed.Designs of schemes for delay control are key research topics in communication networks.Current delay control schemes have main issues below:overwhelming large space of delay features and ignorance of service distributions,both performance factors and service distributions relate to service routes;partly oberservations of delay factors and service routes have different dynamicity.At the same time,high dynamicity of network status and complexities of computation also make it difficult to control the delay.Building ubiquitous,intelligent and coordinated delay control schemes are very important targets to solve these problems on different layers of service paths.Combining above delay indeterminacy,challenges and targets,this paper featurizes delay factors and applies probabilistic graph models and reinforcement learning to analyze and overcome the delay optimization problems,to build ubiquitous,intelligent and colaborate control scheme with "indeterminacy analysis and control policy learning".It specializes delay indeterminacy factors,designs policies to dynamically control delay and enhances the delay control for dynamic services.According to above main challendges,combining trend of“service marginalization and interconnective-intensiveness",the main research contents and innovations of this paper include:1.A proactive service distribution sensitive delay optimization algorithm is proposed.Aiming at overcoming the high complexities in coordinnating performances on selected paths with low delay,this paper analyzes the relations between services and delay&energy consumption.Basing on distributions of services and resources,the proposed algorithm obtains cost of delay&energy.It builds a policy function on delay efficiency,solves the delay optimization problem through reinforcement learning,chooses routes of lowest delay more frequently and lowers the propagation delay.Comparing with other algorithms,the proposed algorithm bases its cost function on features of expected usages on optical links and provides proactive delay control.In simulations,with specific configurations and service distributions of this paper,the proposed online algorithm can determine the cost respect to these distributions efficiently,choose low delay grooming solutions,minimize average path delay and maintain the same energy consumption.Comparing with grooming algorithms without delay optimization,it can improve the probability by about 14%of routing on the shortest path with least propagation delay.2.A distributive optimization algorithm over path delay is proposed.To combat multiple delay factors with incomplete observations on high dynamic routes,this paper introduces reinforcement learning as a framework to measure the indeterminacy of delay factors,to design dynamic reward function for minimization of path delay,to track environment state for delay optimization and to collaborate multiple factors.This paper provides an online distributive optimization algorithm to deal with indeterminacy.Comparing to statical routing with offline optimization,it provides a timely feedback for access network and ability to adapt to the rapid changing environment to lower route delay and to improve the bandwidths and efficiency of communications.In simulations where the terminal devices have high mobility,the performance of the proposed algorithm is compared against other algorithms(GPSR,QGeo)with different data frequencies by route delay,stability and efficiency,and evaluated respectively with end-to-end delay,packet deliver rate and retransmission overhead.Simulations show the proposed the algorithm can track environment state efficiently with overhead proportional to service load,adjust the policy in selecting routes of low delay and decrease route delay.Comparing to GPSR and QGeo with configurations in this paper,the route delay is minimized by about 5%,the success packet deliver rate is increased by 7%maximally and the communication overhead with less retransmissions is cut by 20%maximally.3.A service distribution sensitive optimization algorithm for controlling queueing delay is proposed.Analyzing the complex coupling effect between service distribution and network queues on fixed routes,this paper introduces reinforcement learning as a framework,learns working points and their transitions,and designs a novel active queue management.Comparing with service distribution insensitive algorithms,the proposed algorithm can provide the planning ability for delay control and maintain queue status with low delay and high throughput.In simulations with specific configurations of this paper,aiming at various service distributions,the proposed algorithm can provide adequate control on queueing delay with the ability of planning.Comparing with CoDel,simulation results show that the proposed algorithm can timely decrease queueing delay by about 12%at a sacrifice of throughput by about 1%,and maintain a long and stable status of high throughput.