Research On Smart And Collaborative Routing Mechanism Based On Reinforcement Learning

With the rapid development of Internet,network service is becoming increasingly diversified,but data packet routing of those services is having difficulties in maintaining its efficiency in dynamic network environment.Smart and Collaborative Network,as a future network architecture,decouples the control plane and the data plane and brings about flexible control of data packet routing through central management and component scheduling.However,it still remains to be improved:(1)Faced with routing requirement of specific service,the mechanism is able to intelligently select the forwarding route with the best comprehensive capacity;(2)The mechanism should perform real time perception of dynamic change of network environment and adjust the route selection.Under the aforementioned background,based on the research of routing mechanism in nowadays Internet and future network,with the help of Smart and Collaborative Network architecture and reinforcement learning techniques,smart and collaborative routing mechanism adapted to specific service and dynamic network environment is designed and realized in the dissertation as follows:First,the function and performance requirements of routing mechanism is analyzed;architecture and module design of the whole mechanism are also presented,including the deployment,function,principle and interaction of each module.(1)Faced with routing requirement of specific service,routing algorithms based on offline Q Learning algorithm,online Sarsa algorithm and Sarsa(X)algorithm based on eligibility trace are all designed in reinforcement learning training module to intelligently select the forwarding route with the best comprehensive capacity.(2)Faced with dynamic change of network environment,the real time perception of topology and link state is realized through triggered update and regular update mechanism in network perception module,according to which the mechanism updates the environment reward and then performs another training process and adjust the best forwarding route.(3)Besides,link Qos(Quality of Service)capacity is defined in routing requirement perception module;link grading and environment reward setting are conducted in link Qos capacity grading module;forwarding table is generated to instruct the node in forwarding table generation moduleThen the modules in routing mechanism are realized in a certain process based on specific development environment and framework.Key data structures,functions,class and class methods are as well introduced in detail.Additionally,the process of modeling routing problem into a reinforcement learning task and the principle of environment reward setting are discussed,according to which a routing control method based on manual environment reward setting is proposed.At last,the function and performance of routing mechanism are tested by building a simulation platform and the results are as follows:(1)Routing mechanism based on reinforcement learning successfully select the forwarding route with the best comprehensive capacity for specific service data packet;(2)The deployment of optimized reinforcement learning algorithm brings about improvement in mechanism performance;(3)In dynamically changing network,the mechanism is able to adjust the forwarding route and has advantage in transmission performance over traditional routing mechanism based on shortest path first.