Research On Routing Algorithm Based On Deep Reinforcement Learning In Wireless Mesh Networks

Wireless mesh network plays an important role in many fields,such as emergency communication,private networks in various industries and national defense.Routing technology is one of the core technologies.Routing algorithm based on deep reinforcement learning(DRL)has become a hot issue in current research because of its fast adaptability and excellent forwarding performance in complex network environment.Compared with the traditional routing algorithm based on mathematical model,the routing algorithm based on DRL has the ability to quickly adapt to the network environment and better routing forwarding performance in the highly dynamic and complex network environment.However,the application of routing algorithms based on DRL technology proposed by researchers still faces many problems and challenges in the actual network environment.For example,in wireless mesh networks,the obvious difference of link bandwidth caused by heterogeneous network integration has a great impact on the transmission of different large and small packets,but the existing DRL based routing algorithms are still lack of relevant response design,Secondly,the current routing algorithms based on multi-intelligent DRL technology generally have the problem of too slow convergence speed,poor performance and unstable performance in high load network environment,and so on.Therefore,in view of the above problems,it is of great significance to study the routing algorithm based on multi-agent DRL technology in wireless mesh networks.The main innovative work and research contents of this thesis are as follows.(1)In the heterogeneous wireless mesh network environment,the existing routing algorithms ignore the obvious difference of link bandwidth in the network environment,resulting in the decline of various performance indicators when packets of different sizes are transmitted in the network.This thesis proposes a routing strategy that comprehensively considers many routing factors,such as link bandwidth,packet size,node load,routing path and historical decision-making.In this innovation,firstly,this thesis designs a unique multi input neural network model,and puts forward the coding scheme of the joint action of single heat coding and label coding,so that the routing algorithm designed in this thesis can make effective decisions based on multi-dimensional parameter information,and improve the learning effect of neural network on multi-dimensional parameter features.Secondly,this thesis also designs a reward function considering the node queuing delay and real-time link transmission delay of data packets at the same time,which improves the practical performance of the algorithm.By adding the reward normalization function to the reward function,the obstacles to model training and convergence caused by the continuous large reward value of cumulative time delay are removed.Finally,aiming at the problem that the convergence speed of the algorithm caused by the neural network parameters in the random initialization algorithm is too slow in the initial route establishment stage,this thesis proposes to add a pre training algorithm with the sum of the link transmission delay of the packet and the node queuing delay as the route measurement at the time of the establishment of the initial route strategy,which effectively reduces the convergence time of the algorithm.The simulation results show that in the wireless mesh network environment with both wide and narrow band links,compared with the comparison algorithm,the routing algorithm proposed in this thesis better adapts to the characteristics of the network environment with both wide and narrow band links by considering a variety of measurement parameters such as link bandwidth and packet size,and reduces the average transmission delay and node congestion probability of the algorithm in data transmission,At the same time,it also improves the throughput of the algorithm in the network environment.(2)In the wireless mesh network environment with both wide and narrow band links,aiming at the problems of performance degradation,slow convergence speed and unstable performance of multi-agent routing algorithm based on DRL when the network load is high,this thesis proposes a multi-agent DRL routing algorithm based on Forwarding task division in distributed environment.This thesis analyzes that the reason for the above problems is that with the increase of network load,the network environment becomes more complex and the network state changes more violently.However,the task complexity faced by a single neural network does not decrease.The functions of the above factors make the convergence of multi-agent DRL algorithm more difficult,and inevitably affect the performance after the convergence of the algorithm.In order to solve the above problems,firstly,a simple but very effective task classification strategy is designed.Secondly,the training mode and decision mode of neural network model in the original algorithm are improved.Through the above design the complexity of the corresponding task of each neural network is reduced,so that the neural network model corresponding to each task category is more focused on dealing with a certain category of tasks,and there is no strict requirement that a single neural network model on each node can deal with a variety of different categories of tasks.The convergence time of the neural network on each node in the original algorithm is greatly reduced,and the performance of the converged routing strategy in multiple performance indicators is improved.The simulation results show that when the network load is high,the design of this thesis improves the convergence speed of the algorithm and the stability of the algorithm after convergence.At the same time,it also improves the performance of the algorithm after convergence in the indexes such as average transmission delay,throughput,packet loss rate,node congestion probability and so on.Finally,the thesis summarizes the research work of the full text,and looks forward to the next research work.