Research On Routing Algorithms Based On Reinforcement Learning In Delay Tolerant Networks

Delay Tolerant Networks(DTNs)is a new type of wireless sensor networks with sparse nodes and no continuous end-to-end connections.The intermittentness and uncertainty of communication paths make message routing and forwarding a huge challenge.Therefore,designing an efficient routing algorithm has become an important research for delay tolerant networks.Researchers have proposed many routing algorithms based on node contact probability,node buffer space,and node movement patterns.However,the starting points and application scenarios of these routing algorithms are different.In this thesis,we focus on the characteristics of the delay-tolerant network structure that is dynamically changing,and conduct research on the delay-tolerant network routing algorithm based on reinforcement learning according to the "storage-carry-forward" routing mechanism,.Firstly,the related theoretical knowledge and technologies of delay-tolerant networks have been described in this thesis.On this basis,the current research status of routing algorithms for delay tolerant networks has been analyzed with emphasis,and reinforcement learning algorithms and simulation tools for delay tolerant networks have been described in detail.All above these provide a theoretical and technical basis for the design and simulation of the tolerant network routing algorithm.According to the dynamic topology characteristics of the delay-tolerant network,a delay tolerant networks model based on reinforcement learning is established to solve the Q-Learning algorithm's "overestimation problem" for the next action reward,and a double-Q-Learning algorithm-based delay tolerant networks routing algorithm(Double Q-Learning Routing,DQLR)is proposed.The intermediate value mechanism is designed to calculate the cumulative rewards for the messages passed through the intermediate nodes,which provides a certain reward basis for potential next hop nodes.At the same time,an effective dynamic reward mechanism is proposed to distinguish the reward levels of different forwarding paths based on the number of hops that the message is forwarded to the destination node.Simulation results show that DQLR protocol can improve the delivery ratio and reduce transmission overhead.Furthermore,considering the complexity,dynamics and uncertainty of the nodes and network attributes in the delay tolerant networks,the fuzzy logic theory is introduced and a Fuzzy-Logic and Double Q-Learning Routing(FDQLR)algorithm is proposed.In this algorithm,the node activity,contact interval,and movement speed are evaluated comprehensively by fuzzy logic algorithm,and the obtained fuzzy value is used as the immediate reward fuzzy coefficient of the next hop node.The particularity of destination node movement is taken account in this algorithm,and other nodes that have met the destination node within a certain time period are affixed with the label of a "hot zone" node,which making the message forwarding more purposeful and effectively improving the message delivery ratio.In addition,in order to better manage the buffer space of the nodes,a drop mechanism is proposed so that the nodes have sufficient buffer space to receive new message copies.Simulation experiments show that FDQLR effectively improves the performance of delay-tolerant network message transmission compared with DQLR algorithm.The research in this thesis can not only apply widely to the real world such as vehicle-tolerant networks and social-tolerant networks,but also provide ideas for the research of routing algorithms in DTNs,and has a good theory value and application prospect.