Routing Method For Satellite Network Based On Deep Reinforcement Learning

Satellite communication has been widely used in the field of mobile communication due to its advantages of wide coverage,strong anti-destructive ability,and no geographical constraints.Routing,as the core protocol of satellite network communication,bears the heavy responsibility of data transmission and determines the transmission performance of the satellite network.Therefore,the routing method of satellite networks has important research significance.Due to the characteristics of high-speed movement of satellite nodes,frequent link switching between nodes,uneven network traffic,and complex environments,traditional ground network routing techniques cannot be fully applicable in the current satellite network.Therefore,solving the routing technology problem in satellite networks is a key issue in current satellite network research.Deep reinforcement learning has strong representation and solving abilities for complex problems and can provide new ideas for problem-solving.This paper studies a satellite routing method based on deep reinforcement learning,which takes the entire satellite network as the environment of deep reinforcement learning and the satellite nodes as intelligent agents.This paper conducts research on satellite routing algorithms from the following two aspects.1)To address the problem of fast-changing link states between satellite nodes due to their high-speed movement,a satellite routing algorithm based on DQN improvement is proposed.The algorithm adopts the idea of virtual nodes,with the minimum number of hops as the principle,and sets the number of hops and distance as parameters related to the reward function.At the same time,a priority experience replay mechanism is set up,which enables the algorithm to learn from the highest value samples during the training process.Then,a link state evaluation function is proposed to set three states of idle,busy,and congested based on the evaluation mechanism,and the link state is saved in real-time to the satellite nodes.An effective satellite routing algorithm is implemented that can adapt to the highly dynamic changing link states between satellite nodes.2)Due to the complex environment in which satellites operate,satellite nodes are prone to failure.Based on the study of satellite node states,in order to improve the anti-destruction ability of the satellite network,this paper proposes a satellite routing algorithm based on the improvement of Dueling DQN.The algorithm adopts the idea of virtual nodes in Chapter 3 and establishes a GEO/LEO dual-layer satellite network.For three types of routing forwarding failures: broken links,loops,and link congestion,a routing failure handling mechanism is proposed.The experience replay mechanism of the Dueling DQN model is improved,and a new sampling method is proposed by combining the advantages of random experience sampling and priority experience sampling.At the same time,with the minimum link transmission delay as the principle,the link transmission delay and the distance between the current node and the destination node are used as parameters related to the reward function.A satellite routing algorithm that improves the anti-destruction ability of the satellite network is implemented.