Routing Strategy For LEO Satellite Network Based On Deep Reinforcement Learning

The Low Earth Orbit(LEO)constellation network has the characteristics of low communication delay,global coverage,large communication capacity,and low transmission cost.It is an effective way to achieve seamless global real-time information transmission.Satellite network routing technology is the key to ensuring the smooth operation of satellite network services.How to achieve stable and highly efficient routing in a complex and changeable low-orbit constellation network environment is an urgent problem that needs to be solved.This paper aims at improving the routing performance of low-orbit constellation networks,and proposes a low-orbit satellite routing strategy based on deep reinforcement learning.Firstly,the strategy establishes a low-orbit constellation model and constellation network topology based on the principles of LEO satellites.Based on this,a link state awareness strategy is designed for the satellite link state,and the two-hops link state is maintained in real time through link state prediction and link state update.Then,combining deep reinforcement learning with the LEO satellite routing algorithm,the article proposes a low-orbit satellite routing algorithm based on the Double Deep Q-Learning Network(DDQN).The satellite nodes are used as agents according to the Markov process.To establish DDQN model.It defines the DDQN model as the core of routing calculations,the two-hop link state as the input of the model,and the optimal next-hop node as the output of the model.By using the value function Q network and target Q network,it makes the satellite node observe the satellite network topology and learn routing.Through the constraint of the value function,it enables the DDQN model to learn a routing decision strategy with the shortest path characteristics and load balancing characteristics in a low-orbit constellation environment.In addition,for satellite routing failures,including node failure,endless-loop route,and link congestion,corresponding strategies are set up in the strategy to prevent and deal with routing failures timely,further improving the applicability and resistance of low-orbit satellite routing strategies.Finally,the NS-3 platform is used to simulate the satellite network traffic.The performance of the low-orbit satellite routing strategy based on deep reinforcement learning is analyzed and verified under different network conditions.According to the simulation results,the routing strategy is able to be applied to the low-orbit constellation network environment.It performs well in three aspects:average end-to-end delay,packet loss rate,and system throughput.