Research On Routing Method Based On Rinforcement Learning For Satellite Internet Of Things

Satellite Internet of Things(S-Io T),which integrates satellite networks with Io T,is a ubiquitous Io T system oriented toward the integrated satellite-terrestrial information network architecture.S-Io T can realize the data intercommunication of global land,sea and air.It has the advantages of wide coverage,multiple-type services and high robustness.In addition,as the key of the communication infrastructure in the future,S-Io T is widely used in many fields such as weather forecasting and communication broadcasting.As the fundamental of communication protocol for S-Io T,the routing method is responsible for data packet forwarding,which determines the performance of transmission.Therefore,the research of routing method is of great significance.As a result of the dynamic changes of topology structure in S-Io T,the effective forwarding of data packets is challenging.Reinforcement learning provides a way to solve this problem.In this context,this thesis studies the routing method based on reinforcement learning for S-Io T.The whole S-Io T is regarded as a reinforcement learning environment,and satellite nodes and ground nodes in S-Io T are both regarded as intelligent agents in this method.The main works of this thesis are as follows:(1)In view of the dynamic changes of node status in S-Io T,a routing method based on improved Q-learning for S-Io T is proposed.First,each node maintains a Q value table,and the next hop node of data packets is determined according to the Q value.Second,in order to optimize the Q value,this thesis not only improves the reward value based on the hop count but also improves the discount factor based on the node status.Finally,simulation analysis shows that the proposed routing method can adapt to the highly dynamic changes of node status in complex space environment.Compared with traditional routing methods,it has better results in terms of delivery rate,average delay and overhead ratio.(2)In view of the dynamic changes of the network status in S-Io T and on the basis of studying the dynamic changes of node status,a routing method based on improved Double Q-learning for SIo T is proposed.First,each node maintains two Q tables,which are used for selecting the forwarding node and for evaluating the forwarding value,respectively.In addition,the next hop node of data packets is determined according to the mixed Q value.Second,this thesis not only improves the mixed Q value based on the network congestion degree but also improves the balance factor based on the comprehensive network evaluation.Finally,the simulation results show that the proposed routing method can achieve efficient routing in complex space environments.Compared with the routing method based on improved Q-learning for S-Io T,it has greater improvement in delivery rate and average delay.