Multipath Detection Reinforcement Learning Routing Protocol For Underwater Acoustic Sensor Networks

Due to the inherent characteristics of high latency,low bandwidth and high transmission loss of Underwater Acoustic Sensor Networks(UASNs),designing a long-lived routing protocol for efficient transmission is a major challenge for UASNs.The routing protocol based on reinforcement learning can achieve the approximate global optimality when selecting each hop forwarding node,but there is a problem of slow convergence.To solve the problem,this paper combines partial cooperation reinforcement learning and proposes the UASNs multipath detection reinforcement learning routing protocol.When the algorithm does not converge,the source node sends the data packets and the probe packets to explore the network state space to accelerate the convergence speed of the algorithm,and stops sending probe packets after the algorithm converges.The energy consumption of the probe packet which only carrying the packet header information is far less than that saved by the acceleration of algorithm convergence,which reduces the total energy consumption of the network and prolongs the life cycle of the network.In this paper,the theoretical analysis and simulation verification of the proposed protocol are carried out.The theoretical analysis proves that the proposed protocol has convergence and can accelerate the convergence speed of the algorithm.The simulation results show that the proposed method can effectively improve the convergence speed and have performance advantages in extending network lifetime,reducing total network energy consumption and reducing end-to-end delay.