Research On Adaptive Underwater Acoustic Communication Technology Based On Reinforcement Learning

Traditional adaptive underwater acoustic communication systems are generally effective in environments where channel conditions change relatively slowly,but the effects are not ideal for fast-changing channels or large reception delays.The adaptive problem can be expressed as a Markov decision process,so reinforcement learning methods can be introduced to solve the above problems.In order to achieve low bit error rate and robust mobile underwater acoustic(UWA)communication,this kind of adaptive UWA communication technology based on reinforcement learning that can change the sending signal mode in real time according to the feedback channel state information is particularly necessary.First,this paper introduces the overall flow of adaptive UWA communication,and analyzes the influence of time-varying UWA channels on single-carrier communication.The traditional adaptive UWA communication technology based on threshold division and two adaptive strategies are introduced.The adaptive decision feedback equalizer with embedded phase-locked loop is studied,and the output signal-to-noise ratio after equalization is used as the basis for the feedback state information in the adaptive UWA communication method studied in this paper.Secondly,this paper studies the adaptive UWA communication technology based on reinforcement learning.Firstly,it introduces the theoretical basis of reinforcement learning and points out that the adaptive modulation problem can be abstracted as the problem of selecting actions based on value.Therefore,the Q-learning algorithm is selected to establish a single-carrier adaptive UWA communication algorithm model based on reinforcement learning,which is based on effectiveness and reliability.A single-carrier adaptive UWA communication algorithm based on reinforcement learning of these two strategies.Finally,this paper conducts modeling and simulation experiments based on the measured sound velocity profile of the Arctic,and uses the yellow sea water acoustic communication test data to verify the algorithm studied in this paper.The results show that,based on the reliability criterion strategy,the Q-learning method can select a modulation mode with a lower bit error rate for data transmission in the current environment.Based on the effectiveness criterion strategy,the algorithm can reach a higher system throughput than the traditional algorithm.In the Yellow Sea test,the average throughput of the system can be increased by 7% compared with the traditional algorithm.The adaptive communication algorithm studied in this paper can adjust the strategy according to the demand in the same environment to achieve better transmission quality than the traditional method.The adaptive single-carrier UWA communication technology based on reinforcement learning studied in this paper is expected to provide necessary technical support for communication between underwater mobile nodes and target ships or underwater nodes.