Research On Adaptive Modulation Algorithm Of Underwater Acoustic Communication Based On Reinforcement Learning

The characteristics of underwater acoustic channel,such as large background noise,obvious multipath effect,long transmission delay,seriously affect the quality of underwater acoustic communication.Adaptive modulation is an important technology to improve the quality of underwater acoustic communication.In the traditional adaptive modulation system,due to the time-delay characteristic of underwater acoustic channel,the channel state information of the receiver fed back to the transmitter through the feedback channel is out of date,which makes the adaptive modulation of the transmitter inaccurate and affects the performance of the system.Aiming at this problem,this thesis studies the optimization of traditional adaptive modulation system with reinforcement learning technology in time-delayed time-varying underwater acoustic channel to improve the communication transmission quality.In this thesis,two adaptive modulation algorithms based on reinforcement learning are proposed to improve the accuracy of adaptive modulation mode selection and the transmission performance of the communication system.Firstly,the transmission characteristics of underwater acoustic channel are studied in detail,various factors affecting the transmission quality of underwater acoustic communication are analyzed,the underwater acoustic channel model is established.and the channel is simulated.Secondly,based on the traditional underwater acoustic adaptive modulation system,an adaptive modulation system for underwater acoustic communication based on reinforcement learning is constructed,and several typical algorithms in reinforcement learning are compared.Finally,Q-learning algorithm and SARSA algorithm are used to improve the accuracy of adaptive modulation.The adaptive communication parameters are mapped to the reinforcement learning algorithm,and the change of underwater acoustic channel is learned through the reinforcement learning algorithm,and the behavior strategy is selected.According to the change of channel,the intelligent decision is made,and the optimal modulation mode suitable for the current channel is selected to improve the transmission error code and communication throughput of the system.The simulation system was built,and the system performance under Q-learning algorithm and SARSA algorithm was compared and analyzed with that under fixed modulation mode and direct feedback.The results show that the bit error rate and throughput of the adaptive modulation system after reinforcement learning are better than the fixed modulation method and the direct feedback method,and the Q-learning algorithm is better than the SARSA algorithm.It can be seen that the reinforcement learning algorithm is effective and feasible to improve the transmission error and throughput in the adaptive modulation of time-varying underwater acoustic channel,and can effectively improve the communication performance of the system.