Research On MAC Protocol Based On Reinforcement Learning And Cross-layer Design

The ocean area accounts for 71%of the earth's surface area and is one of the important areas of our human activities.With the continuous development of science and technology,modern communication systems and networks have extended from the air and land to the ocean.Therefore,underwater acoustic communication technology has developed rapidly in recent years.The research of acoustic communication also plays a vital role in the exploration and development of marine resources,the monitoring of the marine environment and the development of marine military strategies,and the reliable transmission of data in the hydroacoustic sensor network is particularly critical.The underwater acoustic channel is different from the wireless channel in that it has the characteristics of narrow available frequency band,high propagation delay,serious multipath effect,and large Doppler frequency shift.Therefore,the data transmission protocol of terrestrial wireless networks is not suitable for underwater communication.We need to design a new MAC protocol to reliably transmit underwater communication data,while increasing the throughput of underwater sensor networks,reducing end-to-end delay and reducing packet loss rate.The main work contents and innovations of this paper are as follows:1.Aiming at the serious collision problem caused by underwater communication packet transmission,this paper proposes an intra-cluster data transmission protocol suitable for UASNs:MAC(RLM)protocol based on reinforcement learning(Reinforcement Learning),the RLM protocol uses an improved time-slot frame structure and learns underwater acoustic sensor networks through reinforcement learning strategies.Each sensor node performs data transmission according to the local Q value table,and then updates the Q value table according to the data transmission status.After many learnings,the network reaches the convergence state,and each sensor node will select the relative best data transmission time slot,thereby greatly reducing the packet loss due to the collision between the data packets and improving the performance of the network.At the same time,a node state judgment mechanism is added to the RLM protocol,so that the RLM protocol has better adaptability to changes in the network topology.Finally,through simulation,the performance of the algorithm is analyzed.2.Aiming at the problem of packet loss in underwater communication due to the harsh environment of underwater acoustic channels,this paper introduces a cross-layer design,which combines the physical layer and the MAC layer to build a PHY/MAC cross-layer model.Through the sharing of the relevant parameters of each layer,the system as a whole performs unnified scheduling of relevant information parameters,so as to achieve the purpose of optimizing the performance of the network system.The physical layer uses Fractional Fourier Transform(FrFT)to synchronously detect the received signal to determine the state of the data frame and the reason for packet loss.According to the channel state information,the adaptive modulation or the signal transmission rate is adjusted for the data frame,and the MAC layer adjusts the transmission time segment or adjusts the data frame length according to the state of the data packet to ensure more reliable data transmission.3.Implement the PHY/MAC joint model on the OPNET platform,and analyze the network performance of the cross-layer solution.Simulation results show that the PHY/MAC cross-layer design can greatly improve network throughput and reduce packet loss rate.