Reinforcement-Learning Based IEEE 802.11ax Dynamic Sensitivity Control Strategy Research

Wireless Local Area Networks(WLAN)provide a low-cost,high-performance network access method that is widely used around the world.In order to meet the request for growing data traffic,the deployment of WLAN networks is intensive.However,the legacy IEEE 802.11 stantard supporting WLAN adopts a fixed Carrier Sense Threshold(CST),which cannot efficiently utilize the spectrum resources,and the inter-cell interference problem in dense deployment seriously affects network performance.In order to solve these problems,IEEE 802.11ax,the next generation IEEE standard,proposes a Dynamic Sensitivity Control(DSC)method to improve spatial reuse efficiency to establish an efficient WLAN.In this paper,we mainly focus on an intelligent DSC strategy of nodes in the next generation WLAN,which makes WLANs efficient,and also on the coexistence of the legacy WLAN nodes and IEEE 802.11ax nodes.This paper first summarizes the issues related to the IEEE 802.11 access mechanism and the key technologies of the IEEE 802.11 ax standard.It also introduces the related work on CST adaption and the theory of the reinforcement learning and simulation platforms.Then,this paper designs an intelligent DSC strategy based on Q-learning.Firstly,the DSC control problem is modeled as a discrete Markov decision process,and we defined the discrete state and action space.Secondly,we design an intelligent DSC strategy based on Q-learning to enable the WLAN nodes to automatically adapt CST.Then,the channel access process in networks is modeled by Stochastic Geometry considering the interferences between nodes,and we establish a throughput-based utility function.Finally each node maximizes the utility function value during training to determine an optimal strategy(i.e.CST).The simulation shows that the proposed algorithm can effectively adapt the CST and maximizes the throughput,effectively improving the spatial multiplexing efficiency.Then,this paper studies the fair coexistence problem of IEEE 802.11ax nodes and the legacy IEEE 802.11 nodes(ie,IEEE 802.11ac/n,etc.),and proposes a control strategy based on deep reinforcement learning to achieve a highly efficient as well as more fair WLAN by jointly adjusting CST and transmit power.Firstly,the throughput-based proportional fair utility function is designed as the reward function.Secondly,a continuous state and action set are defined and we design a joint control strategy of CST and transmit power able to learn from the interaction with the environment.The simulation results show that the proposed algorithm converges well.And compared with the traditional algorithm,the proposed algorithm can effectively improve the fairness of the network of coexisting nodes and achieve much more throughput than legacy networks.