Research On Unlicensed Spectrum Wireless Access Based On Reinforcement Learning

With the rapid development of technologies such as the mobile Internet and the Internet of Things,the global mobile data traffic experiences an exponential growth.The scarce licensed spectrum resources can no longer meet the current demand for data service traffic.In order to alleviate the load pressure on licensed spectrum network,3GPP proposed Licensed Assisted Access(LAA)technology to offload data traffic from licensed spectrum to unlicensed spectrum in the 4G LTE phase,and conducts research on NR-Unlicensed(NR-U)technology to explore the application of unlicensed spectrum in 5G network.Since the unlicensed spectrum is open,5G NR-U network coexists with LAA,Wi-Fi and other networks,and the network environment is complex and dynamic,which poses great challenges for the design and optimization of access strategy.In view of the dynamic characteristics of wireless resources and the complexity of wireless network environment,this thesis investigates the wireless access technologies for 5G NR unlicensed spectrum based on Q-learning and other reinforcement learning theories.The main research work and innovations of the dissertation are summarized as follows:1.In view of the dynamic changes of NR-U network environment,this thesis designs a dynamic channel selection mechanism and proposes an online learning distributed access strategy to improve network capacity on the basis of Listen-Before-Talk(LBT)time retreat mechanism.First,the users' channel access selections are modeled as a non-cooperative game model based on the long-term capacity expectations,and it is proved that there is a pure Nash equilibrium point in the game.Inspired by the multi-armed bandit(MAB)theory,an online learning distributed channel selection access(OLDCSA)algorithm is proposed to optimize the users'access selections,and the upper bound of performance loss is derived by using the probability theory.Finally,the simulation results show that the proposed access strategy outperforms the existing random selection by 18%on average and is close to the exhaustive search.2.It is attractive to deploy the Internet of Things independently with unlicensed spectrum in terms of network deployment costs and flexibility.In view of the congestion and collision problems in the process of unlicensed spectrum transmission,and the characteristics of random arrival of uneven services in machine communication,this thesis establishs the optimization problem of minimizing long-term packet loss of the network.Based on the time slot competitive transmission model,the problem is modeled as a Markov decision process.A dynamic spectrum access strategy based on deep reinforcement learning is proposed to reduce the network packet loss and the duty-cycle is limted to ensure that the networks use the unlicensed spectrum fairly.Simulation results show that the proposed access strategy achieves about 20%lower packet loss rate than the greedy strategy.At the same time,the simulation results also show that this strategy can reduce transmission collision and coexist efficiently with other heterogeneous networks without a-prior knowledge about the surrounding environment.