Research On Dynamic Spectrum Access Of Multi-Channel Cognitive Vehicular Networks Based On Multi-User Reinforcement Learning

With the rapid development of communication technology and the popularity of the Internet of Things and intelligent vehicles,the transportation industry is gradually moving towards intelligence,and Vehicle-to-Everything(V2X)technology has become an important intersection of the two major fields of the Internet of Things and intelligent vehicles.However,the increasing diversification of communication services and the continuous growth of communication devices in the Internet of Vehicles have led to explosive growth in communication demand,making the full utilization of spectrum resources an important challenge for communication of Internet of Vehicles.In order to improve spectrum utilization,cognitive radio can intelligently use idle frequency bands,which happens to meet the communication needs of the Internet of Vehicles.Therefore,the concept of Cognitive Vehicular Networks(CVNs)was proposed,which can support Internet of Vehicles using Cognitive Radio(CR)technology.In the CVNs,vehicle types are mainly divided into two categories: primary vehicles and cognitive vehicles.Primary vehicles use authorized frequency bands for data transmission,while cognitive vehicles sense channel status and try to use frequency bands occupied by unauthorized vehicles.Dynamic spectrum access is one of the effective methods to improve spectrum utilization in the CVNs.Based on the limitations of the existing research on the CVNs’ dynamic spectrum access,this thesis proposes a dynamic spectrum access algorithm based on reinforcement learning.The main work and contributions of the thesis are as follows.1.Previous research only considered a single communication link in the CVNs,either Vehicle-to-Vehicle(V2V)communication links between vehicles or Vehicle-toInfrastructure(V2I)communication links between vehicles and roadside facilities,while ignoring the impact of cognitive vehicle’s spectrum sensing errors.Therefore,this thesis establishes a CVNs communication model that simultaneously includes the V2 V and the V2 I links with certain interference and spectrum sensing errors.In the established communication scenario,there are multiple cognitive vehicles and primary vehicles,and cognitive vehicles opportunistically access channels to communication.2.Based on the different access situations of cognitive vehicles,this thesis proposes a dynamic spectrum access method based on multi-reward reinforcement learning.In the real-world cognitive vehicular network communication environment,the communication collision situations that reduce the success rate of cognitive vehicle access to channels can be mainly divided into two types: collisions between cognitive vehicles and collisions between cognitive vehicles and primary vehicles.The proposed method redesigns different reward function based on the channel capacity under different collision situations,thereby improving the performance and convergence of the proposed method.In the proposed method,cognitive vehicles,as intelligent agents,continuously learn in the CVNs environment,and find the optimal strategy to improve the success rate of cognitive vehicle access to channels.In this thesis,we explore the impact of different traffic densities and spectrum sensing errors on the proposed method.Simulation results show that the performance and convergence of the proposed method are significantly better than other comparative methods,such as the Myopic Strategy,Deep Q-Network(DQN),and Dueling Deep Q-Network(DDQN),not only improving spectrum utilization but also improving energy efficiency.3.In the spectrum access problem of the CVNs,the speed of the vehicles has a certain impact on their communication quality.However,the communication range of the vehicles is generally determined,so the fundamental reason for the impact on spectrum access is the time.This thesis proposes a novel time slot structure for cognitive vehicles,and combines it with the redesigned reward function under different access conditions.Furthermore,as the proposed communication model includes two different communication links,a cumulative reward function design is proposed that combines the number of links in order to balance the impact of the accumulated reward of each link on the overall reward.The simulation results further demonstrate the good generalization ability of the proposed method for different communication environments.