Research On Dynamic Spectrum Access Problem Of Cognitive Vehicular Networks Based On DQN-MLP

As one of the research hotspots in intelligent transportation systems,the Vehicular Ad-hoc Networks can effectively improve commuting efficiency and ensure road traffic safety.However,the spectrum resources used for the Vehicular Ad-hoc Networks communication are far from being able to timely meet all communication demands.To address this issue,the concept of Cognitive Radio enabled Vehicular Ad-hoc Networks(CR-VANETs)has been proposed,which integrates cognitive radio technology into vehicular communication networks.In the CR-VANETs communication system,dynamic spectrum access techniques can effectively alleviate the scarcity of spectrum resources in vehicular networks.Currently,the research on dynamic spectrum access techniques faces the following challenges that urgently need to be addressed.Firstly,how to avoid communication collisions among vehicles when cognitive vehicles access idle frequency bands? Secondly,how to establish a stochastic model for determining the channel occupancy status of authorized vehicles? Then,how to reduce erroneous channel competition among cognitive vehicles in the presence of perception errors? Finally,how to improve the convergence speed and generalization capability of spectrum access algorithms?The problems existing in the current dynamic spectrum access technology in CRVANETs are targeted in this paper,and an approach based on deep reinforcement learning algorithm is proposed to enhance spectrum utilization.The detailed research content of this paper is as follows:Firstly,a CR-VANETs communication system with multiple secondary vehicles and multiple channels was established,and the sensing error of secondary vehicles is considered.In this communication system,primary vehicles are assumed to have priority to use authorized channels,and the dynamic occupancy state of the authorization channel is modeled as a dynamic Markov chain.Secondary vehicles dynamically access idle channels for communication based on their spectrum sense results.Then,based on the access selection of secondary vehicles,a mathematical model with throughput as the optimization objective is established.2.On the basis of this mathematical model,a deep reinforcement learning algorithm is used to maximize the system throughput.Specifically,the throughput optimization problem in the mathematical model is transformed into a reward function optimization problem in the deep reinforcement learning algorithm,and the mapping relationship between the system model and the algorithm is established.Then a novel DQN algorithm based on secondary vehicles multiple selection is proposed(IDQN algorithm).The IDQN algorithm enhances the access success rate of secondary vehicles while ensuring the quality of vehicle communication services.The performance of the IDQN algorithm is compared with that of Q-Learning,Myopic,and unimproved DQN algorithms,and it is found to be superior in terms of various performance metrics such as average channel utilization and average collision rate among secondary vehicles.And the impact of the number of secondary vehicles and sensing error probability is also examined,and simulation experiments demonstrate that the IDQN algorithm has a good ability to adapt to dynamic changes in the environment.3.The aforementioned IDQN algorithm fails to consider the impact of the sensing error probability of secondary vehicles in the design of the reward function.Additionally,it overlooks the influence of reward function normalization on the overall secondary vehicle channel access.Therefore,on the basis of the IDQN algorithm,the DQN-MLP dynamic spectrum access algorithm based on the sensing error probability was proposed(Improved IDQN algorithm),and the network structure of CR-VANETs is further optimized.In this algorithm,the reward function design is further optimized based on the secondary vehicle’s action selection,combined with the sensing error probability and reward factors.Additionally,the number of hidden layers in the network structure is increased to enhance the algorithm’s generalization ability.Simulation experiments show that the improved IDQN algorithm can significantly reduce the collision rate between secondary vehicles and between secondary vehicles and primary vehicles,thereby improving the access success rate of secondary vehicles.This algorithm shows about a 4% improvement in average channel utilization rate compared to the IDQN algorithm.And simulation experiments show that the improved IDQN algorithm enhances the vehicle’s ability to adapt to dynamic communication environments.