Research On Beam Management Technology Based On Reinforcement Learning Of The Mmwave Communication In The Internet Of Vehicles

With the development of mobile communication,Internet of vehicles communication will become the key application scenario of 5G.The characteristics of 5G,such as large connection and bandwidth,play a supporting role in the construction of Internet of vehicles communication.Due to the depletion of sub-6G,millimeter wave communication has been applied in 5G because of its good directivity,high transmission quality,and small component size.However,the beam management of millimeter wave communication in the Internet of vehicles is still facing great challenges.The requirements for link stability and delay are greatly improved.The characteristics of strong time-varying channel also pose great challenges to the design of the actual system.The rapid development of machine learning can provide a fast and accurate beam management solution for millimeter wave communication in the Internet of vehicles.Therefore,the beam management of millimeter wave communication in the Internet of vehicles based on reinforcement learning,especially beam allocation and beam tracking,has become the focus of this paper.Firstly,this work establishes the beam pool model of millimeter wave communication in the Internet of vehicles,and links the two processes of beam allocation and beam tracking through the consumption and fall-back of the remaining beams.From the complexity analysis,compared with the millimeter wave communication system without beam pool model,the use of beam pool model can significantly reduce the complexity of millimeter wave communication network.Secondly,a Many-to-Many Beam Allocation Algorithm based on distributed multiagent reinforcement learning is proposed to solve the problem of many to many beam allocation between vehicles and roadside units.In the framework of the algorithm,vehicle users can independently select the roadside unit set to establish the connection according to the local observation state without exchanging information.Simulation results and complexity analysis show that the system transmission rate of the Many-to-Many Beam Allocation Algorithm is about 19.22%higher than that of the Many-to-Many RoundRobin based Algorithm,and the system performance of the optimal Many-to-Many Exhaustion based Algorithm is about 96.52%,while the system complexity is significantly reduced.In addition,the system stability of the proposed algorithm is 76.32%higher than that of the One-to-Many Beam Allocation Algorithm.Finally,a Multiagent-Cooperation Soft Switching of Beams Algorithm based on millimeter wave adaptive beam tracking is proposed.In this algorithm framework,only one roadside unit connected with the user is allowed to implement the millimeter wave adaptive beam tracking algorithm for the user,and share the beam configuration information with other roadside units,which is convenient for all roadside units to determine the user’s location information.The complexity analysis and simulation results show that the proposed algorithm can reduce the time complexity of beam tracking by about 75%,greatly reduce the computing resources required by the system to implement the beam tracking algorithm and reduce the system energy consumption.At the same time,the proposed algorithm can obtain good beam tracking training results and has good stability.The future research will focus on the green communication,with the goal of reducing the system energy consumption as much as possible,while considering the actual mobile model design and frequency band resource allocation.