Research On Spectrum Resource Allocation Strategy Based On Multi-Agent Reinforcement Learning

With the rapid development of wireless communication technology,the scale of wireless networks has increased rapidly,and a large number of terminal devices have an increasing demand for spectrum resources,which poses a huge challenge to the management of limited spectrum resources.Cognitive radio technology endows the network with the ability to learn independently,opens up more available spectrum,and realizes more efficient allocation of spectrum resources.However,with the evolution of wireless network architecture and the emergence of wireless network applications with high computing requirements,new types of networks pose new challenges to dynamic spectrum allocation technology: 1)Layered heterogeneous network architecture deployed in mixed macro base stations and micro base stations among them,the dense deployment of micro base stations leads to a more complex wireless interference environment,especially when users have different task calculations,how to allocate limited spectrum resources and combine task offloading and computing resource allocation to meet the service quality of differentiated users It is an important problem that needs to be solved urgently under this framework.2)When computing tasks are gradually transformed from single independent task computing to multi-task computing with dependencies on user relationships and execution timing,the user’s resource allocation will be constrained by timing dependencies.The existing offloading schemes do not consider The timing relationship of resource allocation makes it difficult to achieve efficient use of spectrum resources.This thesis studies the problem of multiuser computing offloading and resource allocation with various computing requirements under the heterogeneous network.At the same time,it studies the problem of multi-cell multi-user computing offloading and spectrum resource allocation with user task dependence.The specific research content is as follows:Aiming at the problem of computing offloading and spectrum management in heterogeneous networks,this thesis takes user task processing delay and system energy consumption as optimization goals,and divides channels into primary user and secondary user channel sets according to the difference in task calculation amount.The channel division strategy is further combined with computing offloading,frequency band selection,power control,and computing resource allocation strategies,and a joint optimization model for multi-user computing offloading resource allocation considering user task differences is proposed.In order to solve this optimization problem,this thesis proposes a user spectrum management scheme based on the multi-agent deep deterministic policy gradient algorithm,and finds the optimal resource allocation strategy through cooperative learning among agents.The simulation results show that the algorithm can effectively reduce the user’s task delay and system energy consumption in the proposed heterogeneous network scenario.Aiming at the problem of multi-user computing offloading and spectrum allocation with task dependencies,this thesis regards the user task completion delay and system energy consumption as optimization objectives,models the task dependencies between users as a directed acyclic graph,and proposes a comprehensive frequency band and Multi-users in the time period depend on the spectrum resource allocation optimization model when the task is offloaded.The spectrum resources are divided by frequency and time slot.Users can access different frequency bands at different times.In order to obtain the optimal multi-user spectrum resource allocation strategy,a dependent task computing offload spectrum allocation algorithm based on multi-agent deep Q-network is proposed.The experimental results show that the proposed algorithm can effectively reduce the task completion delay and The overall energy consumption of the system.