Reinforcement Learning Based Communication And Computation Resource Allocation Theory And Approaches In Wireless Networks

At present,mobile terminals and network data traffic are growing explosively and all kinds of new services are emerging.In order to deal with the challenges brought by these trends,heterogeneous wireless networks have been widely concerned.With the development of edge computing,artificial intelligence and network function virtualization,future heterogeneous networks will present the characteristics of cloud-edge collaboration,endogenous intelligence and being capable of orchestration.In order to fully unleash the potential of heterogeneous networks,this paper makes an in-depth study on their resource allocation problems.The main works and innovations are as follows.Focusing on a cloud-edge heterogeneous wireless network,in order to overcome the challenge of resource optimization caused by the tight coupling of communication,computation and caching resources,a coalition game and multi-agent reinforcement learning based multi-dimensional resource allocation approach is proposed.Specifically,a computation offloading scenario with service caching is modeled first.Then,considering that the caching resource is adjusted on a larger timescale than communication and computation resource,a two-timescale resource allocation problem is formulated.On a large timescale,the weighted sum of the expectation of user latency and network caching cost is minimized by optimizing service caching,while communication and computation resources are optimized on a small timescale.To solve this problem,a coalition game based communication and computation resource allocation method under fixed service caching and a multi-agent reinforcement learning based caching resource optimization method are developed.The latter can overcome the challenges brought by the long-term objective without explicit form and discontinuous caching variables,and can well balance the latency and caching cost.In addition,the complexity,convergence and optimality of both proposals are analyzed and their superiorities together with the impacts of key parameters are demonstrated via simulation.Focusing on an open heterogeneous wireless network,considering the potential impact of the software-defined base station function split between the cloud and the edge on network resource allocation,this paper proposes a matching theory and multi-agent reinforcement learning based approach to joint base station function split and communication and computation resource optimization.First,given that base station function split and communication and computation resource allocation operate on different timescales,a two-timescale optimization problem is formulated.Specifically,communication and computation resources are optimized on a small timescale,aiming at minimizing the weighted sum of user latency and system energy consumption,while base station function split is optimized on a large timescale to minimize the expectation of weighted sum of user latency and system energy consumption.Then,a low-complexity matching theory based resource allocation method under fixed base station function split is proposed and multi-agent reinforcement learning is utilized to develop the long-term base station function split between the cloud and the edge scheme.By designing the reward function properly,this method can realize the adaptation to users'dynamic service requests and channel states.Finally,simulation results show the advantages of flexibility brought by open heterogeneous network architecture compared with pure centralized and pure distributed architecture.In conclusion,this paper studies resource allocation problems in different heterogeneous network scenarios and reinforcement learning and game theory combined resource allocation approaches are proposed,which lay a theoretical foundation for improving the performance of future heterogeneous networks with machine learning techniques.