Deep Reinforcement Learning Based Intelligent Resource Allocation In Fog Wireless Networks

With the rapid growth of mobile data traffic and diversified service types,fog radio access networks(F-RANs)have received significant attention from the industry.By utilizing the computing and caching capabilities of fog access points(F-APs),user delay and energy consumption can be effectively reduced and transmission burden on fronthaul and backhaul links can be alleviated.To boost the performance of F-RANs,resource allocation plays a key role.However,traditional approaches often possess high complexity and cannot well adapt to complex and time-varying F-RAN environments.Compared with other artificial intelligence algorithms,deep reinforcement learning(DRL)combines the perception ability of deep learning(DL)with the decision?making ability of reinforcement learning(RL).It can directly learn control strategies from high-dimensional raw data and quickly respond to dynamic changes of wireless networks.Therefore,high-performance,low-complexity intelligent resource allocation approaches based on DRL are proposed,which optimize computation and cache resources in multi-user,multi-F-AP scenarios.The main contents and contributions are summarized below.1.In an uplink computation offloading scenario,a multi-agent DRL based F-AP selection approach is proposed.By moving online optimization complexity to the offline training stage,online decision?making time is greatly reduced and hence the self-adaptation to time-varying channel states and user task requests is realized.Specifically,the dynamics of wireless channel states and devices' computation task requests are first modeled by Markov process,based on which an optimization problem is formulated to minimize long-term system energy consumption.The problem is further decoupled into an offloading request forwarding sub-problem at each F-AP given user-F-AP association and an F-AP selection sub-problem under a given request forwarding policy.The former is addressed by a low-complexity greedy algorithm considering F-APs'limited computing capability,while the latter is solved by using multi?agent DRL,which effectively overcomes the dimensional explosion faced by single-agent DRL meanwhile achieves a compromise between performance and complexity.Moreover,the complexity of the proposed approach is analyzed and its effectiveness is verified through simulation by comparing with several baseline schemes.2.In a downlink content distribution scenario,a distributed DRL based multi-F-AP cooperative caching approach is proposed to fully utilize F-APs' limited cache space while tackle the challenge of no explicit relation between caching decisions and the optimization goal.A cooperative caching model is given first,in which each F-AP can fetch uncached contents from other adjacent F-APs.Then,a multi-F-AP cooperative caching optimization problem is formulated to minimize long-term content downloading latency.Since the problem depends on users'association strategy,the explicit expression of its objective is difficult to derive.To this end,a distributed DRL approach is applied to solve cache placement at each F-AP.The input state of each DRL model consists of the current cache state of its corresponding F-AP and the content requests of previous associated users,and each output action represents a content placement decision satisfying F-AP's cache space limitation.To achieve an implicit coordination among F-APs,the reward function is set as the sum of downloading latency of all users.Finally,this paper demonstrates the advantages of the proposed approach by comparing with various classic caching schemes.In conclusion,this paper studies intelligent resource allocation approaches for F-RANs,which is a useful supplement to F-RAN resource optimization theory and methods.Meanwhile it also provides a new idea for resource allocation research in other wireless networks.