Placement Optimization And User Tracking With Aerial Base Stations Using Deep Reinforcement Learning

The frequent occurrence of sudden public events such as earthquakes,floods and other natural disasters will cause damage to the communication facilities within the disaster area.The communication signals in the disaster area will be blocked,which leads to difficulty of the rescue work,resulting in major casualties and property damage.In emergency communication systems,unmanned aerial vehicles(UAVs)can be used as aerial base stations(ABSs)for keeping wireless connectivity in various emergency scenarios:communication facilities in disaster areas are easily damaged,and ABSs can be applied to assist or replace the cellular network to provide wireless coverage for ground users(GUs);in addition,in major sports events,ground base stations(GBSs)are easily overloaded and cannot meet the communication needs of GUs.ABSs can be utilized to provide more GUs with high-quality communication services.To maximize the coverage rate of N GUs by jointly placing multiple ABSs with limited coverage range is known to be an NP-hard problem with exponential complexity in N.The problem is further complicated when the coverage range becomes irregular due to site-specific blockage(e.g.,buildings)on the air-ground channel in the 3dimensional(3D)space.To tackle this challenging problem,this thesis applies the Deep Reinforcement Learning(DRL)method by 1)representing the state by a coverage bitmap to capture the spatial correlation of GUs/ABSs,whose dimension and associated neural network complexity is invariant with arbitrarily large N;and 2)designing the action and reward for the DRL agent to effectively learn from the dynamic interactions with the complicated propagation environment represented by a 3D Terrain Map.Specifically,a novel two-level design approach is proposed,consisting of a preliminary design based on the dominant line-of-sight(LoS)channel model,and an advanced design to further refine the ABS positions based on site-specific LoS/non-LoS channel states.The DQN and Prioritized Replay DDQN algorithm are applied to train the policy of multi-ABS placement decision.Numerical results show that the proposed approach significantly improves the coverage rate in complex environment,compared with the benchmark Kmeans algorithm.Based on the research on the placement optimization of ABSs,this thesis further studies the problem of adaptive tracking and covering mobile users by ABSs,and uses an improved PRDDQN method to enable ABSs to track and re-cover mobile users within a limited time.