| The Ultra-Dense Networking technology greatly improves the transmission rate and capacity of the next generation mobile communication system through the densely deployed wireless access points,and greatly reduces the delay.Therefore,it effectively improves the overall performance of the network.With the increasing density of access points in UltraDense Networks,on the one hand,numerous scalable access points share limited available resources,which brings dynamic and efficient wireless resource management requirements;On the other hand,due to the coexistence of high-density access points,the distance between users and access points is greatly reduced,resulting in serious co-channel interference.When facing the complex and dynamic network structure,the intelligent algorithm of wireless network performance optimization based on learning has great advantages over the traditional algorithm.Therefore,this thesis will research on the wireless network optimization algorithm based on Federated learning to optimize the performance of wireless networks from two aspects of resource management and interference.The specific contents can be summarized as follows:(1)Radio Resource Scheduling Algorithm Based On Federated Reinforcement LearningThe existing radio resource allocation algorithms can not meet the dynamic and efficient resource management requirements in Ultra-Dense Networks,where densely deployed access points share limited wireless resources.Therefore,this thesis proposes a radio resource scheduling algorithm based on Federated Reinforcement Learning with the quality of service of short video services as the index.Firstly,the resource scheduling problem is transformed into a Markov Decision problem.Secondly,the qualities of videos,the delay and the wireless resources occupied by the video are used to do the modeling of deep reinforcement learning.After that,the model is deployed in multiple access points for learning.Then,the model aggregation method in Federated learning is introduced to further improve the quality of service of the short video services and learning efficiency for models.Meanwhile,This thesis creatively uses the unique reward characteristics of reinforcement learning to propose a reward based weight coefficient allocation method.Next,in the algorithm design,the optimized Federation aggregation algorithm is used to aggregate the deep reinforcement learning model.Finally,the simulation scenario is designed in detail,and the experimental results show that compared with the baseline algorithm,the proposed algorithm inherits the advantages of the integration of deep reinforcement learning and Federated learning,has achieved a significant improvement in the quality of service and learning efficiency,that is,compared with policy gradient-based quality selection and radio control algorithm,the learning efficiency is doubled,and compared with the q-Fair Federated Learning algorithm,the maximum average quality of service has been improved by more than 30%.In addition,when dealing with access of numerous users in local hotspot areas,the proposed algorithm has an improvement of more than 40%.Therefore,the proposed algorithm can better adapt to the dynamic network structure in Ultra Dense Networks.(2)Interference Coordination Algorithm Based on Federated Reinforcement LearningIn order to further improve the quality of service of short video services in Ultra-Dense Networks,an interference coordination algorithm based on Federated reinforcement learning is designed in this thesis to deal with the serious co-channel interference between densely deployed access points.Firstly,the sub-carriers and the power are taken as the main states to model the interference problem by deep reinforcement learning.Then,we innovatively improve the convolution layers of the deep neural network in the deep reinforcement learning algorithm,that is,we deal with the complex state changes in the interference coordination problem by convoluting the interference related states,such as power,sub-carrier allocation,and whether user is covered or not.Next,the interference coordination algorithm is designed by using Federated learning.Finally,the simulation experiments shows that compared with the latest research,such as the reinforcement learning-based interference coordination algorithm,the proposed algorithm considers the allocation of sub-carriers in the design of the interference coordination model of local access points and takes advantage of Federated learning at the same time,so the learning efficiency is doubled,and it takes advantage of reinforcement learning in model selection,and improves the maximum average signal to interference plus noise ratio by 20%.Therefore,the proposed algorithm can not only better eliminate interference to improve the signal to interference plus noise ratio,but also greatly improve the learning efficiency,thus improve the quality of service of short video services to meet the high traffic requirements of local hot spots in the future Ultra Dense Networks. |
