Research On Dynamic Cooperation Cluster Selection Algorithm For Cell-free Massive MIMO Systems

Cell-free massive multiple-input multiple-output(MIMO)is a promising 6th generation(6G)mobile networks technology that shortens the distance between the users and the access points(APs)using the user-centric approach.In this very system,the selection technology of dynamic cooperation cluster(DCC)can significantly improve the energy efficiency and reduce the complexity of signal processing,and is consequently recognized as a critical technology for cell-free massive MIMO systems.However,when serving certain users,the interference between cooperation clusters caused by the overlapping of multiple cooperation clusters can lead to significant performance degradation.Thus,the DCC selection of cell-free massive MIMO systems is hereby studied in this thesis.The specific research is as follows:1)The whale swarm reinforcement learning(WSRL)algorithm is proposed to solve the problem of DCC selection under the time division duplex operation.To initiate the research,the population of whales is first initialized,with the whale position in the population representing the cooperation cluster selection scheme;afterwards,the Q-learning algorithm is adopted to determine the best strategy and implementation for updating the whale position;then,each whale position in the population is evaluated,so as to select out the whale closest to the prey and gather other whales toward it;finally,the whales prey successfully by constantly updating their positions,which indicate that they get the best DCC selection scheme.When combine with Q-learning for strategy updating,the proposed WSRL algorithm not only enhances the robustness of the whale optimization algorithm(WOA)algorithm,but also obtains a better performance.Simulation results show that the proposed WSRL based DCC selection algorithm possessed better system spectral efficiency than the existing algorithms;2)A value decomposition algorithm based on multi-agent deep reinforcement learning(MDRL)is proposed to deal with the DCC selection under the frequency division duplex operation.The global Q-value function is firstly decomposed according to the number of agents in the system;afterwards,the state,action,reward and the next state of the agents are collected as the training data of the network and stored in the buffer zone;then,the Q-network is intensively trained with the global data to calculate the total loss function,and the gradient descent is used to transfer the weight coefficient to the agent-wise value function of each agent for updating;finally,corresponding action of the maximum agent-wise value function of each agent is output,and the best DCC selection scheme is thus obtained.The algorithm approximates the Q-value function through the deep neural network,and effectively solves the problem of highdimensional state-action space caused by a large number of APs and users in the cellfree system.Furthermore,the multi-agent system is introduced to further settle the complexity of multiple users in cell-free massive MIMO systems selecting the cooperation cluster at the same time,and the value decomposition network is applied to improve the environmental non-stationary problem brought by multi agents and endow the algorithm with better scalability and reliability.Simulation results confirm that compared with the traditional swarm intelligence algorithm and the reinforcement learning algorithm,the spectrum efficiency of the proposed algorithm is significantly improved.