The rapid growth of mobile communication business and the increase of the number of wireless termi-nals promote the research of fifth generation mobile communication system.In order to meet the increasing wireless service demands,traditional cellular structure composed of macro base stations(BSs)is evolving into an irregular network topology equipped with low transmission power nodes,which is referred as an ultra dense network(UDN),or a heterogeneous network(HetNet).In the thesis,we consider the improvements of downlink capacity and energy efficiency(EE)from the multicell cooperative transmission perspective.By further employing multiple antennas at the transmitters and receivers,we use the linear precoding and receive filtering techniques to eliminate the inter-cell interference(ICI)and improve the capacity.In small networks,the amount of required channel state information(CST)is moderate,which makes it practical to jointly opti-mize precoding vectors and receive filtering vectors in the whole network.However,in large networks,by contrast to the time needed to get the required CSI,the coherent time is small,hence base station clustering must be applied.In HetNets,traditional user association methods increase the network loss and decrease EE,therefore,it is significant to reconsider the user association in HetNets.For cooperative transmission in dense cellular mobile communication system,we resort to the distributed algorithms due to their low implementa-tion complexity and robustness.In total,we mainly consider three problems in dense networks:base station clustering,coordinated precoding and user association.Firstly,we introduce the characteristics of wireless channel and statistical models,as small-scale fad-ing and large-scale fading,which are the preliminaries for channel modeling.Furthermore,we describe the MIMO channel model and the corresponding capacity formula.Then,we also introduce the definition and feasible conditions of interference alignment(IA).In addition,several typical methods for precoding and power distribution are introduced as the tools for the following algorithm.Subsequently,the dynamic base station clustering in dense networks is studied.Base station clustering is the approach to group the BSs into disjoint clusters to reduce the cooperation overhead in the entire network.There are two popular approaches:centralized algorithm and distributed algorithm.We firstly describe two centralized algorithms via maximizing intracluster interference and maximizing sum rate in this part,which are suitable for medium networks.In addition,a distributed algorithm according to the idea of the game theory is further introduced.A player selects coalition in terms of its utilities,where a BS is regarded as a player,and a cluster is regarded as a coalition.And the distributed algorithm is suitable for large networks.Then,coordinated precoding in dense networks is studied.After base station clustering,CSI is only needed to share in intracluster,which leads to cooperation between BSs in intracluster and limited cooperation between BSs in intercluster.We consider the weighted average sum rate maximization which is averaged in unknown intercluster CSI,the primal problem is then equivalently transformed into a weighted minimum mean square error problem which is easier to deal with.In addition,the block coordinate descent method is used for solving the transformed problem and we obtain an iterative distributed algorithm.Finally,we study the user association in HetNets.The traditional user assoication methods are not suit-able for HetNets,which will lead to the macro BSs’ overload and decrease the network EE performance.We establish a joint optimization problem on user association and resource allocation to maximize the sum rate,then the problem is gradually transformed to a solvable formation.Next,we introduce the lagrange multi-plier to decouple the primal problem into two subproblems.Then the gradient projection method is used for solving the transformed problem and we obtain an iterative distributed algorithm,which allows distributing computing tasks to BSs and users. |