Call Admission Control In Heterogeneous Network Based On Improved Q Learning

Next-generation mobile communication systems are expected to be highly integrated heterogeneous networking environments. Wireless networks can be complementary by providing additional resources to one another if properly integrated. The Long Term Evolution(LTE)system has been specified by the 3GPP on the way towards 4G mobile to ensure 3GPP keeping the dominance of the cellular communication technologies. LTE networks offer increased coverage, but the cost for connecting is very high. In contrast, Wireless Local Area Networks(WLANs) infrastructure offer a limited coverage range while allowing high rates of data transmission at lower cost. The integration of these networks has been the subject of several studies in recent years. Especially the video delivery services has become the fashion services among users nowadays. As an important component of radio resource management(RRM), call admission control(CAC) is to decide whether a new call or a handover request can be accepted into a resource-constrained network. Existing heterogeneous networks(HetNets) have big differences among user quality of experience(QoE), network feature, coverage, and so on. To maximize revenue, operators have to improve QoE to attract users by increasing Quality of Service(QoS). Simply improve some QoS objective indicator cannot embody the subjective service experience of user perfectly. Q learning algorithm neither needs to mathematical modeling of the environment, nor needs specialist training guidance, hence it has very strong unknown environment adaptability. However, common Q learning algorithm exists some weakness such as the convergence is not stable, and the solution space is easy to fall into local optimal solution, which is difficult to converge to global optimal solution.To solve the above problems, proceeding from improving the user QoE and considering influence of multiple QoS factors to video delivery services synthetically, we put forward a CAC algorithm in LTE/WLAN network based on Simulated Annealing Q-learning(SA_QL). The studies are as follows: Firstly, we introduce the theory and model of CAC in HetNets, and discuss the Q learning theory. Then we map the SA_QL to the CAC algorithm properly, and the CAC could implement the autonomous Q learning. Secondly, to balance exploration with exploitation in the learning process, we improve the Q learning by applying Metropolis criterion to CAC Problem for more stable convergence, load balancing and system capacity. Once again, QoE, which is introduced in return function, can provide appropriate different services impacting on user perception, also reduce the handoff frequency, so as to improve the system resource utilization. Finally, establish a call admission control model based on Q learning, by choosing proper system parameters, evaluate the network performance in different users’ arrival rates compared with load balancing algorithm and WLAN priority algorithm.Through the analysis of the data and simulation results, the CAC algorithm based on simulated annealing Q learning in the heterogeneous networks such as LTE and WLAN, can achieve lower new session blocking rate,lower session interrupt rate, lower handoff rate but higher system resource utilization. The network attributes namely the available bit rate(ABR), delay, receive signal strength(RSS) and network connection cost that determines the Qo E of the user has been taken to consideration.