Medium Access Control And Resource Allocation Based On Machine Learning In Cognitive Wireless Networks

The ever-increasing wireless communication applications demand is highly required to the cognitive radio network performance.The combination of the data generated by the individual networking behaviors of cognitive user and artificial intelligence techniques such as deep learning has been certificated as the main development direction that can enhance wireless communications performance.This thesis concentrates on Medium Access Control(MAC)protocol and resource allocation strategy to improve the network throughput performance.The adoption of machine learning helps cognitive users autonomously address the following kernel issues: How to choose a suitable MAC protocol under different network scenarios? How to attain the optimal wireless channel resources in the CSMA/CA-based network? How to select the optimal time slot in the TDMA-based network? The main contributions of the thesis are summarized as follows.As the single-type MAC protocol is application-limited and cannot adapt to the dynamic network scenarios in centralized cognitive wireless networks,this thesis presents a MAC protocol selection model based on classification learning.By employing supervised learning scheme,this model includes learning phase and decision phase.The two phases are iteratively executed to provide the suitable MAC protocol to adapt to the corresponding network scenarios.First,the widely-used CSMA/CA and TDMA protocols serve as the candidate to respectively represent the competitive protocol and the non-competitive protocol,and the related network scenarios for the selected protocols are analyzed.Then,by using fuzzy set theory to formulate the selection criteria for MAC protocols,the MAC protocol pertaining degree is proposed for cognitive users to decide whether it is required to switch the current MAC protocols.In learning phase,the network environment parameters and cognitive user parameters are collected to build the parameter feature data set,and the execution results of the current MAC protocol are applied to build the statistic feature data set.Next,the two feature data sets are combined using multiple-granularity knowledge increment algorithm to generate the data set for classification learning,after which the test set is trained to compare the matching level of different classification learning algorithms.In decision phase,the contribution rates of different sample features are first analyzed for a given model classification accuracy,where the features with high contribution rate are selected as the indicators for evaluating the model classification performance.Next,classification results are obtained by training the target decision samples,which can help to choose the most suitable MAC protocol for the current network scenario.Simulation results validate the proposed MAC protocol selection model can effectively classify different network features of the samples.They also indicate that the features of different sample items have different contribution rates for the model accuracy.Considering the impact of multi-user contention access in MAC layer,,this thesis propose an online decision channel selection algorithm based on non-cooperative game to address channel selection problem in CSMA/CA-based cognitive wireless networks.By employing semi-supervised learning scheme,this algorithm aims at reducing control and negotiation under the premise of message interaction that cognitive users can timely and independently select the optimal channel to access.First,we model the channel selection problem as the throughput maximization problem under non-cooperative game scheme of cognitive user.Second,the influence of channel heterogeneity and multiple cognitive user competition access(i.e.CSMA/CA protocol)on cognitive users is analyzed,and the closed-form expressions of the achievable transmission rate and throughput are derived.Then,we obtain the utility function of cognitive users with Nash Equilibrium.Based on the global optimal theory,the expression of the optimal channel selection problem for the sum utility function is obtained.Next,we design the bidirectional updating algorithm to make the cognitive users can iteration update its channel selection strategy on any direction,in order to achieve the maximization of utility.Finally,we analyze the optimal response closed expression for the continuous game phase,it is proved that the algorithm can converge to the unique Nash Equilibrium solution.Simulation results validate the outperformance of the proposed online decision channel selection algorithm compared against conventional schemes.Moreover,by extending the experiment,the proposed algorithm can be applied to multi-channel and multi-interface network.To circumvent the intensive computations of the online-decision channel selection algorithm,we propose an offline learning channel selection algorithm based on reinforcement learning for CSMA/CA-based cognitive wireless networks.By employing semi-supervised learning scheme,this algorithm aims at enabling cognitive users to select the optimal channel in a "trial and error" way.First,the channel selection problem of cognitive users is modeled as maximizing the "action-value" utility function related to channel selection behavior under reinforcement learning.Second,we analyzed the channel selection behavior of greedy strategy,and derived the closed-form expression of the Bellman Equation optimal expectation,which acts as the evaluation criterion for each round of learning iteration.Then,we modeled the channel selection process of "exploration-exploitation" of cognitive users as two-dimensional markov chain,and calculated the action-value utility function associated with the channel selection behavior in each iteration.Finally,we analyzed the computational complexity of this algorithm and the storage requirement of cognitive users,and proved that by iteratively updating the action-value utility function,it can converge to the channel selection result of approximate Bellman's optimal expectation solution.Simulation results validate the algorithm based on offline learning can significantly improve throughput performance.Compared with the online-decision algorithm,moreover,it can achieve similar throughput improvement with lower complexity and power consumption.To efficiently address the dynamic slot selection problem in TDMA-based cognitive wireless networks,a topology-transparent scheduling algorithm based on reinforcement learning is presented in the thesis.By employing unsupervised learning scheme,the learning process is divided into collision avoidance learning process and redundant slot utilization learning process.The proposed scheme does not involve control nodes,and cognitive users only need to learn about their own slot selection actions,thus it is suitable for distributed deployment.In the topology-transparent slot selection process,a cognitive user should tackle two primary problems: how to reduce the time slot conflicts and how to improve the utilization ratio of slots.First,we model the topology-transparent slot selection problem from the perspective of maximizing the minimum normalized average cognitive user throughput,where the maximum number of possible collisions between any two nodes and the time slots in included in a sub-frame serve as the optimization targets.Second,we derived the closed-form expression of the normalized average cognitive user throughput.In collision avoidance learning process,the problem of collision avoidance between cognitive users and interference users is first modeled as a temporal difference learning model.Then to minimize the quantization error between the time slot selection and feedback expectation,we design a learning algorithm using forward linear time sequence difference to update the slot selection vector until it converges to the approximate optimal feedback expectation result.In redundant slot utilization learning process,the action-value function and its corresponding slot state are considered as a fixed state value function pair.We model the redundant slot utilization problem as a subframe-by-subframe executed prioritized replay process,and collect the fixed function pairs in successive time instances to build the experience replay set.Then,by randomly sampling in the experience replay set to obtain the state value function pair,cognitive users optimize the mean square error between the current time slot selection and the sampled result,and the stochastic gradient descent method is adopted to update the redundant time slot utilization vector until it converges to the least squares result.Simulation results validate the proposed topology-transparent scheduling algorithm based on reinforcement learning can significantly improve throughput performance.In addition,the result of parameter optimization can further help to improve throughput.