Research On Intelligent Learning In Cognitive Radio

Along with the informational progress in current world and society, the wirelesscommunication technology has experienced rapid development. Various wirelessservices and applications increase at an amazing speed, and novel techniques in wirelesscommunication and networking emerge continuously. Simultaneously, the characteristicof wireless networks in high data rate and wide band leads to the increasing requirementfor spectrum resource, so that the contradiction between the supply and the demand ofspectrum resource appears to be more and more extrusive.Cognitive radio (CR) proposes a reform strategy with a dynamic spectrum utilizinginstead of the fixed one, which can effectively raise the frequency utilizing efficiency.Moreover, the high intelligence emphasized by CR accords with the developmentdirection of wireless communication systems and networks.This dissertation focuses on the learning part in CR, which should be the best oneto exhibit its intelligence. The author researches on off-line learning and on-linelearning, and the major contributions are as follows:1. A universal learning and decision making framework for CR is proposed. Theconcrete applications of two learning methods based on the framework, i.e. aneural network (NN) method and the least square support vector machine(LSSVM) method, are investigated. In the aspect of NN, a “direct” radial basisfunction neural network (RBF-NN) based learning and decision making method isproposed. Compared with the traditional “indirect” methods, the “direct” one hasan additional process for optimal cases searching before training stage, which canreduce the number of input neurons and output neurons to decrease the trainingcomplexity, and can improve its real-time performance by means of direct decisionon parameter configuration. In the aspect of LSSVM, several multiclassclassification approaches are discussed and compared in term of complexity andperformance under the constructed CR scenario. The non-dominated sortinggenetic algorithm is adopted to implement the hyper-parameters searching ofLLSVM, so as to enhance its universality. Simulation results show that both theRBF-NN method and the LSSVM method can improve the performance of CRsystems, and that the non-dominated sorting genetic algorithm can search outsuitable hyper-parameters within a few evolutional generations, and that theLSSVM method behaves better decision performance and generalization property. 2. For solving the problem of channel allocation and power control in spectrumunderlay cognitive radios, a multi-agent enforcement learning method based onuser clustering and a variable learning rate is proposed, which can effectivelyimprove convergence of multiuser learning. Firstly, a hierarchy processing methodis used to separate channel selection and power control, and channel allocation isimplemented by fast optimal search combined with user-number balance based onperformance prediction. Secondly, stochastic game framework is adopted to modelthe multiuser power control issue. In subsequent multi-agent enforcement learning,K-means user clustering method is employed to reduce the user number in gameand single user’s environment complexity, and a variable learning rate scheme forQ learning and policy learning is proposed to promote the convergence ofmultiuser learning. Simulation results show that the method can make themultiuser’s power status and global reward converging effectively, and moreoverthe whole performance can reach sub-optimal.3. A Nash bargaining solution based method is proposed for multi-channel multi-userchannel selection and power allocation under the condition of limited total powerin CR. A reasonable Nash bargaining utility function is designed, which makesNash product be able to explicitly express the performance of CR systems;furthermore, the existence and uniqueness of Nash bargaining solution is proved.An iteration bargaining procedure based on the idea of gradient descent andprediction of performance variation is proposed to implement allocation ofchannel and power. The theoretical analysis and experimental simulation show thatthe power allocation method based on Nash bargaining solution conforms toproportional fairness, and the iteration algorithms for channel and powerallocation can achieve sub-optimal performance of the whole systems.4. A cooperative decoupling method and a cross-layer joint method are proposed formulti-layer resource allocation in multi-hop cognitive radio networks. In thecooperative decoupling method, the task of path choosing is accomplishedindependently, and then is the game allocation of channel and power. In thecross-layer joint method, the three-layer resource of path, channel and power issimultaneously allocated by a game process. The both methods syntheticallyemploy the heuristic principles of network layer, MAC layer and physical layer;they assist the path choosing by using the information of interference receivingdegree and interference sending degree. The Boltzmann exploration based onwidth of permitting power is designed to execute the selections of channel and power. A method of replacement and elimination of long link or bottleneck link isused to further enhance the network performance. For improving convergence, asequential game process instead of simultaneous game process is chosen and itsconcrete implement process is provided, since the former has better behavior incurrent scenario. Besides, the Nash equilibrium of the games and the complexityof two related algorithms are analyzed and discussed. Simulation results show thatthe cooperative decoupling method and the cross-layer joint method have betterperformance in the number of success flows, the achievable data transmission rateand power consumption than the cooperative link game and the local flow gamewith simple decoupling.5. A multiuser independent Q-learning method without information interaction isproposed for multiuser dynamic spectrum accessing in cognitive radios. Themethod adopts self-learning paradigm, where each CR user performsreinforcement learning only through observing individual performance reward, sothat it can save the communication resource in exchanging information with others,and where the reward is suitably defined according to the present channel qualityand channel conflict status. A learning strategy to implement multiuser dynamicspectrum accessing is designed, which performs sufficient exploration based on acriterion of trending to the better channel while punishing the conflict channel. Fortwo users two channels scenario, a fast learning algorithm is proposed and it isproved that the algorithm can converge to the maximal total reward. Thesimulation results show that the CR system based on the proposed method inmulti-user and multi-channel selection can converge to Nash equilibrium withlarge probability and acquire high performance in the whole reward.