| With the development of marine resource exploration and national defense security,the expanded observing for marine information imposes demanding requirement for high efficiency of information transmission in underwater acoustic sensor networks.Admittedly,cooperative communication technology is widely employed by various underwater communication nodes to transmit information cooperatively,which can effectively combat the bandwidth limit,propagation delay and frequency-selective fading.Therefore,underwater acoustic cooperative communication strategy optimization has been proposed as a promising solution to improve the underwater acoustic communication performance and enhance the underwater acoustic information transmission efficiency.Unfortunately,due to the harsh underwater environment,several challenges are faced in optimal relay selection and adaptive communication strategy,including time-varying channel state information,serious environment noise interference and dynamical network topology.Thereby,this paper puts forward various low cost,high efficiency and strong applicable relay selection algorithms under different communication scenarios and communication constraints,as well as designs adaptive communication scheme by multi-parameter adjustment,aiming to achieve high-level information transmission efficiency in underwater acoustic communication networks.Firstly,this thesis studies the relay selection problem for static source node in presence of unknown,time-varying underwater channel state information,and proposes an expert-based reinforcement adversarial MAB(Multi-Armed Bandit)learning algorithm.Under adversarial MAB learning formulation,EXP3 is applied to guide adaptive relay decision by analyzing the observed historical reward sequences,thereby successfully avoids the influence of imperfect channel state information on relay selection.Furthermore,in consideration of the timeliness demand for relay selection strategy response,an expert learning mechanism is designed to heuristically learn relay selection probability distribution for enriching the learning information,which essentially improves the learning ability of EXP3 algorithm and achieves high system capacity.Secondly,jointly considering the underwater noise interference and dynamic channel state information,this thesis further presents a new adaptive extended Kalman filter estimator and formulates a hierarchical adversarial MAB learning algorithm.Given the observed reward is inevitably subjected to underwater noise interference,the expectation maximum algorithm is first employed to estimate noise covariance in the reward estimation layer.Then,extended Kalman filter algorithm is proposed to suppress noise interference and provide accurate relay reward estimation.In the relay decision layer,the user evaluates relay quality using the filtered reward sequences,and makes relay decision based on the improved EXP3 algorithm.This learning algorithm has certain tolerance to noise interference and incomplete information acquisition.Thirdly,this thesis investigates the relay selection problem for mobile source node,and proposes a collaboration-aware contextual MAB learning algorithm.Due to the mobility of source,the candidate relay set shows time-space-varying characteristic.Thereby,we first quantize the environmental conditions of relay nodes as contextual information and employ Ridge Regression method to estimate relay reward,to capture the influence of the environment factors on relay behaviors.Further,to combat the space uncertainty of relay set,environmental content vector is allowed to cluster relays into groups.Within each group,the experienced relays share information with new relays to improve the learning efficiency for new appearing relays.The proposed algorithm shows strong self-adaptation and self-organization ability in uncertain network topology condition,and establish high system capacity under time-space-varying candidate relay set.Forth,this thesis deals with multiple relay selection problem for multiple source nodes,and develops a contextual combinatorial MAB algorithm with fairness constraint.Specifically,a combinatorial MAB learning model,wherein the reward of each user-relay pair is observed independently,is first designed for multiple source nodes to reduce the strategy space complexity and computational cost incurred by multi-source decision.Subsequently,considering the dynamic underwater conditions and competing interference,Kernel Regression is employed to learn the nonlinear contextual information related reward function.In addition,we employ the virtual queue technique to properly handle the fairness constraints to provide sufficient learning time and training data for each strategy regressive learning model,aiming to improve the multi-source optimal relay decision accuracy.The proposed algorithm shows high algorithmic scalability so as to allow an effective online multiple relay selection optimization.Finally,this thesis studies the adaptive OFDM communication problem in time-varying underwater communication environment,and proposes a multi-parameter joint adjustment scheme to ensure reliable information transmission.Firstly,we use the adversarial MAB theory to model the multi-parameter configuration problem.Given the multi-parameter configuration space grows to be large with a variety of parameters,an orthogonal learning strategy is tailored to reinforce the learning of parameter decision space to generate low-complexity and small-scale decision space.The aforementioned scheme realizes low-complexity and high-efficiency multi-parameter joint optimization for underwater OFDM system,and ensure the reliable information transmission in time-varying underwater acoustic communication environment. |
