| Recently,the construction of an integrated Space-Air-Ground-Aqua communication network(SAGAIN)for achieving global ubiquitous coverage has become a research hotspot in both academic and industry communities.Compared to airborne,space-based,and land-based communication networks,research on underwater communication networks is still in its early stages.In the underwater environment,underwater wireless sensor networks(UWSNs)are one of the primary technologies to provide communication services.Compared to traditional wired underwater communication networks,UWSNs offer advantages such as flexible deployment,lower costs,and the ability to achieve broader coverage.However,due to the complex and changing underwater communication environment,as well as limitations in available resources,the performance of UWSNs falls far short of expectations and has become a bottleneck for achieving high-performance SAGAIN.Therefore,in the face of the complex communication environment,it is an urgent problem that needs to be addressed on how to optimize the performance of UWSNs with limited available resources and provide high-speed,stable,and reliable communication services for various underwater applications.With the rapid development of artificial intelligence and the emergence of new technologies such as deep reinforcement learning,these technologies are being increasingly applied in research related to wireless networks.Existing studies have shown that the combination of network management mechanisms and the aforementioned emerging technologies,particularly with deep reinforcement learning,can provide novel solutions for sequential decision-making problems in complex environments.However,applying deep reinforcement learning technology to solve resource management and performance optimization problems in UWSNs faces numerous challenges.These challenges include dynamically changing acoustic channel states,cooperative performance optimization under partial environmental observations,huge solution space for performance optimization algorithms,and trade-offs between available network resources and network performance.Therefore,this thesis aims to address the challenges posed by the complex underwater communication environment and constrained network resources.It explores the application of deep reinforcement learning techniques in the field of underwater wireless sensor network resource management,specifically focusing on the management of communication resources,energy resources,and storage resources.The specific research contents are as follows:(1)Deep multi-agent reinforcement learning-based distributed and adaptive resource management algorithm: For the resource management problem in complex communication environments and distributed UWSNs,this thesis proposes a distributed and adaptive resource management algorithm based on deep multi-agent reinforcement learning.This algorithm aims to optimize network capacity by managing communication and energy resources.It leverages the environmental perception capabilities of underwater sensor nodes and the spatial separability inherent in sparsely deployed underwater wireless sensor networks.By adaptively adjusting the node transmission parameters based on the residual energy and channel conditions,the proposed algorithm increases the number of concurrent communications in the network to enhance network capacity while satisfying the network lifetime requirements and optimizing energy efficiency.Experimental results demonstrate that the proposed algorithm effectively guides underwater nodes to adaptively adjust their transmission parameters in response to changes in communication environments and application requirements,thereby ensuring the quality of service in underwater wireless sensor networks.(2)Joint resource management algorithm for link scheduling and power allocation: For the resource allocation problem in underwater wireless sensor networks with constrained energy supplies and imperfect nodes susceptible to potential malfunctions,this thesis proposes a resource management algorithm that jointly schedules the communication links and allocate appropriate transmit power for the scheduled nodes.This algorithm utilizes cooperative deep multi-agent reinforcement learning to manage network communication resources and node energy resources.Its objective is to provide fair,efficient,and reliable network services for various underwater applications in imperfect and energy-constrained communication scenarios.Specifically,this algorithm takes into account the impact of potential node failures on the effectiveness of cooperative optimization mechanisms.During the training phase of the intelligent decision-making model,the curriculum learning mechanisms and transfer learning techniques are included to enhance the generalization capability of the decision model in complex communication environments and reduce the training time required.Experimental results demonstrate that this algorithm effectively improves network concurrency,communication fairness,and delivery ratio.Additionally,the effectiveness of the key components in the proposed algorithm is also verified.(3)Joint optimization algorithm for managing communication resource,energy resource,and storage resource: For the resource management problem in underwater wireless sensor networks with dynamically changing traffic loads and partially observabilities,this thesis presents a novel algorithm that jointly manages the communication resource,energy resource,and storage resource.By leveraging deep multi-agent reinforcement learning,this algorithm aims to optimize network throughput,minimize delivery delay,and reduce energy consumption,ultimately enabling efficient and reliable communication services across a wide range of underwater applications.To account for the long propagation delays inherent in underwater acoustic communication,this thesis introduces the concept of information confidence,which quantifies the validity of overhear information.Then,a traffic-aware mechanism that utilizes the information confidence is devised to effectively tackle the resource management challenges arising from dynamic traffic loads and partial node observability.Additionally,this thesis also proposes a solution space optimization algorithm to enhance the search efficiency of decision models and alleviate the computational complexity associated with the vast decision space in underwater network resource management.Experimental results demonstrate that the proposed algorithm exhibits remarkable adaptability in diverse communication scenarios,thus facilitating efficient and reliable communication within resource-constrained underwater networks. |
