Research On Key Technologies Of Path Planning For Ocean Sensor Networks Based On Deep Reinforcement Learning

Ocean Sensor Networks(OSNs)have been widely used in the fields of oceanography,ocean monitoring,offshore exploration,national defense and sea area security,etc.In recent years,the infrastructure of ocean sensor networks in various countries is developing at an unprecedented scale and speed,and they are obtaining a continuous stream of various ocean data from the ocean sensor networks.These ocean data are gradually making possible many scientific research works and engineering applications,and are becoming more and more important in scientific research and socio-economic development.Our government strongly supports the development of ocean monitoring sensor networks in the "13th Five-Year Plan"and the "14th Five-Year Plan" and has built a large number of integrated ocean monitoring networks.These systems can significantly improve the ability of marine public services,ocean environment perception,which provides excellent technical support and information platform for activities such as ocean environmental protection,sea area security,and ocean economic development.In these OSNs applications,the location information of sensor nodes is usually required as the basic information for the implementation of related technologies(e.g.,data analysis,routing protocols,load balancing,node tracking,etc.).Therefore,the localization of sensor nodes is a prerequisite for researching other key technologies.However,the deployment of a large number of anchor nodes in the currently commonly applied OSNs localization scheme requires high costs and is difficult to use in a large-scale application of actual OSNs.The recent mobile beacon-assisted sensor nodes localization scheme is promising in WSNs,which replaces the more costly anchor nodes deployment scheme by moving a GPS-enabled mobile beacon in the network and periodically broadcasting its location information to assist sensor nodes localization.However,due to the dynamic and complex characteristics of OSNs environment,there are many challenges in applying this scheme to OSNs sensor nodes localization:First,the application of the ranging model in WSNs(Wireless Sensor Networks)in a dynamic and timevarying sea surface environment has large errors,resulting in reduced localization accuracy.Second,the presence of time-varying winds,waves,and currents disturbances in OSNs environment makes it more difficult for the mobile beacon to remain stable and travel on the intended path at the target point.Therefore,it is also a challenging task to ensure the full coverage of sensing nodes’ localization and obstacle avoidance in an uncertain environment.Therefore,to address the above issues,this paper presents a systematic study on three aspects of OSNs ranging model,dynamic position control of the mobile beacon,and optimization of path planning strategies,which include:1)A deep neural network ranging model for ocean environments is proposed for complex and dynamic OSNs environments,which can automatically extract high-level semantic features of signal attenuation for different sea conditions,thus improving the signal ranging accuracy of OSNs in dynamic environments.2)For OSNs mobile beacons vulnerable to wind,waves,and currents,a deep reinforcement learning-based dynamic position control method for the mobile beacon is proposed,which eliminates the localization errors introduced in the position control algorithm.In addition,the method optimizes the position control strategy based on deep reinforcement learning techniques,thus enabling the mobile beacon to learn a control strategy with higher accuracy and fast response.Simulation results show that our proposed scheme improves the position control accuracy by more than 53%and the response speed by more than 15%.3)A deep neural network path planning model is proposed to balance the path selection decisions of the mobile beacon,and then,by training the path planning model using deep reinforcement learning methods,the localization rate and localization accuracy of unknown nodes in the entire network are optimized within the range of mobile beacon energies and maximum movement steps.4)Considering the effects of winds,waves,currents and obstacles on the path planning of the mobile beacon in OSNs environment,a new neural rule path planning scheme is proposed which uses rules to represent the interpretable part of the path planning decision and neural networks to represent the decision-making of the mobile beacon in uncertain environments,so that the trajectory of the mobile beacon can both ensure full network coverage and handle the obstacle avoidance problem in uncertain environments.The simulation results show that the proposed path planning scheme reduces the average localization time by more than 39%and the trajectory length by more than 41%in obstacle-free disturbance OSNs environments.5)To validate the proposed algorithm and train the neural networks model,in this study,we also develop a training platform for OSNs based on deep reinforcement learning.The platform simulates the effect of the physical motion of the mobile beacon affected by the ocean environment by building a 3D simulation environment for OSNs.We implemented the proposed algorithm and model in this environment to evaluate the performance of the algorithm and verify the effectiveness of the proposed scheme.