Research On Ship Path Planning Based On Actor-Critic Algorithms

With the development of artificial intelligence technology,ship intellectualization has become an inevitable trend of the development of shipping industry,and route planning technology has become one of the seven key technologies of intelligent ship research.Therefore,it is of great theoretical value and practical application significance to explore more optimized ship route planning methods for the development of safe and intelligent shipping.There are many algorithms of path planning at present.Most of them deal with the planning environment and actions discretely,which is not in accordance with the characteristics of actual path planning.At the same time,less path planning is carried out according to the characteristics of ship navigation,and most of them adjust their own route dynamically according to the influence of other ship's route on their own ship,so as to guide them.To solve this problem,this thesis explores the use of actor-critic method of deep reinforcement learning to solve the problem of path planning in continuous space,and combines the characteristics of ship navigation and the requirements of path planning to improve actor-critic method.The main work is as follows:1)Research on Collision Avoidance between Ships and Static Obstacles Based on Ship Navigation CharacteristicsBased on the analysis of ship path planning,the ship field,ship collision risk model and calculation method of motion parameters for ship path planning are constructed.This thesis has explored the agreements between actor-critic method and path planning,and re-established the reward function of actor-critic method,based on which the collision avoidance strategy between ships is designed.What's more,researching on a method of constructing map,a static obstacle avoidance method suitable for global path planning and local path planning is designed based on ship domain model.2)Research on Local Path Planning for Ship Collision AvoidanceTaking safety and timeliness as the goal of ship's local path planning,combining the designed collision avoidance strategy and static obstacle avoidance method with MADDPG algorithm,the reward function in MADDPG is reconstructed to achieve safety.To improve the convergence speed of the algorithm,on the one hand,combining ship field and collision risk,the reward function in MADDPG has been reconstructed.On the other hand,the experience replay buffer of MADDPG algorithm has been reconstructed to explore an efficient experience replay method based on mixed sampling.A ship local path planning algorithm based on MADDPG is designed to achieve the purpose of planning local collision-free path for multi-ships at the same time.3)Research on Safety and Economy Oriented Global Path Planning for ShipsTaking security and economy as the goal of global path planning,the static obstacle avoidance method is combined with the single agent actor-critic method(PPO),and the reward function in PPO is redesigned to achieve security.Aiming at the blindness of PPO algorithm in the early stage of path planning,the PPO exploration strategy is designed by expanding the initial state set of the algorithm.A ship global path planning algorithm based on PPO is designed to ensure the economy of global path planning under the premise of safety.4)Experiments and AnalysisTaking timeliness,safety and economy as evaluating indicators,this thesis validates and analyses the global and local path planning method and its improvement strategy.Meanwhile,it compares with other ship path planning methods through experiments.The results show that the proposed method in this thesis can effectively plan safe local path,safe and economic global path.