Research On Intelligent Collision Avoidance Method Based On Deep Reinforcement Learning

With the development of the world economy,the demand for shipping is increasing in recent years,and the navigation density of ships is increasing day by day.The traditional ship collision avoidance technology often cannot deal with the problem of ship collision avoidance timely and correctly,which makes the ship collision accidents more frequent.In order to reduce the occurrence of ship collision and increase the safety of navigation,intelligent collision avoidance of ships has become an important research field of modern maritime traffic.As the core technology of intelligent ship,intelligent collision avoidance decision has always been the focus of maritime researchers at home and abroad.To solve the above problems,this paper proposes a ship intelligent collision avoidance method based on deep reinforcement learning,which mainly includes the following aspects:(1)Conduct in-depth research on key technologies involved in the process of intelligent collision avoidance of ships.Based on the MMG separation structure,the ship’s 3-DOF model was realized and the ship’s motion was simulated.According to the "International Regulations for Preventing Collisions at Sea",the collision avoidance situation is divided in detail,and the collision avoidance responsibility of different ships under different collision situations is determined.Collision risk is calculated by fuzzy mathematics,and three stages of collision avoidance process are divided based on collision risk and TCPA,which provides sufficient theoretical basis for realizing intelligent collision avoidance.(2)Intelligent collision avoidance of ships based on deep reinforcement learning algorithm.According to the ship motion mathematical model,ship motion parameters and three collision avoidance stages,the ship action set,ship state set and reinforcement learning reward function are designed to realize the ship collision avoidance agent.Traditional single-agent deep reinforcement learning algorithm often has problems such as poor sample utilization,slow learning speed and unstable learning effect.To solve these problems,Rainbow algorithm is optimized to speed up the algorithm training speed and improve the decision performance of the algorithm.Based on the realization method of single agent,multi-agent collision avoidance is implemented based on QMIX algorithm to solve the problem of coordinated collision avoidance of multiple ships.(3)Combined with the actual navigation environment information,the simulation environment is built by Open AI Gym for collision avoidance verification.Set up different environments for training according to different encounter situations.The simulation results show that both the single-agent and multi-agent avoidance methods can avoid collision accurately.Rainbow algorithm is compared with traditional deep reinforcement learning algorithm from three aspects of timeliness,security and economy in view of the same encounter situation in single agent avoidance,highlighting the superiority of Rainbow algorithm.In this paper,the single agent and multi-agent ship intelligent collision avoidance methods are realized by reasonably analyzing the constraints of ship collision avoidance process and combining with deep reinforcement learning algorithm,which has good reference significance and application value for the study of ship intelligent collision avoidance technology.