Research On Ship Collision Avoidance Method Based On Deep Reinforcement Learning

The growth of trade has driven the development of shipping industry.Nowadays,the number of ships in the water increases and the waterways become more and more crowded and busy,which increases the risk of ship collision.Once a ship collision occurs,the consequences will be very serious,and it is especially important to ensure the safety of ship navigation.As human operation is prone to various mistakes,with the development of artificial intelligence technology,the society has a more urgent need for ship intelligence.In this paper,we use deep reinforcement learning technology to conduct research on autonomous collision avoidance of ships,and the main research work is as follows:(1)Based on the geographic information of tile map and the information of Yangtze River waterway,a highly realistic ship collision avoidance simulation environment is established for the narrow waterway scenario in the inland river,and the inland river navigation mark is taken into consideration as a static obstacle.The complex ship collision avoidance problem is divided into "single-ship collision avoidance scenario" and "multi-ship collision avoidance scenario",and deep reinforcement learning is used to build a ship intelligence for the two ship collision avoidance scenarios,so that the ship has autonomous decision making ability.In addition,we have built a ship safety domain to evaluate the risk of ship collision by combining the characteristics of inland waters.(2)For the single-ship collision avoidance scenario,a single-ship collision avoidance method based on Deep Q-learning(Deep Q-Network,DQN)is proposed,and the state space,action space and reward function of the DQN algorithm are designed.For the problem that it is difficult for the ship to fully perceive the surrounding environment,a virtual radar range scanning method is used to make the state space more detailed,and a reward function model with penalty coefficients is designed to make the ship’s collision avoidance more efficient.Experiments are conducted in a simulation environment established based on part of the channel waters in Maanshan,and it is proved that under the guidance of the method,the ship can avoid the static obstacles in the environment smoothly.The proposed single-vessel collision avoidance method is proved to be more efficient,the ship’s navigation path is more economical,and a safer distance is maintained from the static obstacles in the environment,which can meet the needs of ship navigation safety through comparative experiments,and the method has certain effectiveness.(3)For the multi-vessel collision avoidance problem of ships,a multi-vessel collision avoidance method based on QMIX algorithm under the rule constraint is proposed under the guidance of international marine collision avoidance rules.For the problem that the collision avoidance rules are difficult to quantify,the meeting situation is clearly divided by using relative bearing and heading difference and other factors.At the same time,considering the ship’s avoidance of static obstacles and dynamic ships,the appropriate state space and reward function are redesigned,and the ship encounter scenarios are combined with the real environment to train the multi-ship collision avoidance algorithm model.In the experiment,a variety of ship encounter scenarios are tested and the navigation paths of the ships are obtained after the ships make collision avoidance.Based on the path results,it is judged that the method can make the ships make behaviors in accordance with the collision avoidance rules,the navigation paths are more economical,and the ships and the obstacles keep enough safe distance,which proves the effectiveness of the method.