UGV Navigation Optimization Aided By Reinforcement Learning-based Path Tracking

Unmanned Ground Vehicle(UGV)is the product of the rapid development of society,and promoting related technologies has become a new focus of international competition.Autonomous navigation is the basic and necessary ability of UGV,and is the premise of effectively completing any advanced task.Specifically,individual UGV systems need to be able to navigate autonomously in static environments under relatively fixed constraints,such as planetary exploration and geographic remote sensing mapping.Then,in complex environments such as urban areas,UGV needs to be able to navigate autonomously in dynamic scenarios when facing unknown moving obstacles.(1)UGV static scene navigation needs to solve the dual technical challenges of path planning and path following.In order to accomplish the navigation task efficiently,it is necessary to obtain the optimized path,which is usually aimed at collision-free,short distance and high smoothness.However,the scene structure is highly diverse and complex,and the universality of the path planning optimization method is difficult to guarantee,which has long been an open problem in this field.The accuracy of path following is the premise of task implementation.However,the uncertain disturbance in open environment is difficult to eliminate.The traditional automatic control method is only feasible in theory.Generally speaking,it is necessary to develop a general path planning method to cope with environmental diversity and acquire the ability to adapt to uncertain disturbances in the course of navigation implementation.(2)UGV dynamic scene navigation needs to solve the problem of real-time response to obstacles and self-motion optimization.Accurate prediction of obstacle motion situation is the premise,optimization and adjustment of its own motion state is the necessary ability,and the ultimate goal is to reach the destination or complete the task without collision.Intelligent navigation of dynamic scenes has long been the core competence of this field.Traditional mathematical methods based on non-linear theory and other deterministic models are difficult to adapt to diverse complex dynamic scenarios.Therefore,the study of "end-to-end" data-driven personification decision-making mechanism will provide a new way to solve the challenges of dynamic scene navigation technology.This paper focuses on the above technical challenges in static and dynamic scene navigation.(1)Aiming at the dual technical challenges of path planning and tracking,a "rope model" algorithm is proposed firstly.This method can successfully simulate the characteristics of the rope deformation along the axis under external forces and keep a fixed distance in the radial direction,so as to obtain the optimal navigation path.Secondly,the deep deterministic gradient strategy algorithm(DDPG)is applied to the field of path following,and the optimal navigation path can be obtained by constructing the abstract training scene.As trained in rope environment,the algorithm can quickly learn the following model with strong generalization ability.A series of experiments and results on classical complex atlas show that: 1)The "rope" model not only shortens the path distance effectively,but also enhances the smoothness of the path.2)At the cost of a small time,the depth deterministic gradient strategy can learn a good following strategy,which can adjust the system to complete the path following task independently.And it can be directly applied in more complex environment with external interference without adjusting model parameters.(2)Aiming at the real-time response of obstacles and the challenge of self-motion optimization,an option-DQN hierarchical reinforcement learning framework is proposed.The navigation behavior of dynamic scene is decomposed into "obstacle avoidance" and "arrival" sub-behaviors.The sub-behaviors are gradually solved and converged to the sub-behavior strategy selection of upper decision-making,which effectively simplifies the complexity of environmental state space search and provides a humanoid decision-making mechanism.Experiments on real video surveillance scenarios show that the proposed method can accomplish specified tasks stably and efficiently in complex dynamic scenarios,and the generalization ability of reinforcement learning decision framework is strong without fine-tuning parameters.Generally speaking,this method provides a general path planning capability.Path following mechanism can effectively adapt to uncertain disturbances in open scenarios.Hierarchical behavior framework has the ability of anthropomorphic navigation in real dynamic scenarios.