Research On Vehicle Adaptive U-turn Problem Based On Deep Reinforcement Learning

With the continuous development of economy and technology,many industries are gradually moving towards intelligence.As an innovative product that conforms to the trend of the times,autonomous driving vehicles have integrated traditional automotive technology with artificial intelligence technology.However,current research is still limited to partial and conditional autonomous driving stages,mainly focusing on driving tasks on regulated roads,such as everyday driving and obstacle avoidance on highways and urban roads.There is still some way to go before achieving fully autonomous driving.Fully autonomous vehicles should be able to handle any driving task in any scenario,such as performing U-turns,overtaking,and parking on rough roads like residential area roads and parking lots.U-turn tasks are the most common and relatively tricky driving tasks,and their solutions can be extended to other autonomous driving tasks.Deep Reinforcement Learning(DRL)has excellent decision-making capabilities and is one of the critical technologies for achieving autonomous driving.In this thesis,DRL addresses three issues encountered by autonomous vehicles performing U-turns on rough roads: 1.Existing autonomous vehicle simulation environments cannot implement custom driving scenarios and are challenging to develop further;2.The proposed U-turn model offers sparse rewards to simulate actual driving scenarios,making it difficult for the intelligent agent to learn the U-turn policy;3.Sparse rewards lead to insufficient exploration by the intelligent agent,making it challenging to find the optimal policy.To address these issues,the main work of this thesis includes:For the U-turn task,suitable parameters are selected to establish the autonomous vehicle dynamics model,and a reward function is designed according to the natural environment.The involved models are simulated,and training and testing are performed using three standard reinforcement learning benchmark algorithms.Due to the poor performance of the benchmark algorithms in sparse reward environments,a proximal policy optimization algorithm based on a hierarchical reinforcement learning mechanism is proposed.The position information and the vehicle’s track angle in the environment are used as inputs.A multi-scale fusion convolutional neural network is proposed for the environment model,serving as a task for extracting state value features.Compared to the benchmark algorithms,the proposed method shortens the U-turn time and increases the U-turn success rate.To address the issue of insufficient exploration by intelligent agents caused by sparse rewards and the tendency to fall into local optima,a proximal policy optimization algorithm based on the curiosity mechanism of previous and subsequent frames is proposed.Matrix data is used as input,and the curiosity mechanism predicts the previous and subsequent state frames.The prediction error is used as a reward to increase the exploration rate of the intelligent agent,thereby finding the optimal policy.Experimental verification shows that the proposed algorithm can obtain higher rewards and learn the U-turn policy.This thesis proposes a solution based on deep reinforcement learning for autonomous vehicles performing U-turn tasks on rough roads.It introduces a proximal policy optimization algorithm based on a hierarchical reinforcement learning mechanism and a proximal policy optimization algorithm based on the curiosity mechanism of previous and subsequent frames to address existing issues.Experimental results show that the proposed methods can effectively shorten the vehicle’s U-turn time,increase the U-turn success rate,and enhance the exploration capability of the intelligent agent in sparse reward environments,providing an effective solution for autonomous vehicles performing U-turn tasks on rough roads.