Research On Vehicle Active Suspension Control Based On Deep Reinforcement Learning

The active suspension system has a strong potential for vehicle ride comfort control due to its real-time adaptability to different road surfaces and external disturbances.In recent years,it has been extensively studied by automobile manufacturers and scholars.On the one hand,traditional model-based control theory relies on the accuracy of the mathematical model,and on the other hand,linear model is usually used to ensure the real-time performance of the system,which leads to its great limitations in the actual suspension control.The reinforcement learning method is based on data-driven and does not depend on strict mathematical models,and has strong application potential in active suspension control.In order to solve the problem of poor adaptability to the parameters or working conditions of traditional suspension control methods,Research on the deep reinforcement learning control method of active suspension was conducted,and Deep Q-network(DQN)reinforcement learning algorithm of active suspension and half-car active suspension-oriented Deep Deterministic Policy Gradient(DDPG)algorithm were designed.The simulation test of vehicle suspension control under random road conditions was carried out,in order to reduce vehicle vibration and improve vehicle ride comfort.The research contents of this thesis include:Firstly,the dynamic models of the 2-DOF,semi-vehicle and full-vehicle active suspension and road excitation model were established.The vehicle suspension system simulation environment for interaction with reinforcement learning agents was built,and the system dynamic response data set corresponding to the suspension state-action space was constructed.Secondly,active suspension deep reinforcement learning control problem was constructed,an active suspension DQN algorithm for speed bumps was designed,a reward function that minimizes body acceleration,tire dynamic travel and suspension dynamic deflection was proposed.The influence of learning rate,discount factor,neural network architecture and other parameters on the training effect of suspension control strategy was studied,so as to optimize the reinforcement learning training speed and suspension control performance,and solve the optimal control strategy of active suspension considering both comfort and handling stability.Simulation results show that,compared with the traditional suspension control strategy,the DQN-based active suspension algorithm has better ride comfort and adaptability to working conditions.Then,in order to improve the convergence speed of reinforcement learning in the semicar active suspension training with higher degrees of freedom(greater state-action space),an active suspension control strategy based on DDPG is proposed,which comprehensively considers vehicle body acceleration,pitch acceleration,etc.Reward function for vehicle suspension performance index was designed,control strategy training and ride comfort simulation test under random road conditions were conducted.Simulation results show that the DDPG-based active suspension control strategy has a faster convergence rate than the DQN algorithm.The test results of different road conditions and driving speeds verify the generalization performance of the proposed algorithm.Finally,in order to verify the feasibility of the active suspension deep reinforcement learning control strategy designed in this thesis,an active suspension hardware-in-the-loop simulation test platform based on the dSPACE real-time simulation system was built.Micro AutoBox was used to simulate the 2-DOF suspension system,road excitation and the reinforcement learning control algorithm.The electromagnetic actuator was used as the actuator to output active control force.The experimental results prove that the proposed deep reinforcement learning control algorithm can effectively improve the ride comfort of the vehicle.