Real-time Policy Approximation Of Autonomous Vehicle Optimal Tracking Control And Its Validation

Motion control is the core module of intelligent vehicles,which is usually formulated as a continuous-time finite-horizon optimal control problem.Existing research generally solves this problem using rolling-horizon optimization,but due to the nonlinearity of dynamics and strong state constraints,the efficiency of online optimization is very low,which makes it difficult to meet the real-time requirements of onboard controllers.To address this challenge,this research explores the solvable optimality conditions of this type of control problem,studies the equivalence criterion of the Hamilton-Jacobi-Bellman equation,designs an intelligent vehicle approximate dynamic programming control algorithm that satisfies state constraints,and tests the trajectory tracking and collision avoidance control performance using offline simulation and true-vehicle experiments,laying a theoretical foundation for the development of high real-time intelligent driving control systems.First,to address the challenge of numerically solving such optimal control problems,a new horizon-decomposition Hamilton-Jacobi-Bellman(HJB)equation is established.The equivalence relationship between the time partial derivative of the value function and the terminal time utility function is established through Taylor series expansion and initial time equivalence principle.The partial time derivative term is transformed into a quantifiable expression,reducing the number of unknown terms in the HJB equation from three to two,laying the foundation for the efficient iterative solution of finite-horizon HJB equations.Next,a finite-horizon policy iteration method with a monotonic convergence guarantee is proposed for optimal control problems containing nonlinear dynamics.Based on the horizon-decomposition structure of the Lyapunov equation,the monotonic decrease of the value function is proved,and the conclusion that the policy can converge to the unique optimal solution is obtained.A two-step iterative algorithm for gradient descent of the value network and policy network is designed to achieve optimal policy parameterization for typical optimal control problems.Simulation results show that the error of this parameter solution is less than 1% compared to the theoretical optimal solution,which not only ensures high-precision control performance but also significantly improves online computation efficiency.Furthermore,an approximate dynamic programming solution algorithm that uses penalty functions to handle vehicle state constraints is developed for trajectory tracking and collision avoidance control tasks in intelligent vehicles.Considering static road topology constraints and dynamic traffic participant state constraints,a constraint-based policy optimization problem is constructed,and a policy update mechanism including penalty factors is designed to balance tracking performance and safe collision avoidance.Simulation results show that the learned policy can achieve high real-time trajectory tracking control functionality while satisfying the vehicle collision avoidance constraints.Finally,applied on the Lincoln MKZ intelligent driving platform,the performance of the approximate dynamic programming algorithm is verified.The deployed control strategy for the vehicle consists of a three-layer fully connected neural network,with the input being the vehicle state,reference trajectory,and vehicle position,and the output being the control quantities of acceleration and steering wheel angle.True vehicle experiments show that the proposed algorithm has an average trajectory tracking error of less than 0.2m and an average single-step computation time of only 0.22 ms,which is 529 times faster than the commonly used Casadi optimization solver in terms of online computation efficiency.