Robust Optimal Tracking Control For Two Classes Of Robot Systems Via Reinforcement Learning

For robot systems subject to uncertainties,the problem of reinforcement learningbased robust optimal tracking control is considered in this paper.Firstly,the disturbances are reasonably introduced into the system to establish the dynamic model with uncertainties.Then,on the basis of some reasonable assumptions,the robust optimal tracking control strategies of autonomous underwater vehicles and Mecanum wheeled mobile robot are proposed.Finally,simulation results show the effectiveness of synchronous learning algorithm and approximate optimal tracking controller.Specifically,a robust optimal tracking control architecture with the help of the online policy iteration algorithm is proposed for autonomous underwater vehicles with model uncertainties and external disturbances.Firstly,the dynamic model of the autonomous underwater vehicle in the fixed frame is derived from velocity transformations,and a disturbance observer is introduced to compensate the total disturbance of the dynamics.Secondly,an augmented system which consists of the tracking error and the desired trajectory is constructed with observation error,and the tracking control problem can be solved by designing a robust optimal controller of the augmented system.Furthermore,the Actor-Critic framework based on neural network is applied to obtain the approximate solution of the tracking Hamilton-Jacobi-Bellman(HJB)equation of the augmented system.Because the disturbance observer increases the state of the system,the convergence rate of the system is slow when the strategy iterative algorithm is adopted.Therefore,the learning algorithm is improved by investigating the problem of robust optimal tracking control for a Mecanum wheeled mobile robot.The Euler-Lagrange motion equation with slipping is established by analyzing the structural characteristics of Mecanum wheels.Then,the tracking control problem is converted into a time-invariant optimal control problem of an augmented system.The online actorcritic synchronous learning algorithm is applied,and the standardized least squares algorithm with forgetting factor is used to train the Critic neural network,and an improved learning law is obtained to solve the optimal control problem of the augmented system.The system output is guaranteed to track the given reference signal and all states of the closed-loop system are bounded.