Robot Control Based On Deep Reinforcement Learning

Deep reinforcement learning has been developed in-depth in recent years,but most of the research is focused on how to improve the speed of neural network learning and optimization effects,and lacks analysis of stability.Since the neural network is a black box model,and in most algorithms,the neural network is directly used as the controller,so the stability analysis needs to be built on the analyzability of the neural network.In recent years,although many scholars have conducted research on the resolvability of neural networks,there is currently no method that can accurately analyze all the characteristics of neural networks.Most methods can only achieve local analysis of neural networks.This paper combines the SAC algorithm and sliding mode control in deep reinforcement learning,and proposes a feature root-based reinforcement learning method,named EBRL(Eigenvalue-Based Reforcement Learning).It solves the method that can not guarantee the stability of the system theoretically when the neural network is used directly as the controller in deep reinforcement learning.In the EBRL algorithm,we do not let the neural network directly control the robot system,but let the neural network design the eigenvalues corresponding to the parameters in the sliding mode controller,and ensure that the eigenvalues output by the neural network can always ensure the stability of the system.And in order to prevent the rate of parameter change in the controller from being too large,spectral normalization is introduced in this paper to limit the rate of change of the output of the neural network.EBRL inherits the robustness and stability of sliding mode control,as well as the optimization characteristics of deep reinforcement learning.The algorithm has good robustness and excellent control performance under the premise of ensuring the stability of the system.The learning speed of this method is higher than the traditional deep reinforcement learning method,and the learning process is also more stable,and the control effect has been greatly improved compared with deep reinforcement learning and sliding mode control.In this article,we will propose two inference processes that are equivalent,but the training process is not equivalent to the combination of deep reinforcement learning and sliding mode control,and compared with the simulation of the control effects of SAC and sliding mode control,and proposed Further optimization methods.In order to achieve the performance comparison between the algorithms,we used Lagrange mechanics to carry out mathematical modeling of the stand-ball robot,the balance car and the inverted pendulum for simulation comparison.In the modeling of the station ball robot,in order to facilitate our analysis of the internal force of the system,we also used Newtonian mechanics to model it,so that we can judge whether the internal force of the system can meet the constraints required for control in simulation.In order to verify the real performance of the algorithm,we designed and fabricated the hardware system of the above three robots to verify the stability,robustness and versatility of the algorithm.