Research On Control Strategy Of Mechanical Arm Based On SAC Algorithm

As a common automation equipment,the research on the control algorithm of the mechanical arm has been a hot topic in related fields.In recent years,with the labor shortage caused by the aging of the society and the release of the industrial 5.0 concept,the application field of the mechanical arm keeps expanding,and the application environment also becomes more complex,which puts forward higher requirements for the planning and control of the mechanical arm movement trajectory.Combining with the current popular artificial intelligence theory,this paper introduces the reinforcement learning method into the mechanical arm control,and proposes a control strategy based on Soft Actor-Critic algorithm(hereinafter referred to as SAC),so as to better solve the trajectory planning problem of multiaxis manipulator in three-dimensional space.It not only overcomes the shortcomings of traditional control algorithms such as high model dependence and low programming accuracy,but also has faster learning efficiency and higher stability compared with general reinforcement learning algorithms.The main work and conclusions of this paper are as follows:Firstly,this paper expounds the relevant theories of mechanical arm control and analyzes the shortcomings of the current common control algorithms.On this basis,this paper proposes to combine SAC algorithm based on maximum entropy theory with manipulator control,and explains the principle of SAC algorithm and the advantages of applying it to manipulator control through detailed theoretical discussion.Secondly,a strategy combination method is proposed for the complex planning task in manipulator control.After splitting the overall task into several sub-tasks,the sub-tasks are solved one by one to obtain the corresponding model.The model of the sub-task is used as the initial condition for training based on the characteristics of SAC algorithm of strategy entropy maximization,and the optimal solution of the overall task is finally obtained.Simulation experiments were carried out on Mujoco platform to verify the feasibility of strategy combination based on SAC algorithm,and the results were compared with those of direct online planning.Finally,the SAC algorithm is applied to the planning and control of multi-axis manipulator in 3d space.The simulation environment was built on Coppelia Sim platform,and the UR5 manipulator was selected as the experimental object.After modeling and analyzing the control task,the reward function of state and action variables and system parameters were designed.The experiment was divided into three groups: the control experiment in the environment without obstacles,which verified the feasibility of SAC algorithm to realize thecontrol of the manipulator by changing the joint angle of the manipulator.In the comparison experiment of two groups of obstacle avoidance planning under the environment of obstacles,different comparison standards were set according to the characteristics of different algorithms,and the planning results of SAC,DDPG and RRTstar algorithms were analyzed and compared.The results show that the SAC algorithm based on the maximization of strategic entropy improves the utilization of training samples and guarantees the optimal learning results.When applied to multi-axis mechanical arm control tasks in 3d space,compared with the other reinforcement learning algorithm and the traditional control algorithm,the planning calculation is faster,more stable,better and smoother.In solving complex planning tasks,the method of strategy combination can further improve the training speed and ensure the optimal planning results,and reduce the planning cost,which has good practical value.