Research On Impedance Control Strategy For Robotic Machining Based On Reinforcement Learning

With the development of robot technology and industry,robots play an important role in industry and even people's daily life.However,since it is difficult to obtain the control model parameters of the robot,there are still no convenient and effective methods on how to select the robot controller parameters or obtain the optimal controller.Moreover,when force is introduced to the controller to perform compliance control,inappropriate control parameters will easily cause vibration and robots can by no means function well.Based on reinforcement learning,this research analyzes the cost function,value function,and strategy model of the control system,and then proposes a flexible method for the optimization of the robot impedance controller.The main achievement of this thesis are as follows:First,the impedance control model is studied.The principle and implementation of the positon-based and torque-based impedance control method are analyzed and their performance is compared.The stability domain of the impedance controller and the overall stability domain of the robotic machining system is obtained.The impact of impedance control on the contact force is analyzed.The principle and shortcomings of constant force control based on zerostiffness impedance control are analyzed as well.Second,an Actor-Critic reinforcement learning method is proposed to obtain the optimal robot controller.The core concepts and update method of reinforcement learning are analyzed.The concepts of the control system and those of reinforcement learning are compared.Methods to obtain optimal robot controller by linear quadratic strategy and by reinforcement learning are studied.The cost function,value function,and the state-action function of the control system are modeled and their update methods are studied.The method to obtain optimal robot controller based on reinforcement learning is proposed and the learning progress is specified.Numerical simulation is performed to show the impact of cost function on the learning progress and the system response.Third,the Actor-Critic reinforcement learning method is employed to obtain the optimal robot impedance controller and a safe learning strategy is proposed.The robot impedance controller is simplified and converted into the form of state feedback of position,velocity,and external force.The form of cost function is designed and the method to obtain optimal robot impedance controller based on reinforcement learning is proposed.A safe learning strategy is proposed where the learning progress is separated from the check progress to improve the safety and efficiency of learning progress.Simulation experiments verify the effectiveness of the above method.Finally,the experiments to obtain the stabilization controller and impedance controller of UR5 are conducted.The controller of the elbow and wrist is designed with the proposed method,which proves that such a method is valid in different control systems.Then the impedance controller of the wrist is designed to realized stable interaction with the user,which means that such a method is of great use to get proper impedance controller.Finally,a processing experiment for a thin-walled workpiece is conducted to show the application prospect of the proposed method in compliant machining.Controllers of the shoulder and the elbow and the impedance controller of the wrist is obtained with the proposed method and a promising machining result is obtained.