Research On The Automatic Compliant Assembly Strategy Based On Deep Reinforcement Learning

With the rapid development of the processing manufacturing and intelligent manufacturing industries,the demand for assembly tasks is also growing rapidly.The assembly operations have become an essential part of industrial production.This change makes it urgent to improve assembly efficiency.With the development of robotics,it has gradually become a reality that robots replace humans to complete highly repetitive assembly tasks,and robotic assembly shows a broad application prospect and development space.However,the traditional assembly strategy requires establishing a contact model between the peg and the hole during the assembly process,and an optimal controller needs to be designed based on this mathematical model.This process is not easy.Therefore,applying this method to automatic assembly tasks with more complex contact states is difficult.To allow robots to complete assembly tasks more efficiently and autonomously,this paper will conduct research on automatic and flexible assembly strategies based on the deep reinforcement learning algorithm.First,this paper proposes a learning accelerated Actor-Critic learning framework(LA-AC).This framework mainly includes a set of priority sampling algorithms for empirical data,a set of algorithms to alleviate Q-value overestimation deviations,and an adaptive exploration noise.Although this algorithm can significantly improve the learning efficiency of reinforcement learning agents,it does not consider the stability issues in the stochastic dynamic system.Stability is the essential characteristic of a complete control system,and it is closely related to the reliability and safety of the system.An unstable system is usually potentially dangerous.Then,this paper proposes an Actor-Critic learning framework based on Lyapunov stability(LSAC).This learning framework mainly includes a set of Actor networks guaranteed by Lyapunov stability,a Lyapunov Critic function based on the median Q value,and a reward reshaping term based on the Lyapunov function.This paper analyzes the stability of the stochastic dynamic system modeled by the Markov decision process and reshapes the Actor network architecture based on this stability theory.The median Q value algorithm effectively alleviates the overestimation and underestimation bias of the Q value and further improves learning stability.The LSAC learning framework not only improves the learning efficiency of the reinforcement learning agent but also effectively guarantees the stability of the stochastic dynamic system.Finally,this paper designs the automatic compliance assembly strategy based on the LA-AC algorithm and LSAC algorithm.The experimental results of the automatic assembly show that the compliant assembly strategy based on the LSAC learning framework can complete automatic assembly tasks with small instantaneous contact force/torque and small fluctuations,and the pose adjustment of the robot during the assembly process is also very stable.This LSAC framework shows higher stability than the LA-AC framework.These strongly prove the practicability and superiority of the assembly strategy based on the LSAC framework.