Research On Obstacle Avoidance Simulation Of Multi-DOF Manipulator Based On Reinforcement Learning

The main problem considered in this thesis is how to effectively and flexibly carry out human-machine collaboration and complete obstacle avoidance during the control process of the cooperative manipulator.Because of its characteristics,the working scenarios of the collaborative manipulator mainly considered in this thesis are collaborative small cargo handling,auxiliary tool picking and assembly device assisted picking,etc.,which have the characteristics of low demand precision and high flexibility.The goal of this thesis is to obtain a deep reinforcement learning model through training in a simulation environment,and use it to replace the decision-making obstacle avoidance function of the collaborative manipulate control algorithm.This thesis takes the robotic end-effector’s position action as the action of the reinforcement learning agent,and takes the human-cooperation interaction physical information of the manipulator as the environment state information of the reinforcement learning.According to the environment state information,the reward function of reinforcement learning is constructed,and the reinforcement learning is constructed until this stage.The reinforcement learning model is constructed to improve the work efficiency and safety of the manipulator.The main research contents and contributions are as follows:First,from the perspective of improving the training efficiency of manipulator,a deep reinforcement learning algorithm model based on the nested model is designed based on the DDPG+HER model,which strips off the inverse kinematics model of the manipulator and speeds up the training speed of the model and performs better than the non-nested model in experiment.Second,considering ways to avoid obstacles,a dynamic target strategy is designed.When an obstacle is detected around the end effector of the manipulator,the strategy will expand the target search range,searching for points near the trajectory as the target point to avoid the obstacle.Third,from the perspective of practical application,this thesis builds a simulation test platform based on Kinct V2,and imports real human motion information as well as tests and proves the effectiveness of the algorithm in experiment.This thesis uses traditional intelligent path planning algorithm and deep reinforcement learning algorithm and its improved algorithm to complete the obstacle avoidance simulation experiment of the collaborative manipulator in the process of human-robot collaboration.The results show that:(1).When the operator(obstacle)is far away from the manipulator,manipulator moves according to the original trajectory.(2).When the operator approaches the manipulator,the performance of the manipulator based on the improved RRT* is worse than that of the manipulator based on the deep reinforcement learning model.The former will take a longer trajectory length to avoid obstacles.Furthermore,in order to analyze the importance of the improved approach,this thesis designs a related ablation study and analyzes the importance of the nested model and dynamic target strategy.The result shows that the dynamic target strategy can provide greater contribution to the thesis’ s target task.Finally,this thesis uses Kinect V2 camera to collect real human bone motion information,and makes it interact with the collaborative manipulator in the simulation environment for test experiments,which further demonstrates the effectiveness of this algorithm.