Q-learning Based Controller For Obstacle Avoidance In Manipulator

In recent years,reinforcement learning has received extensive attention in the field of robot control.Industrial manipulators are widely used in automated production lines as a common tool in industrial production.How to apply the reinforcement learning theory to the motion control of industrial manipulators and make them have a certain ability of autonomous learning,which is of great significance to expand the application and reduce the operation difficulty of industrial manipulators.In this paper,aiming at the problem that the traditional reinforcement learning algorithm cannot be directly applied to the high-dimensional continuous state-action space,a new method is proposed and the problem of the manipulator trajectory planning for obstacle avoidance is successfully solved.The method combines the traditional path planning method with the Q-learning algorithm.By redefining the state space and action space of manipulator,and separating the path planning and trajectory planning,it not only reduces the search space,but also shortens the learning time.And it ensures the safety of the manipulator during the learning process.The specific work of this paper is as follows: Firstly,the traditional manipulator motion control theories are reviewed,mainly including kinematics analysis,dynamic analysis and trajectory planning methods of the manipulator.Besides,the corresponding compensation methods are analyzed for the influences of joint torque fluctuation?load mutation and sensor data inaccuracy in the closed-loop control system of the manipulator,and the effectiveness of this methods are proved in the simulation experiments.Secondly,the reinforcement learning theory is introduced in detail.Based on this,the Q-learning algorithm is applied to the task of learning obstacle avoidance behavior.By redefining the state-action space,designing the reward function and improving the search strategy,a learning controller based on Q-learning is designed.This controller can enable the manipulator to complete the obstacle avoidance behavior through autonomous learning,thus enhancing the ability of the manipulator to perform a task independently.Then,this method is compared with the traditional manipulator obstacle avoidance methods in different experimental environments.The superiority of this method is verified by simulation experiments.Finally,the actual experimental environment of the Dobot Magician manipulator is designed and the method proposed in this paper is run on the manipulator.The experimental results show that themanipulator can complete the obstacle avoidance behavior by autonomous learning.And the feasibility of the method applied in industrial manipulators is verified.At the end of this paper,the advantages and disadvantages of the method are summarized in detail,and the problems that can be improved in the future are put forward.