Research On Obstacle Avoidance Of Robotic Manipulator Based On Reinforcement Learning

This paper reports on the obstacle avoidance problem for robotic manipulators. Machine learning (ML) has become an important means to enhancing the intelligence for the robots, as the increasing requirement in practical application. After a period of learning, reinforcement learning (RL) robots or agent without prior knowledge can avoid obstacles just depend on the exploration and the environmental response. The RL method was applied to this domain because it relies on the sensors'perception on environment, not the need of accurate modeling environment and robot itself.A multi-agent obstacle avoidance system was built for a 3-DOF planar manipulator. The system combines a repelling influence related to the distance between manipulator and nearby obstacles with the attracting influence produced by the angular difference to drive the manipulator moving.According the real-time demand of manipulator control, the Sarsa(λ) algorithm, which is a major method of RL, was selected as a basic control strategy for its on-policy feature and efficiency. The implement process of the algorithm was given and in the end of this paper, a simulation experiment showed the RL method's feasibility and availability.As the obstacle avoidance problem for robotic manipulators has continuous state space, the state space partitioning is a important factor to improve the applicability and efficacy of reinforcement learning algorithms. The k-means clustering algorithm is used to partition a state space. A series of simulations are provided to demonstrate the practical values and performance of the proposed algorithms in solving robot motion planning problems.A simulation platform was developed for the obstacle avoidance problem of a 3-DOF planar manipulator on the Microsoft.NET. It is used to show the simulation trial results and analyze the obstacle avoidance algorithm. A series of experiments demonstrate the system having strong capacity for collision avoidance, even in some complex environments.