Research On Path Planning Based On Markov Decision Processes For Palletizing Robot

With the development and application of industrial robots, industrial automation gradually hasbeen improved. The application of palletizing robot also makes the automatic stacking device andautomatic warehouse technology into the popularity and development stage. Multi-sensor integrationand information fusion technology make path planning for robot more intelligent. This paper planspath for palletizing robot with known or unknown workspace based on robot kinematics model, andprovides theoretical basis for its practical application.This paper establishs the model of kinematics based on the D-H coordinate method forpalletizing robot, and according to the kinematics relationship of the adjacent connecting rod,kinematics equations are listed, then position of end effector could be expressed as parametricequation, according to which the workspace would be analyzed. Kineto-static model is built,equilibrium equations and moment balance equations are established, then the driving mechanism ofpalletizing robot could designate the appropriate servo motor and reducer.Based on Markov Decision Process (MDP), path planning to avoid obstacle is worked in theworkspace of palletizing robot. The combined states, combined actions, transistion probabilities andreward functions are given on the basis of kinematics model of robot, then using the algorithm ofLeast-Squares Policy Iteration solve the optimal policy, and plan simulation of3-D path for robot.On the basis of MDP model, the state clustering is introduced to estabilish Hierarchical MDP toplan the path. By applying the spactial position of obstacles, workspace of palletizing robot aredivided into several state clusters which are used as state space of Hierarchical MDP, then usinghierarchical value function iteration algorithm to simulate2-D path and3-D path of robot.For the uncertain workspace of palletizing robot, the model of Partially Observable MDP is built,and belief state is introduced, then Partially Observable MDP model is converted to MDP modelwhich is based on belief state. Using laser ranger finder detects the workspace in the real-time, thepath of robot is planned and simulated on-line.Finally, control system is designed for palletizing robot, and control programming of optimalpolicy is written. Which are to test the obstacle avoidance ability of palletizing robot and motionprecision of end effector in the all three models.