Deep Reinforcement Learning Based Lifting Path Planning For Tower Cranes

Cranes are important equipment in industrial production.Tower cranes are widely used cranes in the construction industry.With the development of control technology,tower cranes are gradually developing in the direction of automation and intelligence.The crane’s lifting path planning is one of the focuses of the research.The use of intelligent lifting path planning algorithms can improve the lifting efficiency and reduce the occurrence of safety accidents.Most tower crane path planning algorithms require accurate environmental information.On construction sites,the phenomenon that the position of the obstacle moves frequently occurs Traditional planning algorithms need to be re-planned to plan a new feasible path.However,the re-planning time is long and the real-time performance is poor,which is easy to cause safety problems.In recent years,deep reinforcement learning is widely used in motion planning of robot manipulators and mobile robot navigation.Deep reinforcement learning can make real-time autonomous decision-making based on the current state information obtained through self-learning of the agent,which can be used for path planning in complex unknown environments.This research has carried out the lifting path planning of tower cranes based on deep reinforcement learning,solving the problem of autonomous path planning of cranes in unknown environments.The main research contents are as follows:Firstly,the problem of lifting path planning in continuous action space is studied.A Twin Delayed Deep Deterministic Policy Gradient-based tower crane lifting planning algorithm in continuous action space is proposed.An improved hindsight experience replay method is proposed,which solves the reward sparsity problem due to the large space of the construction site.At the same time,the original exploration strategy is modified,which accelerated the training speed of the network and improved the planning success rate.In addition,a simulation environment is established in Webots and simulation experiments are carried out to verify the effectiveness of the algorithm.The result shows that the planning time of the proposed deep reinforcement learning algorithm is much shorter than that of the traditional algorithm,and it can meet the real-time path planning task of cranes.Secondly,considering the problem that it is difficult for cranes to operate multiple degrees of freedom movements at the same time in actual work,the lifting path planning algorithm in the discrete-continuous hybrid action space is studied.A deep reinforcement learning algorithm suitable for hybrid action space is proposed.The continuous parameter value is output by the policy network,and the discrete action is selected by the critic network.At the same time,only one degree of freedom motion is operated and the value of operation is continuous.Simulation experiments are carried out.The result shows the planned path is smoother than the path generated in the continuous action space.The proposed method is more suitable for the actual crane lifting task and the design of lifting planning system to assist manual operation.Finally,a path optimization method is proposed for the particularity that the lifting load of the tower crane must be located above the obstacle.The method can be optimized for path planned by deep reinforcement learning method without interacting with the environment.The experimental results show that the optimized path has improved smoothness,length and movement time of planning path compared to the original path.