Motion Planning Of Six-legged Robot Utilizing Deep Reinforcement Learning

Six-legged robot has the advantages of diverse movement modes,good stability,and strong carrying capacity.It has strong adaptability to complex and unstructured terrain,and its application scenarios are wide,such as rescue and disaster relief,interstellar exploration,etc.For the study of its motion planning,it has been one of the research hotspots of six-legged robot.At present,the motion planning of six-legged robots mostly adopts the method of modular control based on mathematical model construction,and it is usually divided into path planning and trajectory planning.However,the six-legged robot is a multi-input multi-output system with multiple sensors,so its structure is relatively complicated,which brings uncertainty of model and state estimation.When the modular method is used for research,modeling is more difficult.Deep learning is an end-to-end model fitting method.Through reasonable design and training of deep neural networks,action output can be obtained directly from data input,so deep learning can be used to reduce the difficulty of modeling.Reinforcement learning is a machine learning method that learns through continuous trial and error and obtains decision-making capabilities.It can be used for robot motion planning research.It is just a traditional reinforcement learning algorithm that updates the value function in the form of a table,which is difficult to deal with.The six-legged robot's high-dimensional continuous state space and action space.The deep reinforcement learning algorithm that combines deep learning and reinforcement learning combines the characteristics of the two,so the deep reinforcement learning algorithm can be used to study the motion planning of the six-legged robot.This paper takes a six-legged robot as the research object,and conducts motion planning research based on deep reinforcement learning algorithm.The main research contents are as follows:1.According to the kinematic characteristics of the six-legged robot,the state space and action space of the six-legged robot are analyzed and created,and the corresponding reward function is designed according to the actual motion planning problem.2.Based on the near-end strategy optimization algorithm,the design of the entire algorithm is completed on the basis that the state space and the action space have been designed.In addition,in the simulation software V-rep,a simulation model of the robot was built,different terrain environments were designed,and model training and algorithm implementation were carried out.3.Aiming at the problem that the algorithm requires a lot of samples and some of the results are partially optimal,the terrain information in the state space is processed by convolutional neural network to optimize the processing of state information.By changing the size of the terrain and the number of obstacles,course learning was introduced to assist in training.4.By using RGB-D depth camera and other sensors to obtain state space information such as terrain information on the physical sixlegged robot,and perform corresponding actions based on the model obtained in the simulation environment,physical experiments were carried out,and the depth-based enhancement was successfully The learning and implementation of the six-legged robot's motion planning algorithm has been applied to the physical six-legged robot from the simulation environment.