Walking Control Of Planar Biped Base On SoC And Reinforcement Learning

Biped robot walking control research is the most complex research field in robotics,and has been drawn scholars' attention extensively.Traditional robot control algorithms need to establish the precise mathematical model,and the robustness of the control algorithm is very poor.However,the physical prototype structure is too complex and difficult to accurately model,affecting the final control result.In recent years,with the rapid development of artificial intelligence,more and more scholars have used reinforcement learning to realize the walking control of robots.However,current robotic walking control based on reinforcement learning usually require the reference gait.And it is difficult to realize the continuous action and continuous state of the traditional reinforcement learning.Focusing on these problems,this thesis builds robot walking learning system based on the Asynchronous Advantage Actor-Critic(A3C)algorithm,without introducing reference gait.When the training result is verified on prototype,the processing system of prototype is so weak that the real-time performance is bad,which affects the training result.In this thesis,the trained robotic walking neural network is accelerated on the System on Chip(SoC).The main research work of this thesis is:1.The dynamic model of the simplest passive biped robot is analyzed,and the influence of the impulsive push on the walking motion of the simplest biped robot is also discussed.The numerical simulation is carried out.Aiming at its shortcomings,a new model of telescopic knee biped that under the control of hip drive and impulsive push is proposed.2.The improved biped robot model is built in the simulation environment V-REP,and the gait learning system of the planar biped robot is built by using A3 C algorithm.3.According to the actual debugging experience to further optimize the state observation vector and the motion execution vector of the designed controller,the training results of different learning network models are compared.By constructing the related function between the update cycle with the reward value,the stability of the training period is further improved.Finally,the results of the study are that the trained neural network can control the planar biped robot to walk more than 5500 steps at the speed of 1.5km h in the simulation environment.4.An accelerator design working on SoC and scheduling mechanism for complex neural networks was designed.The calculation between the hidden layers is re-scheduled,the data dependence and conflict are eliminated to optimize the system with pipeline design ideas.The multi-modular design idea is adopted to divide the complex calculation of the hidden layer into multiple sub-modules,which partly eliminates the Momory Wall problem,further optimizes the accelerator and improves the parallel computing capability of the system.Finally,the robotic walking control neural network is accelerated on the SoC to,whiche improves the real-time performance of the robot control system,and lays the foundation for the later verification of the physical prototype.