Terrain-adaptive Control Algorithms Of Simulated Dog Based On Deep Reinforcement Learning

With the improvement of people’s living standards,people put forward higher requirements for film and television animation,film special effects,and pictures in computer games and so on.In film special effects and game animation,cool character locomotion pictures are often needed.Traditional key-frame-based locomotion generation technology requires a lot of manual key-frame screening by professionals,with high redundancy.Recently,with the rapid improvement of computer computing power,physical simulation-based method has become popular.It uses computer to simulate the process of natural real character movement.The method considers the interaction process between the real-world attributes of the character and the environment,and does not need to care about the details of the physical movement process.In recent years,it has become a locomotion simulation technology with potential advantages.Most of the existing role locomotion control technologies based on physical simulation consider the environment of role locomotion to be flat terrain,without considering more complex terrain factors.Reinforcement learning is often used to solve the decision-making problem while interacting with the environment,which provides a theoretical basis for controlling the role of physical simulation,and neural network has a remarkable effect in feature extraction.In order to solve the problem that most of the controllers can’t output effectively to adapt to different terrain strategies,this paper uses Bullet physical engine to provide simulated dog model,builds a controller based on deep neural network,and realizes the locomotion control of the simulated dog in different terrain by using reinforcement learning algorithms under off-line strategy to achieve terrain adaptive goals.Firstly,establish a Markov decision-making model with high-dimensional continuous state action space,including the representation of simulation role state,the selection of action parameters,and the construction of reward function,the control problem is transformed into a series of decision-making processes.Second,based on the behavior modeling of finite state machine,a locomotion cycle is divided into four stages.A state transition model of the role in the course of the locomotion cycle is established to complete the low-level control of a complete locomotion.A deep neural network is built to input the high-dimensional continuous state and topographic features of the role,and output strategies of different deep reinforcement learning algorithms are used to realize the output of the action parameters of the role.The parameters of neural network are trained by building simulation platform.This paper verifies the validity of locomotion control of simulated dog under different terrain by combining deep neural network with reinforcement learning algorithms,and realizes the adaptive locomotion of simulated dog in dynamic environment.