Research On Autonomous Exoskeleton Control Based On Reinforcement Learning

Because the world population is gradually increasing and the aging phenomenon is becoming more and more serious,many people with mobility difficulties need technology to assist them in their daily life.Lower limb exoskeleton robots are one of the important technologies.For these patients,lower extremity exoskeleton can be utilized to improve movement function or help training return to normal.However,it is difficult for high paraplegics whose arms and legs are disabled to operate the normal walkingassisting exoskeleton.Therefore,the autonomous exoskeleton was developed for these high paraplegics.It assists high paraplegics to walk without additional auxiliary tools.However,the existing gait generation algorithms are all based on simplified mathematical model,which has poor accuracy and is unfriendly to the wearer.This research aims to propose a gait generation policy based on reinforcement learning,so that the wearer can maintain dynamic balance when using the autonomous exoskeleton with the gait generation policy to walk.Morever,this research proposes path planning method corresponding with the daily use of wearers to bring a good wearing experience to people in need.The main contents are as follows:Firstly,based on the existing autonomous exoskeleton robot platform,an exoskeleton simulation platform close to the actual reality is established.It is able to quickly respond to the external input control instructions and provide real-time feedback.The obstacle information of the actual environment is obtained with depth camera and image processing algorithm.Then,the subsequent path planning policy is applied to avoid obstacles.Secondly,a gait generation algorithm of autonomous exoskeleton robot based on reinforcement learning is proposed.The algorithm can imitate the given reference trajectory and learn a walking policy which is suitable for the autonomous exoskeleton robots.Combined the walking characteristics of biped robots and the increased constraint conditions,the model action space is reduced and the training speed of the model is accelerated.A PD controller is applied to each joint of the exoskeleton during the training algorithm,which is convenient to transfer the gait generation policy to the physical robot.Finally,considering the needs of the wearer to avoid obstacles in the indoor environment,a path planning and gait generation algorithm based on hierarchical reinforcement learning is proposed.The high-level controller is responsible for path planning,and the utilized reinforcement learning algorithm has the characteristics of high accuracy and fast path generation.With the depth camera,the reasonable exoskeleton foothold can be quickly planned in the indoor environment.Based on the above gait generation algorithm,the low-level controller increases the limitation,which enables it to follow the foothold,so as to achieve the wearer's goal of avoiding obstacles and walking.It is verified in the simulation and actual environment.