Hierarchical Interactive Learning For Control Of A Human-Powered Augmentation Lower Exoskeleton

As a human-centered interaction wearable robotic system,lower exoskeleton has gained considerable interests in recent years.For different application scenarios,lower exoskeletons can be separated as three kinds: power augmentation,walking assistance and rehabilitation.In this thesis,we focus on the power augmentation lower exoskeletons,which can be applied in military use,rescue and instrustrial load carrying.The goal of control system of power augmentation lower exoskeleton is controlling the exoskeleton follow the pilot's motion as soon as possible.In this thesis,the author designs a hierarchical interactive learning framework based on reinforcement learning method,which aiming to make the exoskeleton can follow the pilot's motion faster.Furethermore,the propsed learning framework can also make the exoskeleton adapt different pilots and different pilot's gait patterns.Main contributions of this thesis are described as follows:1.There are two main drawbacks on traditional Sensitivity Amplification Control(SAC)algorithm: 1)SAC algorithm need accurate dynamica model of lower exoskeleton;2)SAC algorithm can not adapt different pilot and different pilot's gait patterns.In order to solve these problems,an interactive learning control approach is proposed in this thesis which name is Adaptive Sensitivity Amplification Control(ASAC)strategy.In the proposed ASAC strategy,reinforcement learning method is utilized to learn the sensitivity factors of SAC controllers online,which aiming to adapt different pilot and different pilot's gait pattens.In the porposed ASAC stategy,Local Gaussian Process Regression(LGPR)is employed to estimate the pilot's motion of next gait cycle.2.In the proposed ASAC strategy,amount of data should be collected to train the LGPR model which aiming to estimate the pilot's motion of next gait cycle.Moreover,this kind of method also can not model the gait for a particular pilot.Therefore,a Hierarchical Interactive Learning(HIL)strategy is proposed to learn the gait model of pilot and controllers simutanously.In the porposed HIL stategy,two learning hierarchies are set: high-level motion learning hierarchy and low-level controller learning hierarchy.In motion learning hierarchy,Dynamic Movement Primitives(DMP)are utilized to model the gait of the pilot,and Locally Weighted Regression(LWR)method is employed to update parameters of the gait model whenever the pilot change his/her gait pattern.The control learning hierarchy is designed as the same in ASAC strategy,which aiming to learn controllers online to adapt different pilot and different pilot's gait pattens.3.In the original HIL strategy,DMP needs a whole gait cycle to update a new gait model if the pilot change his/her motion patterns,which leads when the pilot change his/her motion pattern frequently,original DMP can not handle the problem.Therefore,a Coupled Cooperative Primitive(CCP)model is proposed to model the gait of the pilot dynamically,which can adapat the pilot's motion trajectories through physical HumanRobot Interaction(pHRI)between the pilot and the exoskeleton system.For adapting different pilots,a policy gradient reinforcement learning method is employed to learning the coupled parameters of CCP models.4.In order to validate the proposed learning-based control strategy,a Humanpowered Augmentation Lower Exoskeleton(HUALEX)system is built with hydraulic actuators.The HUALEX system has three kinds of subsystems: mechanical and hydraulic system,sensory system and control system.The mechanical system is described as mechanical structure of the HUALEX system,with combination of hydraulic system.The sensory system is design to collect information of sensors,include encoders,IMUs and force sensors.Then,the control system using these sensory information to control the HUALEX system to follow the pilot's motion.