Research On The Learning Method Of Service Robot Arm Manipulation Skills Based On Behavior Tree And Reinforcement Learning

With the expanding usage of service robots in home environments,researchers in the field of robotic manipulation skills learning are focusing on how to make robots learn complicated manipulation abilities successfully.The thesis examines the complex manipulation skills of opening the door and placing the medicine bottle in the home environment by using behavior tree and reinforcement learning methods from two perspectives: imitation learning and reinforcement learning.The following are the main research findings of the thesis:(1)An experimental system and skill learning framework for service robotic arm operation has been established combining software and hardware.The eye-to-hand calibration work and visual recognition system between the Kinecct V2 camera and the Kinova Jaco2 robotic arm are completed by the operation experiment system,which connects the system software and hardware via ROS.The operation skill learning framework is mainly composed of an upper and lower hierarchy,with the lower level performing sub-skills characterization learning,and the upper layer learning the sub-skill organization sequence,and finally combining the software and hardware systems to achieve the reproduction and generalization of service robotic arm operation skills.(2)A complex manipulation skill learning method based on the Behavior Tree framework is proposed from the perspective of imitation learning.The learning of complex manipulation skills is divided into two levels using this strategy.The upper layer is task planning and behavior correction based on the Behavior Tree framework,while the lower layer is segmentation and learning representation of complicated manipulation skills.First,the BP-AR-HMM method is applied in the lower layer to segment the teaching data and obtain sub-skills;the DMP algorithm is then used to learn and represent the sub-skills obtained from the segmentation,and the manipulation primitive library is established.The upper layer then creates a task planning behavior tree based on the task execution logic,with the action node being the operation primitive obtained from the low-level DMP representation,the condition node being set according to the task,and a behavior correction module being added to solve the dumping of the medicine bottle situation.The proposed method may successfully reproduce and generalize the manipulation skill to open a door and place a medicine bottle,according to experimental results.The proposed method’s efficiency and applicability are confirmed.(3)A complex manipulation skill learning method based on hierarchical reinforcement learning is proposed from the perspective of reinforcement learning.This method proposes a subgoal-based hierarchical reinforcement learning method based on the results of automatic task segmentation by the BP-AR-HMM algorithm to solve the problems of traditional reinforcement learning,which is difficult to train the entire complex manipulation skills,sparse rewards,and large state space.There are two hierarchical structures in the method: low-level and high-level.The lowlevel learns each sub-task strategy and its representation using the SAC algorithm;the high-level learns the meta-strategy for planning the complete complex task using the maximum entropy objective algorithm.The experimental results show that,compared with other algorithms,the proposed method shows better results in policy convergence and learning performance.Finally,through virtual-real conversion,the high-level meta-strategy is deployed into the real world,and operation skills reproduction and generalization experiments are conducted.The experimental results verify the feasibility and generalization of the proposed method.