Research And Design Of Performance Robot With Dress Based On Deep Learning

In recent years,with the huge success of deep learning in many fields,robot researchers tend to combine deep learning with robot research in order to give robots more capabilities of perceptions and analysis.More and more robots leave the laboratories and factories to go into people's daily lives.Among these robots,the performance robot can "autonomously" perform and interact with people for serving human.And the quality of performance service depends largely on the quality of the 3D bone driving sequence.However,the existing acquisition of 3D bone sequences mainly depends on the 3D human motion capture equipments or motion designers.These methods are costly,making the research of intelligent performance robots relatively difficult.What's more,most of the existing work about performance robots ignore the effects of clothing on robots.In order to solve the problems above,this thesis researches and designs the dressing categories and acquisition methods of 3D bone sequence of performing robots based on deep learning.The research work in this thesis is divided into the following three parts:(1)Clothing detection and classification for performance robots: This section mainly detects and classifies the clothing of the presenters in the scene,so that the robot can choose the corresponding clothing type according to the detection results.And then this thesis' target tracking the clothing on presenters in videos,which is helpful for human pose estimation later.This thesis made a dataset for the clothing detection task,and then improved the target detection network YOLO v3 according to the application scenario and the characteristics of the dataset.In the later work,this thesis changed the multi-scale process,introduced down sampling and pruned the net so that the improved network will be used in this thesis' task with a better performance.(2)3D bone sequence acquisition: This part consists of two stages.The 2D pose of the human body is estimated from the scene videos,and then the 3D human bone sequence is obtained based on the 2D pose through inference.In the first stage,this thesis used a top-down approach.First,expand the clothing positioning information of the clothing detection part into the positioning information of human body detection.Then,the spatial transformation network is used to affine the detection part so that the center of the pose of the target human body coincides with the center of the image.In addition,HRnet the current best single-person pose estimation network was used to estimate 2D human poses.In the second stage,this thesis deduced 3D skeleton sequence from 2D human pose by a temporal convolutional network.In order to solve the problem of insufficient 3D human pose dataset,a semi-supervised training method is introduced in this thesis.In a result,a high-precision,long-term,and stable 3D human skeleton sequence can be extracted.(3)The verification of action imitation and costume imitation in the robot system of animation:This section combines the research results of the first two parts and applies these to the performance robot.First,this thesis use Marvelous Designer to make clothing and put them into the clothing base for the performance robot.Then,select the clothing for the performing robot based on the clothing category from the clothing detection section.In addition,3D human bone sequence was extracted by the human pose estimation part,and the bone sequence was transformed into a Euler angle sequence that can be read by the performance robot,and passed to the robot in a certain data format.Eventually,the same-type clothing replacement and action simulation of the performing robot were realized.A large number of experiments have proved the effectiveness of this thesis' methods,and the superiority of this method is demonstrated by comparison with other methods.