Manipulation Activity Recognition For Service Robot Learning By Demonstration

Facing with complex and diverse manipulation tasks,whether robots can learn complex work skills is the key factor that influence their autonomous intelligence development.Through Learning by Demonstration(LbD),it is possible for robots to analyze and understand the target-oriented activity made by the subjects and transform it into a robot-executable task sequence through a generalized process,which has become a new hot topic in the field of robot researches.In this dissertation,focusing on typical linkage manipulation activity,based on RGB-D sensor,the key technologies in activity recognition,description and missing details reasoning are systematically studied,and the robot action planning method of instance is explored,which lays a foundation for the further study of the generalization of the robot's reproduction activity.In this dissertation,the research status of the key technology and methods of manipulation activity for service robot learning by demonstration is summarized,and the research work and achievements are discussed in three aspects.Different from the traditional direct trajectory mapping method,this dissertation identifies and reproduces the demonstration manipulation activity from the task level,thus making the demonstration learning have a certain generalization ability.Firstly,aiming at the unknown manipulated objects,on the basis of objectness method which generating preliminary candidates,a fast object recognition method combining color histogram and image texture features is proposed,which is robust to illumination and significantly reduces computation complexity.Secondly,in order to discover human pose changes and the spatio-temporal characteristics generated within the human-objects interaction,a spatio-temporal conditional random field(CRF)model is constructed,the manipulation RGB-D video sequence segmentation is realized and object recognition results enriches object-related features.The parameters of the CRF model are solved by Structural SVM,the recognition of corresponding sub-activities and object affordances are recognized by quadratic pseudo-boolean optimization(QPBO)and overall activity is discriminated by k-th nearest neighbor(KNN)and dynamic time wrapping(DTW).The necessary details such as the opening and closing of the claw are difficult to be obtained only by observing human demonstration.For this purpose,based on the programming domain description language(PDDL),combining object,sub-activity and affordance recognition results,a description and inference method is proposed,the missing detailed information is reasoned and refined robot action planning sequence are generated.Based on the above research,a manipulation activity recognition experiment system for robot learning by demonstration is designed,and typical functions such as manipulated object recognition,manipulation activity segmentation and recognition,the missing sub-activity detail information inference and robot action planning sequence generation are developed.Based on single arm mobile robot,task reproduction simulation platform is built.Typical linkage tasks,such as assembly tasks,installation of cartridge,pouring water and pouring cereal,validate the functions of learning by demonstration,which verified the effectiveness of the method.