Action Recognition For Programming By Demonstration In Industrial Assembly

With the development of robotics,the applications of industrial robot have greatly expanded and have begun from the traditional manufacturing industry,such as auto-mobile manufacturing and machining,to hardware,3C,furniture and other emerging manufacturing industry.Considering that there are many small part assembly require-ments in the emerging manufacturing areas,which have the characteristics of multi-species,small quantities and short cycle,the demand of emerging manufacturing in-dustry requires the industrial robot to be more easy-of-use.Programming by Demon-stration(PBD),as an important way to simplify robot programming and improve the usability of robots,has re-drawn the focus of community in recent years.In this thesis,the action recognition problem in PBD for industrial assembly is studied.As the complexity of assembly action recognition,the features which are used to describe assembly action are extracted from assembled object and human hand in the video and continuous assembly action recognition is achieved based on them.The main contributions are as follows:1.An improved compressive tracking algorithm is proposed.In order to solve the problem that the target size and the learning rate are fixed in the original com-pressive tracking,the low-dimensional Haar-Like features are extracted from the image pyramid which includes different scale images using the sparse mea-surement matrix with channel tag.Besides,the normalized color histogram is introduced to describe the target,and the Bhattacharyya distance is calculated based on the histogram of the current tracking result and the tracking target.The distance is used to determine whether the current tracking target is occluded,and thus adaptively change the learning rate parameter.Experimental results show that the proposed object tracking algorithm can deal with the effects of small target tracking,scale change and occlusion.2.A hierarchical model is proposed to recognize the human assembly action by multi-feature fusion.In the underlying model,the assembly gesture is recog-nized based on improved HOG feature and Zernike moment feature with rotation-invariant properties.The type of assembly height change is obtained by analyzing the variation of the trajectory's Z component using the linear regression,and the plane motion trajectories of the assembled object and the hand are used to extract the direction histogram proposed in this paper.The high-level model takes the gesture,the height change type and the direction histogram as the features,and realizes the assembly action recognition by Multi-class SVM classifier.In the experiment,the accuracy of video recognition with 5 kinds of assembly actions including place,labeling,push,screw and take is up to 98%.Compared with the other action recognition methods,the proposed method has higher accuracy and better performance for PBD system.3.A method of continuous action segmentation and recognition is proposed in this paper,which combines the sliding window method and the iterative dynamic programming method.Considering the coupling of assembly action segmenta-tion and assembly action recognition,firstly the sliding window method is used to detect segmentation point based on action feature description of multi-feature fusion.Then the method of iterative dynamic programming maximizes the sum of assembly action recognition confidence,while optimizing the number and the position of split points.The experimental results show that the proposed algorith-m is effective and practicable for continuous action recognition and the assembly PBD system.