Research On Hand Gesture Interaction Technology For Home Service Robot

With the needs of aging society, public safety incident, education and medical care for serve robots arising, the demand for service robot grows rapidly. However, in order to walk into the home environments, service robots should communicate with people in a natural interactive way, which can meet the people’s habits. The goal of intelligent human-computer interaction is to build a natural and harmonious interactive environment. As one of human’s basic biological characteristics, hand gesture is natural and intuitive, which plays an important role in the human-home service robots interaction system. Therefore, hand gesture based human-home service robots interaction technology is an indispensable technology to achieve intelligent human-home service robots interaction.For the problems existing in gesture-based interaction between human and home service robots, this paper made an in-depth research of gesture detection, gesture feature extraction and gesture recognition in gesture-based interaction of home service robots mainly through saliency and depth information. The details are as follows:Firstly, a gesture detection approach based on RGB-D and saliency computation is proposed. The method integrates color based multi-scale global regional contrast, texture based multi-scale global regional contrast, object measurement and skin probability to construct multi-feature and multi-scale global regional contrast based saliency computation model, and then integrates depth information, 3D skeleton and 3D point cloud information with the previous saliency computation model to detect hand gesture. This method can reduce the negative influence of occlusion, illumination changes, uneven illuminations and shadows in home environment to some extent, which provides a new way for hand gesture detection in complex environment.Secondly, a multi-feature and multi-scale saliency driven bilateral filtering based hand gesture feature construction method is designed. The proposed approach designs a saliency driven bilateral filtering to better preserve hand gesture and smooth noises in the hand gesture region, and then proposes a hierarchy-based multi-feature fusion method, which fuses mixture model of probabilistc canonical correlation analysis(M-PCCA) and maximum margin dimension reduction(MMDR). Finally, based on saliency driven bilateral filtering and multi-feature fusion method, a multi-scale hand gesture feature is constructed and the validity of the proposed feature extraction algorithm is verified. The constructed feature can reduce the influence of background, illumination and noise on the subsequent gesture recognition, which lays an important foundation for the subsequent recognition.Thirdly, because there are problems existing in sparse representation based hand gesture recognition and saliency can effectively describe the characteristics of hand gesture, a static gesture recognition method based on saliency and histogram intersection kernel is proposed. Due to the ability of saliency to remove complex background, uniformly highlight object and effectively describe the characteristics of hand gesture, saliency is presented to describe hand gesture. To improve the performace of sparse representation classification, hand gesture dictionary construction algorithm based on learning is constructed, which improves the performance well. Considering the nonlinear characteristics of the data, sparse representation in the original training feature space may produce bad results, so histogram intersection kernel is employed for sparse representation and then used for hand gesture recognition. Lastly, the validity of the proposed method is verified.Then, a dynamic hand gesture recognition method based on RGB-D and motion context feature is proposed. In this paper, dynamic gestures are detected by the proposed hand gesture detection method based on RGB-D and saliency computation. Considering the dynamic and continuity between adjacent frames of dynamic hand gesture sequences, this paper proposes a feature which integrates motion context to represent dynamic hand gesture. In addition, Because of the performance of the existing HMM depending on the initial parameters of the model, this paper presents a new method for determining the state number and the state probability density function of model. Finally, the validity of the proposed method is verifiedFinally, based on the theoretical research of previous chapters, a design scheme of a specific human-home service robots gesture interaction system is presented. The gesture instruction set is defined. Software mainstream of interaction system is designed and development example of IN-RT development system is presented.