Action Recognition Of Both Arms And Its Application In The Control Of A Biped Climbing Robot

The Human-Robot Interaction system requires the instruction input equipment to be cheap,portable,accurate and easy to use.It is suitable for many indoor and outdoor scenes,such as bedroom,hospital,classroom,square and forest.Human natural interaction mainly relies on speech and action to convey information.In recent years,there has been extensive research on language-based interaction scheme,but there is little research on action recognition based interaction scheme.The existing equipment,such as Qualisys and Xsens,which can acquire manual action with high precision,is expensive and complex in use,configuration and maintenance.However,cheap and easy to operate equipment,such as Kinect,will be difficult to track the movement correctly because of the hand position occlusion and other factors.Under the above background,a dual arm motion tracking and motion recognition system based on depth camera and inertial measurement unit multi-sensor information fusion is developed,and some common problems in this kind of system are discussed.Finally,the system is applied to the high-altitude climbing task of the biped climbing robot,which helps the robot to carry out the task more efficiently and safely.The main work and innovation of this paper are as follows.When the position of the depth camera and the inertial measurement unit is time-varying,it is difficult to calibrate the relative attitude between the sensors.There is no public report to solve this problem.In this paper,an innovative method is proposed to solve the problem.The model is constructed by using the principle of rigid body transformation invariance,and the Singular Value Decomposition and Least Square Method are used to solve the problem.Finally,a reference standard for error evaluation is given.Experiments show that the precision of calibration is within ± 40,and the effect of hand motion tracking after calibration is obviously better than that of uncalibrated results.When the angle of view of the depth camera is occluded,the wrong position data of both hands will be obtained from the camera,which will lead to the failure of motion tracking and seriously affect the subsequent action recognition.Based on the correlation of the motion data between the depth camera and the inertial measurement unit,a new feature descriptor of the occlusion state of the hand is constructed,which is combined with the Support Vector Machine to recognize the occlusion.Finally,the Kalman Filter is used to deal with the error data by sensor information fusion.The experimental results show that the average accuracy of occlusion recognition is as high as 92%,the average recall rate is up to 89%and the position of the hand after occlusion can be tracked.When the position and pose of the operator relative to the depth camera change,the original data often change.In order to solve this problem,an action feature vector construction method based on the human body is proposed.Gaussian Mixture Model is used as the method of action modeling and recognition.The biped climbing robot has the ability to perceive environment and plan the path,and can move ahead independently by alternately changing the fixed end on the wall surface or pole.However,due to its weak ability of environmental perception,it is often unable to efficiently and safely carry out the task when it is blocked by obstacles or interfered by strong light in the process of climbing at high altitude.In order to solve this problem,a semi-autonomous human-robot interaction method is proposed in this paper,which combines the strong environment perception of human beings with the autonomous motion ability of the robot.Three kinds of action instructions corresponding to the basic climbing gait of the robot are designed for the application scene,and the above motion recognition algorithms are used to process them.The experimental results show that the correct rate of motion recognition is 93.3%.