Human Upper Limb Pose Estimation Based On Inertial And Visual Sensors

Human pose estimation is a technique which obtains the poses of segments and the positions of joints by various sensors.It has found a wide range of applications in varieties of areas such as human-machine collaboration,medical rehabilitation,sport training,etc.,and it has been one of the hot research topics in artificial intelligence now.Especially,it is because the upper limbs are the most flexible parts of the human body,it is meaningful to study the pose estimation starting with the upper limb pose estimation.At present,there are four kinds of human body pose estimation methods,that is the mechanical-based method,the electromagnetic-based method,the inertial-base method,and the visual-based method.The inertial sensor-based methods and the visual sensor-based methods are the most commonly used methods.However,three problems in the human pose estimation that still need to be investigated: 1)further reduce the computational complexity of the pose estimation method,and enhance the real-time performance for pose estimation system 2)a way to integrate multi-sensor information and improve the accuracy and robustness of the human pose estimation system 3)design a simple and easy multi-sensor coordinate calibration method.An upper limb pose estimation based on visual and inertial sensor fusion is investigated in this thesis to solve the aforementioned problems,and the main work and results are listed as follows.1.An upper arm pose estimation method with low-complexity and high-precision is proposed,which mainly consists of three aspects.Firstly,the Inverse Free Kalman Filter is introduced to reduce the complexity of matrix inversion operation in the existing human pose estimation method and real-time performance.Secondly,a dual strategy is proposed to further reduce the computation burden by dividing the human motion into two categories.Thirdly,to avoid anti-trigonometric operation in calculating the dip angle,an anti-trigonometric free criterion is presented to reduce the computation burden.2.An easy-to-use calibration method is designed to determine the relative rotation of the inertial sensor and the visual sensor coordinate.The multi-sensor coordinate calibration is necessary for multi-sensor fusion.Existing methods focus on the calibration of a rigidly connected camera and inertial sensor,and the problem of visual sensor and inertial sensor without connection is rarely considered.The ground coordinate and calibration plate coordinate are introduced as a medium to build the rotation between inertial coordinate and visual coordinate.Then,the rotations between coordinates are calculated by attitude determination and Zhang's camera calibration method.3.An event-trigged upper limb pose estimation method with multi-sensor fusion is designed.There exist some restrictions on human pose estimation with low-cost sensors.For example,Kinect suffers from occlusion and low sampling rate.Drift inevitably happens for position estimation with inertial sensors by double-integrating acceleration for a long time.The proposed fusion method combines visual measurements to revise the drift of position estimation.The measurement gaps of visual sensor caused by occlusion and fast motion of human are compensated by inertial measurements.Several experiments are conducted in the human pose estimation system based on inertial and visual senor to show the effectiveness of the proposed design methods.Finally,conclusion and future work are presented.