Human Motion Capture And Action Recognition Based On Wearable Sensors

Human motion capture and action recognition has broad market and application prospects,and has been widely used in cross-disciplinary fields such as film and animation production,human-computer interaction,virtual reality,sports training and medical rehabilitation.Compared with the optical human motion capture system,the wearable sensor based human motion capture system has many advantages such as low cost,simple operation,free from space limitation and occlusion problems.In this dissertation,the wearable sensors are used to build a low-cost,high-precision,easy-to-wear,easy-to-operate human motion capture system,and the motion capture data is used for human action recognition.The main contributions and results of this dissertation can be summarized as follows:The human kinematics model is the basis for human motion capture,reconstruction and analysis.In this dissertation,the human body is simplified as a bone-joint model composed of several joints and bones.With the constraints of rotation degrees of freedom of each joint and the limitation of joint rotation angles,we have established a constraint-based hierarchical joint chain skeleton model.According to the structure of the human skeleton chain model,the human skeleton model can be divided into five movement branches.Combined with the forward kinematics of robot and the homogeneous coordinate transformation,the human joint rotation model and the human skeleton posture model are established.High-precision,low-latency limb attitude and position measurements are the key to human motion capture and reconstruction.For orientation estimation,the nine-axis MEMS inertial sensor fusion scheme of ”accelerometer + gyroscope + magnetometer”is utilized.The proposed quaternion-based indirect Kalman filter is proposed to fuse the measurement of nine-axis inertial sensors.In order to reduce the influence of inertial sensor measurement error,the inertial sensor is first calibrated and compensated.The quaternion-based indirect Kalman filter can be divided into two layers.The inner layer uses the integral of angular velocity measured by gyroscope to calculate the prior quaternion estimation.The outer layer is an error quaternion based Extended Kalman Filter.By combining the prior quaternion estimation in the inner layer and the posterior estimation of the error quaternion in the outer layer,the posterior quaternion estimation canbe obtained.In addition,an adaptive covariance operator is introduced to reduce the interference of linear acceleration and magnetic disturbance.The process covariance of linear acceleration and the measurement noise covariance of magnetometer are adjusted according to the measured amplitude of accelerometer and magnetometer.In order to reduce the position measurement error,a UWB(Ultra Wide Band)positioning system is added to correct the position estimation.The UWB positioning system is first calibrated to reduce the influence of clock error,then the Gauss-Newton iterative algorithm is adopted to calculate the position.Finally,the Extended Kalman Filter and the Rauch-Tung-Striebel(RTS)smoothing algorithm are combined to estimate the position based on the measurement of the inertial sensors and UWB positioning system.The experimental results show that the proposed algorithm has high accuracy and stability.System initialization calibration is an essential procedure of wearable sensors based human motion capture system.Since the sensor measurement coordinate system and the body segment coordinate system do not coincide,a sensor-to-segment calibration method based on hand-eye calibration is proposed to calculate the relative rotation matrix.For the whole body calibration,the proposed calibration algorithm only needs three simple calibration postures.In order to suppress the shaking during the calibration process,an intrinsic average algorithm is presented to smooth the orientation measurements.In addition,the proposed calibration algorithm can provide calibration quality feedback.The user can be reminded to recalibrate when the calibration postures are not standard.For the human body parameters estimation,we propose a joint constraint based joint parameter estimation method and a closed joint chain based limb length estimation method.These two methods do not need any external equipments,and the human body parameters can be estimated with a few simple movements or postures.Finally,for algorithm verification,the nine-axis MEMS inertial sensors and UWB sensors are utilized to establish a wearable human motion capture system.The experimental results show that the proposed human motion capture system can accurately and smoothly capture and reconstruct the human action.Compared with the motion information such as acceleration and angular velocity,the attitude measurement based on multi-sensor fusion are quite accurate and contain abundant information,which can reconstruct the human motion trajectory more accurately.Therefore,the relative orientation sequence or joint rotation sequence is adopted to describe the human motion.In view of the variability of human action sequence in time,space and integrity,a DTW(Dynamic Time Warping)based template matching algorithm is utilized for human action recognition.Combined with the similarity threshold,the DTW algorithm and orientation metric are utilized to calculate the similarity percentage between the test action sequence and each action template sequence.For the DTW based template matching algorithm,the recognition rate mainly depends on the quality of the template.Therefore,a DTW-based adaptive global time sequence averaging algorithm is proposed to calculate the action template.With the DTW distance as the similarity metric,the proposed time sequence averaging algorithm can average the sequence set in amplitude space as well as in temporal space.By averaging the training action sequence set,an action sequence with moderate execution speed and amplitude can be obtained.Taking the average action sequence as the action template sequence,the influence of execution speed and range on action recognition can be reduced.In addition,a template pre-screening and sorting method and a non-optimal matching advance cutoff scheme are proposed to optimize and accelerate the recognition algorithm.Finally,the gesture recognition is adopted to verify the recognition algorithm.The experimental results show that with the average sequence of gesture training sequences as the gesture template,the recognition rate can be improved significantly.