Human Joint Multi-view Fusion And Human Pose Estimation

The understanding of human activities is one of the active research fields,and has broad application prospects in the fields of intelligent security monitoring,elderly care,patient rehabilitation training,human-computer security interaction and somatosensory entertainment.With the advent of depth sensors,the field of three-dimensional human joint estimation algorithms has been greatly developed,and more accurate joint estimates can be obtained.However,a single sensor has the disadvantage of a visual blind zone,and the sensor network can improve this problem.Under the sensor network,there are generally two kinds of estimation algorithms:centralized and distributed.Distributed algorithms have the advantages of strong fault tolerance and scalability compared to centralized algorithms.Therefore,we use a distributed estimation algorithm to track joints of the human body under the sensor network.However,the human perception algorithm of depth sensors has limitations:when the human body is facing away from the sensor,the human joint estimation is not accurate.Instead we can combine two-dimensional pose estimation obtained from RGB image and three-dimensional depth information to get three-dimensional pose.Therefore,we studied the pose estimation of single and multi-person human bodies.For the occlusion problem of human joints resulted from the single angle of view of camera,this paper proposes to use information weight consensus filter to track each joint point of the human body under the camera network.The distributed estimation algorithm includes average consensus,distributed Kalman consensus filter and information weight consensus filter.We mainly use distributed Kalman consensus filter and information weighted consensus filter for estimating human joints.Due to the stable property of the human torso,it is used as a reference point for camera calibration,and the joint points obtained by different cameras are converted to the same coordinate system.The joint motion dynamic model is modeled separately and the camera observation model is modeled.The distributed Kalman consensus filter and information weight consensus filter are applied to joint motion estimation,and their performances are compared.Experimental results show that the information weight consensus filter has better performance than distributed Kalman consensus filter.To achieve higher accuracy of single person pose estimation,this paper studied single person pose estimation based on the stacked hourglass network.The stacked hourglass network is an advanced single human pose estimation algorithm.Its basic unit is the residual network.We have designed a new dense concatenated convolution module to replace its basic unit.Finally,experiments were carried out on two human pose estimation data sets,and a comprehensive comparison was made,which shows that the proposed method can improve the accuracy,reduce the model parameters and computational complexity,and slightly increasing the running time.In addition,a parametric exploration of the newly designed dense convolution module was carried out.Currently,multi-person pose estimation can hardly run in real time because of huge network model.So this paper studies object detection based the multi-person pose estimation that can run in real time.First,two bottom-up multi-person pose estimation algorithms are introduced.Both methods introduce an association mechanism between certain joints.Both labels are two sets of heat maps,one for the joint detection and one for the joint correspondence.However,their network mechanisms are extremely complicated and the inference time is long.Then,we reproduce the multi-person estimation algorithm based on object detection.It is inspired by the object detection algorithm,and the human posture problem is formulated as the object detection problem.It includes part detection and the limb detection,and then the previous two detection are combined and analyzed.Experimental results show that the algorithm can run in real time.This work effectively solves the problem of single-person,multi-person pose estimation and occlusion problem of single view.This work has important theoretical and practical value for motion recognition and human-computer interaction based on the estimation of human joint position.