3D Human Motion Capture Based On Multi-view Camera Array

In recent years,with the continuous progress of camera hardware and deep learning technology,human motion capture technology has gradually become an important direction in the field of computer vision.Accurate 2D human skeleton,3D human skeleton and human model can be obtained through pictures or video data.It also has high research value and development prospects in the industrial field,such as virtual fitting,game and movie special effects,virtual reality games and sports-assisted training.However,in practical industrial applications,most actors wear special clothes with marking devices for human motion capture.With the help of equipment,very accurate human motion poses can be obtained.However,such equipment is expensive and cumbersome to use,and cannot be promoted.Therefore,the markerless motion capture system is very important for the wide application of human motion capture,but when the number of people in the scene increases,there will be a serious occlusion problem which has a serious impact on the accuracy of2 D human pose estimation and 3D human pose estimation.Therefore,markerless human motion capture is still a challenging problem.Under the above background,this paper focuses on the problem of markerless human motion capture technology for in-depth research,and aims to improve the effect of 2D human pose estimation,3D human pose estimation and human model reconstruction.The main work is as follows:Firstly,2D human pose estimation is performed based on a multi-view camera array.A multi-view camera array which consists of 20 industrial cameras which are calibrated is built and used as a capture device for multi-person strong interactive motion picture data.The human joint point feature extraction network is used to extract the initial features and joint position heatmap from the collected multi-view image data.The 2D human pose estimation techniques are then improved based on multi-view feature fusion and heatmap fusion to enhance detection results.It can make full use of the advantages of multi-view data and reduce the impact of the unavoidable occlusion problem of single view on the detection effect of human joint points.For the occluded joint points,the problems of undetectable and incorrect matching have been significantly improved,and the method is more robust.Secondly,3D human pose estimation is performed based on a multi-view camera array.The human motion data is collected in the scene through a multi-view camera array.The 2D human pose estimation is performed on the input image,and cross-view matching is performed on the 2D human joint points of each view.The accuracy of cross-view matching of human joints can be improved by using the consistency of human body regions and the consistency of the cross-view 2D human joint points and the consistency of single-view matching relationship of 2D human joints and the spatial consistency between 2D human joints and 3D human joints.The optimal 3D human joint point position is obtained by using the least squares estimation method for the 2D human body joint points after optimized matching,and the motion relationship between the human skeleton of sequential frames is used to ensure the consistency of the human skeleton in terms of time series,and the tracking can be maintained on the same human skeleton.Finally,the human model reconstruction is performed based on the multi-view camera array.The human body data in the scene is collected by the multi-view camera array,and then the 3D human pose estimation is performed on the input image to obtain the position information of the human body joint points.The rotation matrix is used to obtain the human pose parameters of the model.The parameters of the human model are optimized by minimizing the objective function containing multiple error terms,to reduce the influence of the errors of human joint points and obtain a better SMPL human body model.The human motion capture technology in this paper is based on a multi-view camera array,making full use of the more comprehensive human body information from multi-view data,and has achieved good results in 2D human pose estimation,3D human pose estimation and human model reconstruction.