Research On Human Motion Capture Based On Deep Learning

The human motion capture technology is always one of research hotspots of Computer Vision and Computer Graphics,and is widely applied in robot,virtual reality,film and TV Animation,gait rehabilitation,kinematic analysis,etc.But the experimenters need to wear relevant motion sensor in most of the existing motion capture methods,and the expensive equipment has high requirements to the acquisition environment;Other methods drive human modeling and complete reconstruction of 3D motion based on the database.But they rely too heavily on the features of database,with low expandability.Hence,the method of combining the deep learning and binocular stereoscopic vision is proposed to capture of human motion.Firstly,the pictures shot by ZED camera are used as input sources,then deep learning Mask R-CNN is used as the basic framework to achieve the 2D pose detection of human action.Next,the mapping relation among human skeleton joint points obtained through binocular stereoscopic vision is researched,and the 3D information of human joint points is obtained through binocular camera calibration,parallax principle and joint point stereo matching.Finally,3D reconstruction of human motion is carried out by reference of linkage mode of human skeleton,SCAPE model,etc.The main work and innovations of this thesis are as follow:(1)The human two-dimensional joint points on images are detected with the deep learning algorithm,which avoids the inconvenience due to constraint of wearable device and restriction of environmental condition in the traditional motion capture method.Since deep learning algorithm needs the computer with high configuration and huge dataset shall be trained.A kind of Mask R-CNN deep learning algorithm based on transfer learning is proposed for the detection of human 2D joint point.Compared with traditional methods,it can accurately detect the goal 2D joint point based on small sample dataset,with high accuracy.(2)The deep learning algorithm and binocular stereoscopic vision are combined to reconstruct the human motion 3D skeleton,and the stereo matching and binocular parallax principle are utilized to recover the 3D information of human skeleton joint point on the basis of human 2D pose information.A kind of binocular partial matching algorithm based on characteristic constrain is proposed in allusion to the error existing in joint point information to improve key point matching algorithm,which improves the matching precision of joint feature point and provides the foundation for human 3D motion reconstruction.(3)The motion capture system integrating deep learning and binocular stereoscopic vision is analyzed and developed.The image and 3D joint point coordinates are obtained,2D joint points are detected,and binocular calibration and stereo matching are achieved.Finally,any human action is captured and achieves three-dimensional reconstruction based on the relevant technology of human motion modeling and through the linkage method of human skeleton.The experimental results and data analysis results show that 2D joint point are detected and 3D joint point information is withdrawn in accordance with 3D motion reconstruction requirements.Both the establishment results of motion capture system and3 D reconstruction results of human motion are ideal,with experimental error of about 3%,which confirms that the method in this paper owns certain feasibility and application prospect.