3D Shape And Pose Estimation For Closely Interacting Persons

Markerless human motion capture is a popular and challenging topic in computer vision and computer graphics.Its main task is to restore the temporal coherent representation of dynamic 3D shapes by tracking the motion of moving objects in the video.Unmarked single-player motion capture has grown tremendously in the last decade,but existing methods require careful adjustment of the camera and console,and rely heavily on segmentation techniques.In the case of many people,the application of the existing single-person motion capture technology to a multi-person situation directly cannot produce satisfactory results due to the di culty of multi-person segmentation and pose estimation.Although some methods can handle multiplayer situations,the captured scenes are very limited and limited to very simple interactions without occlusion,such as playing basketball face to face.However,the interaction between many people in reality is usually very close,for example,hugs,double dances or boxing,etc.,which are also very common in games and movie scenes.Therefore,reconstructing the shape and pose of closely interacting people is very important for existing practical applications.To the best of our knowledge,the existing methods are unable to fully and automatically estimate the 3D shape and posture of a multi-person human body under close interaction.More importantly,because of the huge ambiguity,it is not possible to assign commonly used features(such as colors,edges,or key points)to everyone.When people interact closely,problems become more complex and challenging due to occlusion,truncation,and inherent ambiguity.In this paper,we propose a new markerless human motion capture method that automatically realizes the 3D pose and shape estimation of the human body from a multi-view interactive multi-view screen.The main contents and contributes of this thesis are as follows:1.Fully-automatic 3D pose and shape estimation for close interacting persons.Our method do not need manual intervention,template mesh scan and segmentation for each person.It has more flexibility and less computation time(about 1min per person per time instance without GPU acceleration).2.Multi-person spatio-temporal pose tracking.The same person in multiple perspectives is inferred by considering the correspondence between the spatio-temporal.Our tracking strategy uses bounding box and pose information,which helps reduce the singularity of single-person information,making estimates more accurate and effective.3.Multi-view 3D pose and shape estimation.The 2D human keypoints detection is performed on the human body under multiple views,and then the detected key points are used to perform the pose and shape fitting of the human body based on the multi-view constraint.Multi-view pose and shape estimation solve the pose and shape parameters in an optimized way.Based on multi-step solution optimization,the optimization results are more robust and reasonable,and the local optimal situation is avoided.