Pose Trajectory Extraction And Body Novel-view Synthesis From Visual Content

Motivated by its extensive applications in human behavior understanding and scene analysis,human pose estimation has witnessed a significant boom in recent years.Mainstream research fields have advanced from pose estimation of a single pre-located person in individual images to multi-person pose estimation in an unconstrained video.Multiperson articulated pose tracking is an important while challenging problem.In this essay,going along the road of top-down approaches,we propose a decent and efficient pose tracker based on pose flows.First,we design an online optimization framework to build the association of cross-frame poses and form pose flows.Second,a novel pose flow suppression module is introduced to robustly reduce redundant flows and re-link temporal disjoint ones.This approach can achieve real-time pose tracking without loss of accuracy.To better understand human activities in visual contents,only human skeleton is far from enough,clothes texture and geometric details also play an indispensable role in human modeling.So we will also discuss how to predict detailed 3D human body and synthesize novel views from a single view.However,extrapolating novel views of bodies is much more difficult than rigid objects due to its large variations in pose,shape,and cloth.Directly copying pixels from the input view guided by appearance flow is unable to hallucinate the unseen areas.Deep generative models can synthesize pixels from scratch,but the results are usually smoothed and distorted.We present a two-stage pipeline that combines the capacities of these two techniques.Firstly,a 3D body model is fitted to the input view,and details are recovered by leveraging human silhouette.Secondly,a GAN-based network is trained to do inpainting on fragmentary novel views.Extensive experiments demonstrated that our approach could synthesize more photorealistic and detailed novel views than previous methods,even at very extreme viewpoints.