Research Of Human Action Transformation Based On Pose

The trend of short videos provides a wide range of application scenarios for personalized video creation,and people-oriented video creation is even more sought after.The human action transformation,given a source human image and a target action image,the person in the source image can complete the action in the target image.The core is to generate a real human body,including matching with the target posture,real texture,complete body,natural shape,reasonable shadow,etc.Based on the existing problems and challenges,this paper conducts corresponding research.First of all,human keypoint heatmap guidance cannot express the clear topological connection of the limbs and lead to the confusion of the connection,thus this paper proposes the Identity Consistency Guiding Block(ICGB).This module maintains the topological relationship of the limbs by constructing the body topology map during transformation and progressively completes the task by learning the transformation field between the source action and the target action.Quantitative and qualitative experiment results prove that ICGB can generate limbs with correct connection relationship and make more consistent with the target action.Secondly,background reconstruction occupies part of the generator’s generating capacity.This paper proposes the Object Supervision Mechanism(OSM).This mechanism decouples background reconstruction and action transformation by supervising the target area,improve the texture authenticity and limb integrity.Furthermore,a reasonable shadow can be generated.This paper not only proves that the background reconstruction affects the generation preference of the convolution kernel,but also constructs a comparative experiment to prove the effectiveness of the OSM mechanism.Finally,a progressive topological pose transformation network is proposed to generate a more realistic human body.A large number of comparative experiments have been constructed to prove the effectiveness and advancement of the network.Interpretability is provided by the visualization experiment of the transformation process.Quantitative and qualitative experiment results on DeepFashion,iPER,and FSDV demonstrate that our method has achieved the current best results.