Research On Human Pose Transfer Generation And Application On Person Re-identification

Human Pose Transfer(HPT)aims to transfer a person image with arbitrary poses while retaining the appearance details of the source image.It has shown great potential applications in many tasks,such as video generation,virtual clothes try-on,data augmentation for person re-identification,etc.Limited by non-rigid human body deformation structure,HPT can be exceptionally challenging particularly when only given local observation of the human body.It causes difficulties in directly transforming the spatially misaligned body.Motivated by the aforementioned discussion,this thesis profoundly studies on HPT and propose corresponding methods.Moreover,due to the importance of person re-identification(Re-ID)in the security industry,we also study the application of the human pose transfer in Re-ID to further improve the accuracy and generality of the Re-ID model.The main contributions of the thesis can be summarized as follows:(1)To address the background clutters which prevents the network from learning a robust HPT model,we introduce the binary segmentation mask to construct the body region served as the input of the generator,then design a segmentation mask-guided person image generation network(SMPIG).The binary segmentation mask has the capability of removing the background clutters in pixel-level,and contains more details about the edge information,where better shape consistency can be achieved for the generated image with the input image.In addition,we design a lightweight attention mechanism module as a guider module,which can assist the generator to focus on the discriminative features of pedestrians.The experiment results are introduced to demonstrate the effectiveness of the proposed method and the superiority performance over most state-of-the-art methods without over-computing in the design process of the Re-ID model.(2)To address the non-rigid human body deformations structure in captured under different locations or across different time,we propose a novel Semantic Parsing Attention Network(SPAN).SPAN is constructed with several Semantic Parsing Attention Blocks.Each block focuses on a local transfer of the human manifold,which can attend to put the sample condition patches to the corresponding location of the target image.The introduction of the binary segmentation mask and the semantic parsing map is not only significant for the seamless stitching of the foreground and the background but also decreases the computation load considerably.Compared with other methods,our network can characterize better body shape as well as keeping clothing attributes during the pose transfer.And our synthesized image can obtain better appearance and shape consistency related to the source image.Furthermore,extensive experiments are also conducted on person Re-ID systems trained with the augmented data,where our network has the ability to improve the person Re-ID accuracy.(3)Person image generation becomes a challenging problem due to the non-rigid spatial deformation.We propose a novel pose transfer generation with semantic parsing attention Network(SIN)to address this problem.Instead of directly learning the relationship between the source and target images,we decompose the process into two accessible modules,namely Semantic-guided Attention Network(SAN)and Pose guided Attention Network(PAN).SAN is proposed with introducing the semantic parsing to embed the human attributes into the latent space as the image code.In our PAN,an attention module is constructed to abstract the mapping between the corresponding regions of the image and pose,where each block can selectively integrate features to complete transformation of the human pose.Additionally,we design a semantic layout loss featuring the ability to improve the textures and styles of the generated images,where the appearance consistency between the source and generated image can be well guaranteed.Compared with other methods,our network can not only characterize better body shape,but also keep clothing attributes simultaneously.On the basis of SIN,our network can further achieve the data augmentation for person re-identification(Re-ID)with dramatically improving the person Re-ID accuracy.(4)Limited by data insufficiency and significant difference between poses,Re-ID module suffer from the influence of the invalid intra-class difference.To address the problem above,we propose an Adversarial Training-based Person Re-Identification with Eliminating Poserelated Information(ATPR).ATPR aims to learn identity-related and pose-unrelated representations for Re-ID module.The proposed method utilizes the HPT method to decouple the appearance and posture of the human,and the generated image is treated as the negative sample to improve the ability Re-ID module based on adversarial training.Additionally,a new hard sample loss is proposed to prevent the vast number of easy samples from overwhelming the identification model during training.It has ability to assist the Re-ID model to focus the hard negative sample.Compared with the other methods,our network has no auxiliary pose information and additional computational cost during the test.Experiments results that our network achieves state-of-the-art performance on two Re-ID datasets,and it can be combined with other high-level Re-ID and HPT models to further improve the Re-ID accuracy.