Research And Application Of Self-Supervised Part Segmentation Method Based On Mean Vision And Affine Transformation

Perceiving and analyzing objects at the part level by extracting part representations using part segmentation algorithms has received increasing attention.Compared with semantic segmentation,part segmentation faces more challenges due to the need for fine-grained data annotation and part-level framework design.There is less work on self-supervised part segmentation,and low data-dependent semantic segmentation work is difficult to transfer to this task.In this thesis,we improve and propose various self-supervised part segmentation algorithms to reduce the dependence on labeled data samples,and explore the application of the parts generated by the selfsupervised algorithms in downstream tasks.The main work and innovations of this thesis are as follows:(1)To address the problem that constraint-based work locates parts in feature space only,this thesis proposes a self-supervised part segmentation algorithm MVCPS based on mean visual constraint to find the visual consistency of the object itself in image space rather than feature space.The mean pixels within each part are first extracted by the part segmentation result,followed by filling the mean pixels by part to generate the reconstructed image,constraining the difference between the original image and the reconstructed image,and guiding the network to generate more accurate part segmentation results.Compared with the SCOPS algorithm,the overall MAE error is reduced by 6.4% on the CUB-200-2011 dataset.(2)To address the problem of locating parts using only a single static image,this thesis proposes a self-supervised part segmentation algorithm AFFPS based on affine transformation by combining object motion information.The method generates reconstructed images through part splitting,affine matrix acquisition and part independent affine transformation.Accurate part segmentation results are generated by reducing the discrepancy between the reconstructed image and the reference image.Compared with the current best-performing MSCS and MRAA algorithms,the MAE error is reduced by 3%-12% and the IOU metrics are improved by 5%-11% on the TaiChi-HD dataset.The MAE error is reduced by 6%-19% on the Vox-Celeb dataset.(3)Exploring the application of self-supervised parts in downstream tasks.For the fine-grained recognition task,a part-based Part Mix data enhancement algorithm is proposed which improves the fine-grained recognition accuracy of Res Net-50 by 1.34 on the CUB-200-2011 dataset;For the person re-identification task,a part-based global-part-foreground three-feature branching person re-identification framework,Part Re ID is proposed,and the m AP of Market-1501 and Duke MTMC-re ID datasets reach 87.3 and 78.0,respectively.Compared with the basic scheme Reid-strong,the cross-domain re-identification m AP and Rank-1 on the pedestrian dataset HNU-re ID constructed by the research group are improved by 6.5-7.3 and 9.4-12.4 respectively;For the face editing task,a part-based face editing framework Partfake is designed,and a user-friendly demo system is developed.In this thesis,we propose the self-supervised part segmentation algorithm MVCPS based on average visual constraint and the self-supervised part segmentation algorithm AFFPS based on affine transformation to address the problems that existing work only locates parts in feature space and locates parts using only a single static image,and verify the effectiveness of the algorithms through practical applications in several downstream vision tasks,which provide references and ideas for the practical application of part segmentation technology.