Clothing Parsing Based On Dual Context And Inter-Class Correlation

Clothing is not only the basic need of people's daily life,but also an important way for people to express their personality.In recent years,with the rise of various online shopping platforms and the continuous development of deep learning technology,emerging applications such as clothing recommendation,matching,classification and virtual try-on system have emerged in the clothing field.As the key technology to realize these applications,clothing parsing can divide clothing images into multiple parts and assign corresponding labels to each pixel,thereby providing accurate semantic segmentation results for subsequent clothing feature extraction,classification,and recommendation.Due to the complex patterns,textures and styles of clothing images and existence of complex backgrounds,changeable human poses,and nonrigidity,clothing parsing faces huge challenges.This paper proposes an improved clothing parsing algorithm based on dual context and clothing combination information to solve the problems that the same clothing is divided into multiple non-adjacent regions and the scale difference between clothing is large in the current clothing parsing algorithm.The research contents mainly include:(1)Aiming at the close correlation between the features of different regions in the fixed style clothing in the clothing image,an Attention Class feature Module(ACFM)is proposed to capture the class-level context information of the image.ACFM calculates the average feature of each class in the image by a coarse to fine segmentation structure and compares the consistency of each pixel's feature with the average feature of each class.In addition,Pixel Correlation Module(PCM)is used to capture the global context information of the image.PCM calculates the similarity between any two pixels in the image based on self-attention mechanism and uses the pixel similarity to improve the output of ACFM,thus obtaining an enhanced feature map that aggregates class-level context and global context to effectively correct misclassification.(2)According to the combination characteristics of clothing,an improved clothing parsing method based on inter-class correlation is proposed.The correlation between each two types of clothing is used as clothing combination information.Its key is to calculate the corresponding inter-class correlation matrix of current input image to represent the clothing combination information,which is based on Gram matrix.And we use the information of this matrix to continuously update the Inter-class Correlation Module(ICMM)through the moving average strategy in the training phase,so as to obtain the clothing combination information at the dataset level,and realize the automatic filtering of unreasonable clothing combinations.(3)In view of the large difference in scale of some objects in clothing images,the above clothing parsing method is further improved based on multi-scale aggregation.To obtain a feature map that aggregates multi-scale information,multiple convolution kernels of different sizes and global average pooling are used to extract the information of objects of different scales on the basis of preliminarily aggregating dataset-level clothing combination information,which effectively improves the segmentation effect of small objects.In this paper,validation experiments are conducted on the CFPD dataset and CCP dataset,and the results show that the parsing accuracy of each dataset is improved over SOTA(state-ofthe-art)algorithms such as PSPNet,DANet and Deep Lab V3.The Res Net for clothing parsing,which is based on aggregation of class-level context and global context,achieves 93.03% PA and50.95% m Io U on the CFPD dataset.And by aggregating clothing combination information in a multi-scale manner,the accuracy of the CFPD dataset is further improved to 93.15% PA and51.24% m Io U.The experiments demonstrate that the method proposed in this paper can segment the clothing objects of different scales more accurately and can exclude some unreasonable clothing combinations,which has good practical application value.