Research On Clothing Parsing Method In Natural Scene

Clothing parsing is a pixel-level classification task.It is a branch of semantic segmentation.However,clothing parsing is different from general semantic segmentation,it has its unique characteristics.Firstly,clothing parsing is further refinement of general semantic segmentation.Secondly,the clothed human body present a hierarchical structure in the clothing parsing.According to the two characteristics of clothing parsing,clothing parsing is regarded as a process from coarse to fine and simple to complex.However,in the process of clothing parsing from coarse to fine and from simple to complex,it faces the following difficulties:(1)The process of clothing parsing from coarse to fine involves multiple hierarchies of clothing semantic.(2)There are structural relationships between different hierarchies of clothing semantics.(3)The edge of clothing is affected by human body,showing irregular geometric deformation.(4)There are long-range dependencies between clothing semantics at the same hierarchy,which are complex and diverse and difficult to model directly.Aiming at the above difficulties,this paper has carried out the following work:(1)A Hierarchical Parsing Method of Clothing from Coarse to Fine(C2F).Firstly,this paper analyzes the clothing parsing problem from coarse to fine,and proposes a Clothing Hierarchical Parsing Decoder which can parse clothing from coarse to fine.Considering that the clothing parsing process from coarse to fine involves multiple hierarchies of clothing semantic,this paper provides hierarchical semantic prior information guidance for different hierarchies of Clothing Hierarchical Parsing Decoder.In view of the geometric deformation of clothing semantic edge,an Edge Detection Module is added to extract the edge semantic information.Finally,in order to establish the semantic dependency between hierarchies,a Semantic Embedding Module is proposed to embed high-hierarchy semantics into low-hierarchy semantics from coarse to fine.In addition,in order to obtain more informative feature pyramid,this paper proposed a Bidirectional Deformable Convolutional Feature Pyramid Network.This paper conducted extensive experiments on three challenging datasets,i.e.,Look into Person(LIP),Active Template Regression(ATR)and PASCAL-Person-Part.In LIP and PASCAL-Person-Part datasets,C2 F obtain 54.50% and 69.57% on Mean Intersection over Union(m Io U)respectively,and surpassing most previous methods.Experimental results demonstrate the viability and efficiency of the proposed C2 F.(2)A Hierarchical Decoding(HD)Transformer for Clothing Parsing.Firstly,considering that convolution operations are limited to local receptive fields,Convolutional Neural Network usually fail to establish long-range dependence among clothing semantics at the same hierarchy,a HD Transformer is proposed in this paper.HD Transformer not only establishes the long-range dependence among clothing semantics at the same hierarchy via self-attention operations,but also decodes the long-range dependent features in a hierarchical manner to realize clothing parsing from coarse to fine.In addition,considering the semantic dependence between hierarchies,a Clothing Masked Transformer Block is proposed to establish the semantic dependence between hierarchies by using Cross-Attention.This paper verifies the feasibility and effectiveness of HD Transformer on three challenging datasets,i.e.,LIP,ATR and PASCAL-Person-Part.In LIP and PASCAL-Person-Part datasets,HD Transformer obtain 57.58% and 73.52% on m Io U respectively.In ATR dataset,HD Transformer obtain 85.90% on F-1 score.Finally,this paper provides an indepth analysis of hierarchical semantic prior information on PASCAL-Person-Part dataset.