Research On Instance Segmentation Of Clothing Image Based On Deep Learning

With the rapid development of e-commerce industry and the upgrading of people’s consumption patterns in recent years,images have become one of the main carriers for people to obtain clothing information.Therefore,the analysis and processing of clothing image with the help of computer technology has become an important research content in the field of digital development of clothing.Image segmentation separate the clothing region from the image,which is convenient for the subsequent analysis and processing of clothing,have become one of the basic research direction of clothing image processing.At present,the research on image segmentation focuses on the deep learning algorithm,especially the convolutional neural network（CNN）,such as Deep Lab,Mask RCNN and its improved model.Although these CNN models have been greatly improved compared with traditional algorithms,there are still two shortcomings:（1）the coincidence degree of clothing regions is not ideal;（2）the adaptability to the deformation of clothing edges is weak.In order to improve the performance of clothing segmentation,this paper attempts to optimize the above two points,and proposed a new algorithm for clothing image instance segmentation.The algorithm predicts the category of each pixel in the clothing image,so as to extract each clothing object from the image.The new algorithm combines the semantic segmentation network with the object detection network,and finally realizes the instance segmentation of clothing images.The neural network model has been improved in this paper in order to improve the performance of clothing image segmentation.The algorithm scheme is designed and the dataset is established,which provides the basis for the following main research contents.The research work was divided into the following two steps: the identification and segmentation of clothing region;the location and separation of clothing objects.The first step was to separate all the clothing pixels in the image from the complex background;the second step is to separate and extract the clothing object.Step 1: the identification and segmentation of clothing regionIn this paper,semantic segmentation is used to predict the categories of each pixels in the image,so as to realize the pixel-level fine segmentation of the clothing region.By improving the CNN model of Deep Lab V3+ with redesigning the receptive field module and the decoder,a new semantic segmentation model is obtained.The Atrous Spatial Pyramid Pooling（ASPP）in the Deep Lab V3+ model is replaced by an improved receptive field block.The transposed convolution is used to replace the interpolation upsampling in the decoder and the upsampling step is adjusts to 2.The Improved network achieved 97.26% pixel accuracy,93.23% mean intersection over union,and 90.56%,71.41% and 44.80% average precision when 0.75,0.90 and 0.95 is selected as the iou-threshold.Compared with Deep Lab V3+,the performance of clothing image segmentation has been greatly improved on each index.Step 2: the location and separation of clothing objectsIn this paper,two parts of work are carried out on the localization and separation of clothing objects:（1）object detection model is used to achieve the localization and classification of clothing;（2）machine learning is used to separate the pixels of adjacent boundaries between different clothing object,so as to achieve the pixel-level segmentation of adjacent boundaries between clothing.In The first part,the structure of different model is improved based on YOLOX and Faster RCNN.The spatial pyramid pooling in YOLOX is modified into the improved receptive field module,which improved the perception ability of clothing features,and the structure of Faster RCNN network and segmentation network is merged to reduce the computation redundancy by sharing the backbone.The object detection models are trained and tested,the 67.4% average precision and 77.1% average recall are obtained when 0.75 is set as the threshold of intersection over union in clothing image localization and classification,which have increased significantly.In the second part,with the help of the clothing region got from part one,the region containing adjacent boundaries of different clothing objects is extracted,and the clothing pixels in the region are classified and predicted by machine learning,so as to realize the separation of different clothing.In this paper,the spatial and color features of clothing pixels are constructed,and the unsupervised classification（clustering）and supervised classification（support vector machine,logistic regression）are used to classify different clothing single pixels.Using the clothing pixels with certain categories as training dataset,the support vector machine and logistic regression are trained,and the clothing pixels in the border area are classified and predicted to realize the separation of the clothing objects.With the comparison of supervised classification and unsupervised classification,the logistic regression based on supervised classification achieves higher separation accuracy with less time consumption.The improved semantic segmentation model and improved object detection model are combined and test on Deep Fashion2 dataset,the new algorithm obtains 76.83% of the mean intersection over union of pixel classification.which is significantly improved compared with the segmentation result of Mask RCNN（69.79%）.This shows that the overlap between the clothing region segmented by the algorithm in this paper and the actual region is higher.In addition,the experimental results show that the prediction results of previous algorithms are presented as smooth curves for the non-smooth and mutation clothing edge line,which cause the fitting degree is not high;However,the algorithm proposed in this paper can still achieve pixel-level recognition,and the processing algorithm for the edge line does not depend on the interpolation results of the surrounding pixels,and got a high degree of fitting with the real edge line,thus greatly improved the adaptability to the edge deformation of clothing.In summary,the new instance segmentation algorithm of clothing image proposed in this paper realized the pixel-level segmentation of different clothing units in the image.Compared with previous studies,the new algorithm effectively improves the coincidence degree of clothing image recognition region and the adaptability of edge deformation.The realization of the new algorithm reduces the restriction on the complexity of image information in the research work of clothing image,and lays a certain foundation for the work of clothing image processing under complex background and human posture.