Development Of Multi-Scenario Product Image Synthesis And Style Transfer Algorithm

In the task of product detection and identification,collection training images and labeling are much time-consuming and labor-intensive.The main challenge of this problem comes from the large-scale,fine-grained nature and continuous update of the products as well as the interference caused by various uncertain environmental factors such as shooting angles,lighting,and occlusion.To fill this gap,this thesis develops a multi-scenario product image synthesis and style transfer algorithm.It can generate the training set required by the detection algorithm,reducing the burden of manual labeling,which has good engineering application value.Firstly,this thesis develop a syntheticing multi-scenario product image algorithm,which consists of two modules:single-product image segmentation and multi-product image synthesis.The single-product image segmentation algorithm based on semantic segmentation adopts the state-of-art segmentation network,DeepLabv3+,as a basic framework.This thesis uses ResNet 101 as the backbone,and make improvemets for Atrous Spatial Pyramid Pooling.The improved DeepLabv3+can achieve 96.77%mIoU on the RPC segmentation dataset.Based on the single-product image segmentation,a multi-product image synthesis algorithm for checkout and shelf scenarios has been developed.Next,the synthesis method for random placement of product,boudingboxes generation strategy and blending method have been optimized,which can achieve the average speed of synthetizing a multi-product image per two seconds on PC.Large domain gap exists between the synthetic images and the real images.In order to reduce domain gap,an improved CycleGAN is apopted to perform style transfer on the synthetic images.First,the upsampling method of the generator network is adapted to avoid the checkerboard artifacts caused by deconvolution.Second,two different types of generator network are designed.In the shelf scenario,the identity mapping loss is adapted and a weakly supervised training method is proposed to make the training process more stable and the rendered images closer to the real images.Finally,the validity of the synthetic images is verified by using product detection experiments.In the checkout scenario,using FPN as the detector,training on the synthetic images and rendered images,cAcc can reach 16.87%and 47.05%respectively,which both exceeding the baseline.In the shelf scenario,an improved R-FCN is adopted as the detector.By mixing a small number of real images in the rendered images for training,mAP50 and mAP70 can reach 94.45%and 82.98%respectively,and achieve the same effect as the real samples,which can effectively reduce the manual labeling workload.