Researches On Large Scale Product Image Classification Based On Deep Learning

With the rapid development of e-Commerce platform,the number of product images is increasing explosively.The automatic classification of product images has therefore become an important topic.In recent years,great progress has been made in the research of large-scale visual classification based on convolutional neural network(CNN),in which the flat one-vs-all structure is generally used,but this kind of structure is not optimal for product image classification.A product usually has multiple category labels at different levels on e-Commerce platforms(for example,shoes and sneakers).At the same time,the product also contains some descriptive attributes such as style,color,brand,and so on.The above two types of information can obviously be used to assist in classification task.However,it is rare to use these information to explore product classification methods.In addition,it is not uncommon for goods on e-Commerce platforms to be labeled incorrectly.It is also a topic with definite application value to correct these intentional or unintentional wrong tagging categories by visual analysis for the platform.Based on the above understanding,this thesis studies the classification method of large-scale product images under the hierarchical category label and attribute description of product categories,and proposes two solutions based on deep learning.At the same time,a category-based error correction method based on visual analysis is proposed for the error of category labeling.The details are as follows.First,a classification method ssCNN-MCR based on spatial salience learning and multi-class regression is proposed.In terms of spatial salience learning,by inserting a spatial saliency module in CNN for spatial weight learning,the importance of product foreground regions in classification can be highlighted.For multi-class regression,first-ly,a deep classification model is constructed based on different levels of category labels.Then,multi-class regression methods are used to fuse the classification results of these deep models,and the correlation between them is explored.Experiments on the Taobao dataset have shown that the integration of spatial salience learning and multi-class regres-sion has indeed improved the performance of large-scale product image classification.Second,this thesis proposes an end-to-end product image classification method ssMTL-CA based on multi-task learning.This method regards product classification and attribute prediction as different subtasks which can be mutually enhanced in the same network.By designing a reasonable network structure and loss function,product image classification that integrates attribute information can be achieved.In contrast to the ssCNN-MCR classification method achieved by two-stage learning described above,the ssMTL-CA can incorporate more information to help classification and realize end-to-end modeling learning.Experiments on the Taobao dataset show that the method based on multi-task significantly outperforms the single-task approach in classification performance,espe-cially when the attribute information is relatively complete.Especially,the ssMTL-CA method built on fine-grained product categories and descriptive attributes achieves the best performance.Finally,an error correction method for product image classification based on deep visual classification model is proposed.First,through the data analysis,the categories that are easily confused or mislabeled are identified.Then,by extracting the score vec-tor distribution of products in the deep classification model and integrating the confusion relationship matrix between the categories and the preset category label information of products,this thesis puts forward three kinds of error correction methods,aiming to as-sign the mislabeled images with higher confidence score as much as possible to make them easier to be found.Experiments show that the error correction model incorporating multiple types of information significantly surpasses other methods in error correction precision and retrieval efficiency.