Fine-grained Image Classification With Click Prediction

Image classification serves as a core problem in computer vision community,and fine-grained image classification becomes a hot research topic in recent years.It aims at distinguishing images that have high visual similarities among different categories,so fine-grained image classification is more challenging than traditional image classification tasks.Traditional classification approach based on visual features is difficult to deal with the high visual similarity among subclasses,and it always brings in semantic gap.Therefore,many researchers propose to recognize images by user click data.Different from visual features,click data contain rich semantic information and are constructed by search logs without additional manual annotation.Owing to the advantages of click feature,we focus on fine-grained image classification method based on click data to improve classification accuracy.However,in the large scale query text in the original click data,the click features constructed from click count data are often with high dimension,sparsity,and noisy degree,resulting in unsatisfactory performance in practice.In addition,in the real environment,image datasets often do not contain to original click data,which greatly limits the application of click data.This thesis focuses on image classification based on predicted click feature in order to solve the problem of lacking of click data in traditional datasets.Firstly,we study the construction of real click features,secondly,we focus on an effective click prediction model from visual features,and finally,we study an efficient image classification framework using predicted click features.A novel multi-domain multi-task transfer deep model is proposed.It uses nonlinear word-embedding and position-sensitive loss to construct an improved click prediction model,and uses multi-task and multi cross-domain learning method to jointly learn the image classification and click prediction model.Experimental results show that our method has lower prediction loss and higher classification accuracy than other state-of-the-art approaches.In addition,in order to further enhance the click features,we use clustering algorithm to construct a hierarchical click feature model,with feature selection,and propose a classification model based on the predicted hierarchical click features.Experimental results show that hierarchical click features can better describe structured semantics,and the feature selection can effectively improve accuracy and reduce noise.