Query Modeling For Click Data Based Image Recognition

For fine-grained image classification and recognition,users'click information are proved to be useful for construct image semantic features.With user-click data each image is represented as query-click-frequency vector.Compared with traditional visual features,this feature leads to higher recognition accuracy,because it contains more abundant semantic information.However,due to the redundancy and huge size of the text set,there are also many challenges in using the click feature directly for classification and recognition.For the application of fine-grained image recognition,this paper proposes to use text clustering to reduce the text space and optimize the original click feature,so as to establish compact and effective text space to represent the image.In particular,a novel method of text clustering based on click information is proposed.Compared with the traditional clustering method based on textural features,this method can better merge semantically similar text,and achieve text merging across languages.There are two advantage about this way:One is the text representation based on click information,which represents each text as an image click frequency vector,meanwhile,in order to solve the sparsity in this representation,a click propagation algorithm based on visual similarity consistency is proposed.The second is the clustering model based on click information.At the same time,in order to improve the clustering effect,a novel deep learning model is devised for efficient clustering.Extensive experiments have been carried out on Clickture-Dog dataset.Experimental results show that:1)In image representation,click vector feature is superior to traditional image visual feature,and the accuracy of image recognition task is higher;2)The propagation algorithm based on visual similarity can help improve the representations of click features;3)In the large text clustering,clustering method based on sparse coding is superior to the traditional k-means algorithm,especially in the sparse coding is used in the acquired by means of 'hot word dictionary,and get the better performance;4)the image's text click and deep visual features can be fused to achieve robust recognition performance.