Fine-Grained Image Recognition Using Image Click-Through Logs

With the continuous progress on image recognition technology and the develop-ment of deep learning,image recognition is becoming more and more practical,more and more science and technology firms have begun to get involved in the field of image recognition, image recognition has become an important field of artificial intelligence,widely used in face recognition, character recognition, fingerprint recognition, license plate recognition and so on. The research focusing on image recognition in two dif-ferent positions, one is the ordinary image recognition, mainly to distinguish between different types of objects,such as "flower","bird","fish" and "insect",these objects of different kinds are relatively easy to distinguish because of the large difference be-tween them. The other is a fine-grained image recognition, these categories are usually sub categories of common categories,such as "dog" which is a general category can be further divided into different breeds,identify different breeds is fine-grained image recognition. Compared with the ordinary image recognition, fine-grained image recog-nition is more difficult, one reason is that the difference between different sub categories is relatively small, and is easily affected by the angle, brightness, occlusion, and other aspects of the influence of background. When the apply some of the more complex classifiers (such as the depth neural network) for fine-gained image recognition, anoth-er difficulty is how to get large-scale high quality annotation data sets used for training,such as the recognition of dog breeds, may need specific professional knowledge, but ordinary people may not have the expertise, so manually label and collect images is very difficult and expensive. In this paper, we propose two kinds of methods to improve the recognition accuracy pointed at the difficulties of fine-grained image recognition.First, we choose the widespread data that is weakly labeled on the Internet - image click-through logs, as the training data set. With the popularization of Internet and multi-media equipment, multimedia data represented with images showing explosive growth,there are tens of thousands of users active in the major search engines every day. The keywords user used when searching for images can be seen as image annotation. Com-pared with gaining images with manual annotations, getting these images on Internet are more easily, can save more time and money. However, the properties of these im-ages can not clearly labeled an image sometimes, so the datasets getting from Internet usually contain too many noisy images which can not be ignored. In our method, a part of images are randomly selected from the dataset for training a deep neural network model, and then use the network model to classify the whole dataset, we can get the probability corresponding to the weak labeled category of each image. We choose im-ages with large probabilities forming a new training set and then train the deep network model again. The experiment results show that this method can effectively improve the classification accuracy of some data sets.Then according to difficulty of the large-scale datasets is not easy to get for fine-grained image recognition, in order to distinguish the subtle differences of differen-t species and take diversity of the same species into consideration, we propose a loss function which is suitable for fine-grained image recognition, can significantly improve the accuracy of fine-grained image recognition when the training datasets is small. We first compose image pairs between classes and within the same classes using the images in training set as the input of Siamese neural network, and then extracted the probabil-ities of the last layer output in two subsets of Siamese neural network, then use the loss function we proposed to calculate loss. When the difference of probability in im-age pairs coming from different classes exceeds a certain threshold there is no loss, the same, when the difference of probability in image pairs coming from the same classes is less than a threshold there is no loss. The experiment results show that this loss func-tion based on Siamese neural network can further improve the classification accuracy of fine-grained image recognition.Finally, we set up a website to achieve online recognition to show the results of our fine-grained image recognition.