Image Retrieval Algorithms Based On Generative And Adversarial Schema

Image retrieval is one of the core problems in the fields of digital image processing and computer vision,which is often formulated as a problem of metric learning.Based on the mapping from image space to feature representation,the images from same category are clustered together while the images from different categories are pushing away.Image retrieval has a wide range of applications in real-world scenarios,such as Internet image search engines,e-commerce product retrieval,and human/car searching in traffic and security scenarios.On the other hand,the introduction and development of Generative Adversarial Networks(GANs)has been one of the focuses in the field of machine learning in recent years.GANs not only perform well in generative tasks such as image and text generation,but are also applied to discriminative tasks such as transfer learning,semi-supervised or unsupervised learning.This dissertation proposes several algorithms for applying the generative and adversarial schema to image retrieval applications.The main works are as follows:Firstly,we discuss image generation with controllable content and style.Unlike most existing GAN models which typically generate images from noise vectors and conditioned one-hot keys of class labels,the proposed approach generates images based on two components: style feature and content feature,encoded from real images or attribute vectors.In addition,a contentstyle constraint loss is proposed to replace the common used reconstruction loss,thus enables the network to be trained with unpaired image dataset.Experimental results on several datasets demonstrate that the proposed method achieves superior results for image generation and remarkably improves the corresponding supervised recognition task.Secondly,this dissertation applies the adversarial training to image retrieval.The current state-of-the-art is to mine the most hard triplet examples from the mini-batch to train the network.However,mining-based methods tend to look into these triplets that are hard in terms of the current estimated network,rather than deliberately generating those hard triplets that really matter in globally optimizing the network.For this purpose,we propose an adversarial network for Hard Triplet Generation to optimize the network ability.We evaluate our method on the real-world challenging datasets and show that our method outperforms the state-of-the-art methods significantly.Furthermore,this dissertation investigates person re-identification(reid),one of the typical application of image retrieval.Most existing video re-id methods fuse features by temporal average-pooling,without exploring the different frame weights caused by various viewpoints,poses,and occlusions.In this dissertation,we propose an attribute-driven method for feature disentangling and frame re-weighting.The features of single frames are disentangled into groups of sub-features,each corresponds to specific semantic attributes.The sub-features are re-weighted by the confidence of attribute recognition and then aggregated at the temporal dimension as the final representation.Extensive experimental results verify the effectiveness of the proposed method.The high labeling cost of person re-id task promoted related research on unsupervised learning algorithms.This dissertation uses an image generation algorithm based on camera style transfer for unsupervised training.Besides traditional self-learning method which only relies on the Euclidean distance,this dissertation proposes a distance metric based on topological similarity,which is used to generate pseudo-labels and further used for network fine-tuning.Extensive experimental results validate the superiority of our proposed algorithm,which outperforms the state-of-the-art unsupervised and domain adaptation re-identification methods.In summary,this dissertation focusses on generative adversarial network and its application in image retrieval tasks,and explore the following topics in depth:(1)Image Generation Based on Content-Style Constraint;(2)Image Retrieval Based on Hard Triplet Generation;(3)AttributeDriven Feature Disentangling and Temporal Aggregation for Video Person Re-Identification;(4)Unsupervised Person Re-identification with Image Generation and Topological Similarity.The researches above creatively apply the idea of generative adversarial networks to image retrieval applications with certain theoretical and experimental results,which promote the research and application of image retrieval algorithms.