Deep Joint Semantic-embedding Hashing

With the popularization of digital products and the rapid development of Internet technology,image data is constantly pouring into the Internet with the characteristics of large scale,high dimension and complex semantic relationship.Because of their fast retrieval speed and low storage cost,hash methods have become a popular research direction in the field of contentbased image retrieval.The typical hash method maps the image into a compact binary code,i.e.,hash code,and maintains the original similarity relationship of the images.The fast XOR operations between compact hash codes speed up the speed of retrieval and reduce storage.In recent years,deep learning has been widely used in hash algorithms.Compared with the traditional hash algorithms,the performance of deep hashing has been greatly improved,but there are still some problems to be solved:(1)The mainstream deep hash algorithms use image labels to construct a similarity matrix and use the similarity matrix to supervise the training.The similarity matrix-based approaches face two challenges.a)the unbalanced feature of single-label data: the number of similar pairs of single-label data is much smaller than the number of dissimilar pairs,resulting in a sparser similarity matrix,which makes it difficult for neural networks to learn and converge;b)incomplete semantic mining of multi-label data: multiple labels lead to more complex evolution of data similarity,similarity matrices are only reduced to similar and dissimilar discrete relationships,which results in rich semantic relationships,and accurately engraved.(2)Deep learning requires a large amount of label data to be driven.Some methods should be explored to reduce the dependence of the algorithm on the existing annotation data,and leverage other similar data to help the model to learn and reduce human consumption.To deal with the above two problems,this paper proposes two corresponding deep hash algorithms:(1)Proposed Deep Joint Semantic-Embedding Hashing.The algorithm includes two neural networks: Lab Net and Img Net.Lab Net is designed to mine the semantic feature information of labels and learn the accurate hash codes and embed its learned semantic features and hash codes into the semantic space and Hamming space shared by the two networks.The semantic feature learning and hash codes learning are both supervised by Img Net.Using the hash codes learned by Lab Net to directly supervise the hash learning for images,instead of the direct constraint of the original similarity matrix,the imbalance of the single-label dataset can be avoided.Lab Net mines rich semantic features from labels and constructs a more accurate similarity relationship,which overcomes the problem of incomplete semantic mining of multi-label datasets.(2)Proposed Deep Adversarial Learning for Transfer Hashing.The algorithm introduces confrontation learning in migration learning and uses the labeled source domain dataset to help unlabeled target domain dataset for hash learning.The algorithm constructs an end-toend neural network,including domain feature extractor,domain classifier and hash encoder.The domain classifier introduces domain classification loss,and it is expected that the features from the two domains are more easily distinguished.The domain feature extractor introduces domain confusion loss,and it is expected that the feature data from the two domains are difficult to distinguish.Finally,the domain feature extractor extracts the feature distribution from different domains,so that the data of different domains cooperate with each other,which reduces the dependence on the image annotation information.