Research On Cross-Modal Hashing Method Based On Class Semantic Embedding

In recent years,with the continuous diversification of data types,how to effectively retrieve cross-modal data has become a hot issue.Because the hash-based cross-modal retrieval method has the characteristics of low storage cost and high retrieval speed,cross-modal hashing has become the focus of many scholars' research.However,most of the existing research neglects the proximity of neighbors and proximity of classes,which degrades the discrimination of hash codes.In addition,with the explosive growth of data,a large number of emerging concepts(unseen data)bring great challenges to traditional cross-modal retrieval,and most existing approaches mainly focus on improving cross-modal retrieval performance of seen classes,which may fail in the unseen classes.In order to solve the above problems,this paper proposes two cross-modal hashing methods based on class semantic embedding,which uses class semantic information to establish links between classes,thereby improving the performance of cross-modal retrieval.The specific content includes:(1)Targeting for the problem of proximity preservation,a manifold-embedded semantic hashing algorithm is proposed.This method uses Local Linear Embedding to model the neighborhood proximity and uses class semantic embeddings to consider the proximity of classes.By so doing,manifold-embedded semantic hashing can not only extract the manifold structure in different modalities,but also can embed the class semantic information into hash codes to further improve the discrimination of learned hash codes.(2)Aiming at the zero-shot problem,an attribute-based orthogonal projection cross-modal hashing algorithm is proposed.It projects cross-modal features and class attributes onto a Hamming space,where each projection of cross-modal features is orthogonal to the mismatched class attributes.By so doing,the model can learn a discriminative and binary representation of each modality.In addition,the class attributes build a bridge to transfer knowledge from seen classes to unseen classes,thereby completing the zero-shot cross-modal retrieval task.This paper conducts a large number of experiments on widely used datasets.After comparing with a variety of advanced methods,the experimental results show that the manifold-embedding semantic hashing proposed in this paper can effectively improve the performance of cross-modal retrieval,and at the same time,it can achieve superiority performance in depth features.The attribute-based orthogonal projection cross-modal hashing proposed in this paper can obtain great results in the retrieval of seen and unseen categories,which verifies the effectiveness of this method in dealing with the zero-shot cross-modal retrieval problem.Therefore,the algorithm proposed in this paper can effectively handle traditional cross-modal retrieval and zero-shot cross-modal retrieval tasks.