Research On Semi-supervised Cross-modal Hashing Retrieval Algorithm

In recent years,with the continuous development of mobile Internet technology and the continuous popularization of mobile devices,a large amount of multi-modal data(images,texts,audio,video,etc.)has emerged on the Internet,and its scale has become more and more diverse.With the rapid growth of multi-modal data,how to realize the retrieval between different modal data has become a research hotspot in the field of information retrieval.Hashing methods can perform cross-modal retrieval quickly and easily due to its low storage requirements and fast retrieval speed.The core of hashing methods is to binary the characteristics of the original data,and then obtain the retrieval result by calculating the Hamming distance between the hash code of the data to be queried and the original data hash code,which greatly improves the data retrieval efficiency;Replacing the original data store with hashing code also greatly reduces the need for storage.The research directions of existing hashing methods can be divided into single mode hashing method and cross mode hashing method.This paper focuses on cross-modal hashing methods,whose main purpose is to achieve cross-search between various modal data.Crossmodal hashing methods can be divided into: supervised cross-modal hashing methods,unsupervised cross-modal hashing methods and weakly supervised cross-modal hashing methods according to whether or not the tag information of the training data is used.W eakly supervised cross-modal hashing methods use a small amount of supervised information and characteristic information of all training data to learn hash codes and hash functions,which is suitable for the case where the supervised information of a small amount of data in the data set is known.Weakly supervised cross-modal hashing methods face the problem of unclear pairwise relationship between training data and incomplete label information,and face the gap between the underlying features and high-level semantics.Most of the existing weakly supervised cross-modal hashing retrieval methods only use the semantic similarity relationship between paired data,lack of deep mining of the relationship between unlabeled data,and do not explore the relationship between the internal data features of the modality and affect retrieval accuracy.This paper studies these problems and proposes three weakly supervised cross-modal hashing methods.The main work of the thesis is as follows:(1)A semi-paired and semi-supervised multimodal hashing via cross-modal label propagation is proposed.The method uses cross-modal relationship propagation to transfer the similarity relationship between a small number of pairs of data to unsupervised data,and obtains a relationship matrix between modal data,thereby associating different modalities.Then,using the obtained relational matrix as a bridge,hash codes and hash functions are trained by the existing supervised cross-modal hashing methods.This method overcomes the problem of the lack of unsupervised data pairing in the weakly supervised cross-modal hashing retrieval methods.Experimental results show that this method has higher retrieval accuracy than the existing weakly supervised cross-modal hashing methods.(2)A semi-supervised cross-modal hashing based on cross-modal joint label propagation is proposed.The method first uses the cross-modal joint label propagation technology to obtain the label of the unsupervised training data,and then the label information is represented as the linear combination of the hash codes.The method obtains the label information of unsupervised data by cross-modal joint label propagation.The advantage is that the hash information of all training data is used instead of a small amount of label information when constructing the hash objective function.Experimental results show that the proposed method is better than the existing weakly supervised cross-modal hash method.(3)An efficient semi-supervised cross-modal hashing based on cross-modal joint label propagation is proposed.The method first uses the cross-modal joint label propagation technique to obtain the label of the unsupervised training data,and then represents the hash codes as a linear combination of the label information.In this way,the iterative formula is simplified and the iteration is reduced when solving the objective function.Compared with the semi-paired and semi-supervised multimodal hashing via cross-mpdality label propagation,this method greatly reduces the computational complexity of iteratively solving hash codes and hash functions,speeds up the training speed,and effectively reduces The accumulation of errors improves the accuracy of the search.Experimental results show that the method has faster training speed and higher accuracy.In summary,this paper proposes three kinds of weakly supervised cross-modal hashing methos by using cross-modal relationship propagation and cross-modal joint label propagation,from high-level semantic similarity search in modal and high-level semantic similarity search between modal.a large number of experimental results show the feasibility of the proposed methods and its superiority with the existing methods.