Research On Semantic-Guided Hashing Algorithms For Multimedia Data Retrieval

With the astonishing development of Internet,multimedia data have become one of the "largest data".Their quantity,complexity,diversity,high-dimensionality and multi-modality put forward higher requirements for efficient retrieval.Hashing technology encodes data into a set of binary hash codes which can be compared efficiently in Hamming space,and shows great advantages in efficient retrieval of large-scale data.In the era of multimedia big data,learning semantic representation for multimedia data and realizing efficient hashing retrieval have become urgent problems to be solved.In addition,semantic information plays a vital role in hashing algorithms.The supervised hashing algorithms based on semantics usually perform better than the unsupervised hashing algorithms using only unlabeled data.Therefore,this dissertation proposes several semantic-guided hashing algorithms for multimedia data retrieval,and explores multiple semantic information to improve the performance of hashing retrieval.The main research topics and solutions are as follows:1)This dissertation presents hashing algorithms based on label semantics for multimedia data retrieval.Extended semantic labels of training data can effectively bridge the heterogeneous modality gap in multimedia data.There are two problems in the existing hashing methods for multimedia data retrieval.First,the "relaxation + rounding" hash optimization strategy leads to significant quantization errors.Second,constructing graph structure to represent the relationship between multimedia data samples leads to great computational pressure.Based on the above considerations,this dissertation first proposes a Discrete Latent Semantic Cross-modal Hashing(DLSCMH)method for cross-modal hashing retrieval,which finds a latent shared space of heterogeneous multimedia data in the unified hash learning framework.Explicit semantic labels are used to enhance the semantic representation capability of hash codes.An iterative discrete hash optimization strategy is proposed to learn binary hash code and reduce quantization loss.Besides,this dissertation proposes a Supervised Discrete Multi-modal Hashing(SDMH)method to enhance the performance of multi-modal hashing retrieval.Multi-modal features and category labels are used to learn the discriminant hash codes and support efficient discrete hash optimization.2)This dissertation presents multi-modal hashing algorithms embedded in pair-wise similarity semantics.Pair-wise similarity semantics help achieve better retrieval performance by modeling the semantic correlation between multimedia data.Firstly,this dissertation proposes an Online Multi-modal Hashing with Dynamic Query-adaption(OMH-DQ)method.A self-weighted fusion strategy is designed to adaptively store the complementary multi-modal feature in the hash codes.Hash codes are learned under the supervision of pair-wise similarity semantics to enhance their semantic representation capability and avoid the huge space and time consumption caused by symmetric similarity matrix factorization.The discrete hash optimization strategy improves operation efficiency and avoids quantization error.The parameterless online hashing module adaptively learns the query hash codes according to the dynamic query contents.Secondly,this dissertation proposes a Flexible Online Multi-modal Hashing(FOMH)method to flexibly learn discriminant hash codes for streaming multimedia data.FOMH automatically learns the weights by multi-modal binary projection and captures the variations of streaming samples.The asymmetric online supervised hashing strategy is used to enhance the semantic representation capability of hash codes.A discrete online optimization strategy is used to update hash codes directly and avoid the propagation of binary quantization error during online hashing.3)This dissertation presents a multi-modal hashing algorithm based on topological semantics,named Flexible Graph Convolutional Multi-modal Hashing(FGCMH)method.Capturing and modeling the topological correlation between semantics can improve hashing retrieval performance.The graph convolution networks are used to preserve the modality-specific structure and the modality-fused structure,thus eliminate the heterogeneous modality gap.Then the interdependence between semantics in the high-level semantic space are modeled to guide the discriminant hash learning.By combining adaptive multi-modal fusion loss,classification loss,quantization loss and discriminant hash learning loss,the hash codes are learned and can be regarded as multi-modal fusion descriptors.