Research On Cross-modal Hashing Algorithm Based On Kernel Canonical Correlation Analysis And Neural Network

In recent years,with the rapid growth of the multimedia data,how to realize retrieval among different kinds of data has become a research hotspot in the field of information retrieval.Hashing method is a very effective method for information retrieval,which has the advantages of small storage space and high retrieval speed.Hashing method can be grouped into two categories:uni-modal hashing and cross-modal hashing.Uni-modal hashing method aims to retrieve information across single modality.Cross-modal hashing method aims to retrieve information across different modalities.The core idea of the hashing method is to transform into the corresponding binary codes,and the Hamming distance of binary codes is corresponding to the semantic similarity of the original data.Then we can obtain results by comparing the hamming distance between binary codes.In this thesis,we study the cross-modal hash method in depth,and propose two different cross-modal hash method:kernelized cross-modal hashing for multimedia retrieval(KCMH)and cross-modal hashing based on neural network(NNCH).The main content are as follows:1.Kernelized cross-modal hashing for multimedia retrieval(KCMH)is proposed in this thesis.First of all,we learn a common kernel space using Anchor Kernel Canonical Correlation Analysis(AKCCA),and map the data from different modalities into the kernel space.In this part,the intra-similarity and inter-similarity among original data are considered in the common kernel space.Different from the existing common space learning method,AKCCA combines the canonical correlation analysis,K-means algorithm and kernel technique,which can learn the nonlinear relationship very efficiently.After learn common kernel space,by using the equivalence between a hamming distance and a code inner product,we propose an objective function which can make the the similarity between the binary codes consistent with the semantic similarity of original data.Finally,we learn hash functions bit by bit through iterative optimization algorithm.2.Cross-modal hashing based on neural network(NNCH)is proposed in this thesis.Neural network algorithm is a simplified biological model,which has been widely used in artificial intelligence,computer vision and speech recognition and other fields.NNCH is successfully applied neural network technology to the field of cross-modal hashing method.NNCH proposes unique and effective neural network structure,loss function and the training method of neural network.Firstly,the neural network structure of NNCH is composed of two sub neural networks:image neural network and text neural network.The main task of the image neural network is to.transform image into binary codes and the the main task of text neural network is to transform image into binary codes.The two sub neural networks are connected by sharing the weight of the classifier.Then,NNCH proposes unique loss function,which is composed of softmax classifier loss and hash characteristic loss.Finally,we use this loss function and caffe to train neural network.