Mobile Visual Search Method Based On Deep Hashing

With the rapid development of mobile Internet technology and the widespread popularity of mobile devices,the visual data such as pictures and videos in the Internet are showing explosive growth.Mobile Visual Search(MVS)method uses visual data,such as images,videos,3D models or maps,collected by mobile intelligent terminal as retrieval object,and obtains relevant information.It can provide effective visual information retrieval method for users in mass visual content.MVS has become an important research subject in the field of information retrieval.However,due to the mobility and generalization of the mobile visual search scenes,there are many challenges in the related research:the image data in the mobile scene is easily affected by the external environment,the image noise reduces the efficiency of the Mobile Visual Search,and it is difficult to extract the semantic information of the image;besides,Mobile devices are usually limited by computation,storage and network transmission,it's difficult to meet the needs of users' rapid retrieval under the restrictive conditions.In recent years,deep learning methods have been widely used in the field of computer vision.Through layer by layer abstraction and iteration of neural networks,deep semantic information of images can be learned.Hash method is designed to convert high dimensional data into low dimensional representation through hash mapping,and can make the image feature sequence more compact.Deep learning method and hash method provide an effective solution for the problem of image semantic feature extraction and fast retrieval in the research of Mobile Visual Search.In this paper,the deep learning model and hash method are used to study deep semantic feature extraction and fast retrieval in the field of Mobile Visual Search.The main work and research results are as follows:(1)Based on deep hashing method,an image semantic feature extraction model is proposed.The model has the ability to learn the deep semantic features of the image by using the characteristics of layer by layer iteration and abstraction of the deep convolution neural network.By embedding a hash layer in the structure layer of the neural network model,the depth learning and hash algorithm are organically combined.The model can learn more compact image semantic feature representation and satisfy the requirements of image semantic feature extraction and fast retrieval in the field of Mobile Visual Search.(2)We propose a model loss function,which is suitable for the scenes of Mobile Visual Search.Considering the semantic sorting and model overfitting problems of Mobile Visual Search,the search ordering loss and L2 regular term are added to the softmax classification loss.Then,the image semantic feature extraction model based on the deep hashing is trained with this loss function,which can effectively enhance the learning ability and generalization performance of the model.(3)We construct Mobile Visual Search process based on deep hashing.In the processing of Mobile Visual Search,the image semantic feature extraction model based on deep hashing is used as the image feature extractor,and the extracted image feature hash sequence is used to match the image samples in the database,and the similarity between the samples is calculated by the Euclidean distance.Finally,according to the distance,we sort the semantics and return Top K search results for the users.(4)We build the experimental environment based on the open source deep learning framework MatConvNet,and make experiments on the open source data set PASCAL VOC 2012,Both of the image semantic feature extraction model based on deep hash and the mobile visual search process proposed in this paper are implemented,and the indicators of mAP,P@k=5,P@k=10,R@k=5,R@k=10,are applied to evaluate the accuracy and comprehensiveness of Mobile Visual Search process.In addition,considering the influence of model parameters,a comparative experiment is carried out for the super parameters in the model.The experimental data verify the effectiveness of the proposed Mobile Visual Search method based on deep hashing.