A Study Of Learned Hashing Index Based On Machine Learning

Nowadays,people are living in a highly developed information age.As a carrier of information,data records everything in our daily lives.In the context of data explosion,how to retrieve information efficiently has been becoming a hot topic in data science and related fields.It is not easy to query something efficiently and accurately when facing massive data.It is difficult to satisfy the requirements in a query process which takes both high efficiency and high precision.Hash index as an efficient information retrieval method,is widely used in various information retrieval systems;however,when the environment turns into a big data environment,the performance of the traditional hash index is severely challenged.To ensure retrieval efficiency,hashing methods have to sacrifice query precision in exchange for query efficiency.Besides that,the conventional hash index methods are mainly constructed by artificial spatial mapping,which also makes the hashing function have limitations in expressing the intrinsic correlation of data.In recent years,with the continuous development of machine learning,machine learning methods are gradually involved in various fields,which also provides a new idea for the construction of hash index,that is constructing the hash index by using machine learning models,and the learned index architecture proposed by Google is the product of this idea.This paper summarizes the previous work of using machine learning to construct the hashing method and propose a deep learning based local sensitive hash framework,which named LLSHF.This framework is based on learned index architecture,and consists of four layers,each of those layers define specific task rules.Compared with previous work,this paper proposed a general framework firstly by generalized the previous work,which builds a hash index by using machine learning.This framework could construct a specific hash function model according to specific requirements.In this paper,LLSHF is used to implement a hash method for low dimensional data and another hash method for image retrieval.The low dimensional data hash method introduces Hilbert Curve mapping into the framework as the target of hash model training.Besides,this paper designed an experiment to verify that the result of the hashing method,whether disburses the generalized LSH definition or not.The LLSHF-based image retrieval hash method has built an end-to-end solution by using transfer learning,so it can input an image directly into the hash model and generate a hash code.Finally,the LLSHF-based low dimensional data hash method is verified as a local sensitive hash method which fully satisfies the definition of generalized LSH.and the LLSHF-based image retrieval hash method also has excellent query performance.