The Design Of Bloom Filter Algorithm For Key-value Storage

With the development of Internet technology,the rise of cloud computing,Internet of things and the mobile communication technology,hundreds of millions of users generate huge amounts of information at every moment.The age of mass data has come.The Bloom filter is a random data structure with high spatial efficiency and time efficiency.It is used to detect whether an element belongs to a set,which can be used to process massive data and query fast collection elements.The key-value storage is the simplest form of organization of database.Basically all programming languages have key-value pairs stored in memory.Traditional key-value storage are very challenging for mass data processing.Therefore,this paper extends the structure of Bloom filter and designs two query algorithms for key value pairs of Bloom filters.The main research points of this paper are as follows:Based on the analysis of the key value pairs of Bloom filter structure,an extensible Scalable Bloom Filter Tree structure is proposed,and an algorithm for inserting,querying and adding new values for key value pairs is designed.The structure distributes a number of Bloom filter vectors on a tree,and this paper uses a H3 hash function to further expand the structure.This novel structure not only improves the processing speed of key value pairs,but also supports dynamic data processing,and is more suitable for the real network envi-ronment.Finally,theoretical analysis and simulation experiments verify the effectiveness of the structure in dealing with key values and data.On the basis of the analysis of the structural characteristics of a variety of processing key values resulting in conflict,a key-value Bloom filter based on Bh sequence is proposed,and the algorithm for inserting,querying,deleting and updating key value pairs is designed.In this paper,a special encoding method,Bh sequence,is used to solve the multiple sit-uations of the conflict when the key value is inserted into the data,so as to improve the efficiency of the query and reduce the rate of misjudgment.Finally,a real network data set is used to carry out simulation experiments.The experimental results show that the Bh-BF structure can tolerate more conflicts,lower error rate,higher processing efficiency and more suitable for use in real network applications to handle key value pairs than the existing two kinds of structure based on the Bloom filter using cell.