Research On Data Generation And Index Method Based On Hbase

As the time has developed from Internet era to Internet of things (IOT) era, the traditional network has evolved into the connections between "thing" and "thing". These changes brings not only changes in the scale of the network and data, and also changes in the structure of the data we processed. Distinguish from the traditional structured data that we processed in the IOT data processing system will always be unstructured. The RDBMS is not only insufficient in dealing with unstructured data, but also unable to effectively deal with large-scale data. Thus we should introduce a new database system that can be used to deal with the large-scale unstructured data. HBase is a large-scale distributed database based on the Hadoop cloud computing platform, has an advantage in dealing with the large-scale unstructured data that generated in the real life, but it also has a disadvantage. The only index used in the HBase is the key-value index, that can make the user more convenient to use the database to deal with the unstructured data, but it also block the detail of data within the HBase, that means we can only use the key to locate the value in the HBase, but we cannot use the value we have knew to get the key of the data, thus make it inconvenient to use, mainly incarnate we can not processe queries on some conditions. But some data processing system, like GIS, has the need that use the detail value of the data within the database to process multidimensional query, meantime due to the complexity of the multidimensional query, and the current requirements of the large-scale multi-dimensional data query’s speed and accuracy continue to increase. Therefore build an effective multidimensional index mechanism in the HBase to accelerate the procedure of the multidimensional query.In the paper we will build an effective multidimensional index mechanism for the HBase by applications, provide a new read/write procedure, make it convenient for the user to use the HBase, and provide a new function for the HBase. We first analyze the multidimensional index mechanism used in the nowadays and analyses the typical and efficient multidimensional index algorithm—R-tree, mainly focus on the build, query, delete and adjust the R-tree. And combined with HBase, transplant R-tree index file into the HBase by applications, during the data-reading procedure we use the applications to read the index files in HBase to achieve the purpose that we can proccess the condition queries. Given the HBase a new ability to process the condition query, make the HBase can be used in more situations. At the end of the paper, a field service manage system that based on the application that can multidimensional index the HBase data is given to verify the correctness of system design.