The Design And Implementation Of HBase Large Object Storage

The age of big data is coming. With the rapid development of Internet, the huge size of the dataset which is generated by the amount of network access, network access logs, communication records, video and variety of intelligent terminals is expanding sharply. An important feature of the data set is embodied in:more than80%of the data is unstructured. Traditional technologies can not be qualified for the analysis, management and mining of the big data, and currently a popular solution for big data is HBase.HBase is not like the general relational databases, it is suitable for unstructured data storage database. And the HBase processes the unstructured data, that is a Large Object, as the same as other structured data. Therefore, during the insertion process of HBase, due to the large amount of unstructured data, the size of HBase region will grow quickly and the split and compaction of the region will frequently priming which will block the clients’ write request, then the insertion performance will be unsatisfied.Therefore, if we can reduce the split and compaction count, we can improve the HBase insertion performance largely. Besides, we should still need to take into account its read performance and complexity of storage management. And with the minimal modifications of the HBase source code, we don’t want to import external factors. Based on these factors, we put forword our solution of Large Object Storage (LOB) in HBase.