| The crown number is the RMB's identity which consists of crown and number.The financial department can supervise the circulation of money through the statistics of the crown number.It plays an important role in grasping the origin of the money,revealing the financialconnections and discovering the problem money.The huge amount of crown number data collected by cash equipment mainly consists of two parts of text and picture.There is a huge challenge that the traditional relational database in dealing with massive crown number data because the system expansion is difficult and cost high.Although the cloud-based technology has been applied in the storing and querying of a lot of massive data,but there are some deficiencies in the practical application of the crown number such as the multi-dimensional querying,index constructing and querying the small file in the existing cloud platform.In this paper,the crown number storage and query system is designed and implemented based on cloud computing,and some of the key technologies of the system will be discussed.The querying of crown number text records often involve multiple query conditions,while the NoSql database based on cloud computing uses the Key-Value structure and only supports the fast query by primary key.For multiple non-primary key query,the efficiency is not high because it is likely to cause the full table scanning operation.In this paper,a multi-dimensional indexing framework based on HBase is proposed to achieve the quick query of the crown number.Firstly,the index framework designs a different granularity space index to achieve multi-dimensional data to one-dimensional projection.Then,the size of data block is limited by the number of records in each time segment.It can divide similar data into the same block when control the number of HBase data block;Finally,the construction of a time series of B + tree to reduce the search range and improve the efficiency of the query.The experimental results show that the proposed method achieves good results in both data inserting and querying,among them the results in multi-dimensional query are obviously better than CCIndex and MD-HBase.According to the storing and querying of crown number picture,this paper designs a storage scheme based on distributed file system(HDFS).Firstly,it uses multiple queues to merge files base on MapFile to reduce the number of small files and relieves the pressure of NameNode by metadata.Then,a secondary index structure based on HBase is proposed.The global index is the mapping between the merged files and small files,the local index is the mapping between small file and the pictures,it helps to reducing the addressing time of the picture and improving the query efficiency by two-stage index;Finally,the use of cache technology base on Redis to prefetch the picture data and calculate the file heat expression combined with time and accessing count.Improve the existing cache replacement algorithm with the heart expression to increase the cache hit rate.The experimental results show that theproposed method of the picture storing and querying can get better response time,and the caching mechanism can improve the efficiency of related queries further.Based on the method of storing and querying the crown number text and image,this paper designs and implements a massive crown number query system based on Hadoop cluster and HBase distributed database.The overall experiment results show that the system solutions proposed in this paper have achieved good results in storing and querying the crown number data,and the optimization method is feasible and effective. |
PDF Full Text Request