Research Of Database Incremental Extraction Based On MD5and HASH Algorithm And Its Application

In today’s society, information infrastructure more and more comp leted, andinformatio n excha nges increasingly frequent, leading to a mass ive growth o finformation. Data information’s loss and damage will cause immeasurable loss, anddata backup is the only so lutio n to solve this proble m. In a piece of time, only part ofthe data has changed in the vast a mounts of data, there would be a proble m of loweffic ie ncy if use full backup, but incre menta l backup can solve this problem asconvenie nt. In existing database incre menta l extraction technolo gy, theimp le mentation based on the third-party tools, which need conditions in most o f thetime, and it costs a lot at the sa me time; The shadow-table method is one of themethods that we usua lly use, but its effic ie ncy can’t suit our needs when incre menta lextraction with a huge a mount of data. So this paper focus o n improving theeffic ie ncy of the shadow-table method, and the work is as follows:(1) Proposed an improved shadow-table method based on Hash algorithm (ST-HJ).In the process of shadow-table method, sort-shadow-tab le should to sort the mass ivedata which ineffic ient ly. In order to solve this prob lem, this paper studies on theprinc ip le of existing shadow-tab le method and hash a lgorithm, fo und a way that makebackup table records mapped into a has h table, the n let the origina l tab le records tofind in the hash table at the same time. This method does not require any collection,and it can obtain increme nta l recording with linear sweep the collection. Theexperiments show that the effic ienc y of the ST-HJ algorithm improved by nearly60%,while in the case of the multi-attribute fie ld less than10%.(2) Improved ST-HJ algorithm. As we fo und in the ST-HJ algorithm that theeffic ie ncy wo uld be degrade in the multi-attribute fie ld. In order to resolve this iss ue,this paper stud ies on the “shadow-primary key” and MD5a lgorithm, fo und out a waythat use MD5algorithm turned multi-attribute fie ld into a fixed-le ngth string, thenused the ST-HJ algorithm. This method defined the length of the string, while havingthe advantage of the ST-HJ algorithm. The experime nts show that the a lgorithmeffic ie ncy improved nearly40%in the case of multi-attribute fie ld.(3) Imp le mented a database increme nta l backup system which based on ST-HJalgorithm. This syste m has been used in a project whic h named “The monitoring andanalys is syste m of China Mobile in Hunan Province”, and the effic ient backup has been come true, while reduced the impact of the origina l syste m. This technica lsolutio n has been accepted by customers.