The Design And Implementation Of The Rule Based Data Quality Management System On Ali Cloud

With the development of the data collection of the State Administration of Taxation and the application of big data,the demand for data quality in the provinces has become more and more urgent.In order to meet the needs of administration to improve the quality of data,quality of data in order to guide the work,establish a complete and accurate data quality management system,data quality monitoring platform to set up a professional and efficient,has become an inevitable trend.This paper describes the use of B/S network architecture,Sword framework,J2EE technology and Java language to realize data quality management system,at the same time,the use of cloud technology and products Ali mature,including the object storage(OSS),big data,MaxCompute services(ODPS).Relational Database Service(RDS)and cloud server ECS,etc..Using four layers of Architecture:display layer,service layer,middle layer and data layer,the cloud database(RDS)is used for data storage.The system adopts the key technology,in addition to using Ali cloud components,but also uses the JSqlParser plug-in to resolve SQL statements,using Quartz components for task scheduling.Through the design and implementation aspects,the realization of data quality management system has the function of quality demand management function,quality rules management function,task management,quality management and quality quality monitoring function.Among them,quality requirements management ultimately achieves quality requirements definition and quality requirements maintenance.Quality rule management,final implementation,quality rule definition and quality rule maintenance,two function menus.Quality task management ultimately implements quality function definition,quality task maintenance,quality task approval,quality task monitoring and quality task outcome,and five function menus.Quality problem management,ultimately achieving quality issues,defining quality issues,maintaining and quality issues,and pushing three feature menus.Statistical quality monitoring and ultimately monitoring overview,quality monitoring and quality rectification problem of query three function menu.The application of data quality management system,to a certain extent,standardize the process of data quality management,and make the division of quality and responsibility of data more clear and definite.The whole process of the system is realized from top to bottom,and then from bottom to top,a loop,continuous cycle forward,based on the rules,to achieve continuous improvement of data quality.In the loop,a unified and standard quality rule base has been established,which can be used for reference by all provinces of the country.Using a large data system with advanced technology,to realize the analysis and monitoring of the massive data;give full play to the advantages of cloud computing,breaking the previous ORACLE database for the massive data analysis of low efficiency and even unable to get the final result.