Design And Implementation Of Enterprise Data Governance Platform

In the age of big data,data governance has become an increasingly important issue.Data governance platform has become an important guarantee for data management and data security.It has become the consensus of the industry and academia.For companies with large-scale and complex data,data governance is even more urgent.The field of data governance covers a wide range,and usually needs to be designed in line with actual needs and actual programs.As the core content of data governance,metadata management,authority management and data standard management provide powerful assistance in enterprise management of enterprise data,enhancement of enterprise data value,and reduction of enterprise data risk.The topic of the thesis comes from the "Data Governance Platform" entrusted to us by a certain enterprise.This paper has made a detailed design and implementation of metadata management,authority management,data standard management in the enterprise's data governance process and user management.The data that the company needs to manage are hardware equipment data,material data,personnel information data,etc.These data are stored in the Hive database and provided to the platform as the underlying data source.The system extracts the metadata information of the Hive database,and uses HBase to store the metadata information,and uses My SQL to store the data permissions set by the user to the Hive data according to the business classification,user information,as well as information about data standards.The main work content of the thesis includes the following aspects:(1)Analysis of system requirements.According to the needs of the enterprise data manager,it is determined that the data source of this platform comes from the Hive database,and the platform is divided into four modules: a metadata management module,an authority management module,a user management module and a data standard management module.Describe the interaction relationship between the external system and the system,and for different modules,refine the specific functional requirements of each module,and analyze the logical relationship between the functions.Finally,the four modules to be realized by the system are described in detail by UML modeling,and the non-functional requirements of the system are analyzed.(2)System design and implementation.Combining functional and non-functional requirements of the system,determine the technical selection,design the overall architecture of the system,and split the system functions.Combining class diagrams,sequence diagrams and flowcharts,the design and implementation of the sub-functions of the system modules are introduced.Abstract the relationship between system entities and entities,and design the database,and then introduce the physical table structure in the database and the field information in the table.(3)System test.After the implementation of the system,detailed test cases are designed according to the functional requirements of the system.We deployed the test environment required by the system.The system is tested according to the test cases.We tested and compared the non-functional requirements of the system.According to the test results,verify whether the functional and non-functional requirements of the system meet the requirements.According to the test results,the functional and non-functional requirements of the enterprise data governance platform implemented in this article have been met.The system can run normally and correctly process the functions of each module,and provide a visual page for users to use.In summary,the enterprise data governance platform meets the requirements of enterprises.The platform is practical and has great significance in helping enterprises achieve efficient and reliable data management.