The Design And Implementation Of Data Integration And Analysis Model Of Yitian Data Management System

Business data plays a more and more important role in the operation of modern enterprise with the rapid development of enterprise informatization,as a result,managers need pay more attention on the usage of data.To solve the problem,YiTian Group co.tld prepare to develop an enterpeise data management system,which could help them collect data from different departments and provide more information for decision making.The thesis focuses on the implementation of data integration model and data analysis model.The system was developed on B/S pattern,and the server side was developed with standard multi-tier architecture.Besides,server offers RESTful API for browsers to separate the front from end.The data integration model aims to collect data from different departments,we abstract the process of data integration and implement the model with strategy pattern to support different data sources and data types.The data analysis model was designed to offer data analysis services,we made it through SparkSQL and MLlib.Besides,we provide the support of customized script by Java inflection,which contributes to the expansibility of the system.In addition,this thesis also introduces the implementation of auxiliary modules such as job model and dataset model,which achieve the dispatch of regular tasks and the control of data access.The system has been delivered and works well.It could complete the task of data integration and data mining,and we are making efforts to make it better with the demand side.