Design And Implementation Of Data Analysis Platform Based On IMM Standard Data Set

Under the background that data has increasingly become the most important competitive resource,many Internet companies are forced to continue to accelerate the pace of expansion due to competitive pressure,and inevitably carry out chimney-style data resource construction,thus generally forming the phenomenon of data islands.In particular,in the process of building a multi-business collaborative data analysis platform for enterprises,the consequences of the above phenomena are mainly manifested in: On the one hand,the phenomenon of data indicators with the same name being different from time to time occurs,which brings great ambiguity to data consumers..On the other hand,the data query layer and the data analysis layer are highly coupled,which seriously affects the timeliness of data analysis.Data analysis platforms based on traditional heterogeneous data sets have been difficult to meet the needs of data consumers,and it is difficult to accurately and efficiently drive business decisions through data to convert data value into business value.In view of the problems of poor data consistency,low query efficiency,poor visualization effect,and limited analysis scenarios in current data analysis platforms of this type,this paper adopts Index Model Management(IMM)in the process of designing and implementing the data analysis platform.)standard dataset as the underlying data source for the platform.The main work of this paper includes the following aspects:(1)Design of unified query service based on IMM standard dataset.Based on the standardized definition and disassembly of data indicators and dimensions,IMM uses data modeling methods to produce standard data sets.Based on the IMM standard data set,this paper designs a unified query service for the data analysis side,combines the design of the query cache service to improve the efficiency of data query,and provides a unified query interface,so that the data consumers on the analysis side do not need to care about the data caliber and the underlying layer.The engine and modeling method effectively solve the problems of low data acquisition efficiency and poor data consistency in the application of traditional heterogeneous data sets supporting cross-business data analysis platforms.(2)Design and implementation of data analysis platform.In this paper,the software life cycle methodology runs through the whole process of the design and implementation of the platform,and successively carries out requirements analysis,outline design,detailed design and testing,and realizes data portal management,data visualization,OLAP,self-service retrieval from top to bottom.It provides an interactive and friendly one-stop data analysis service for enterprise users.In addition,this paper conducts related research on anomaly detection and alarm algorithm,and successfully applies it to the data visualization module.The data analysis platform built in this paper has been running stably and has practical functions since it was launched and put into use.By realizing the decoupling of data query services and data analysis services,the scalability and reusability of the system are improved,and the problems faced by enterprise users are solved.The problems of low data retrieval efficiency and poor data consistency also meet the more diversified data analysis and application needs of users,further magnify the value of data,and bring huge room for improvement for the development and profitability of enterprises.