Design And Implementation Of Visual Analysis Tool For Multidimensional Data

The technological development of the Internet and big data provides people with a variety of online services such as online shopping and online meetings.More than 4 billion Internet users around the world generate large-scale data sets,while enjoying convenience.Related companies or people can easily obtain massive amounts of data.But,the data of most enterprise is scattered in the business systems of various departments or business lines,and there is a problem of data inconsistency,how to present the data and mine the value of these data has become a difficult problem.In recent years,the development of data warehousing,OLAP(online analytical processing),React and other data visualization analysis technologies has made the use of graphical interfaces for multi-dimensional data visualization analysis to attract widespread attention.However,the existing visual analysis products still have some shortcomings,including lack of data preprocessing,data drilling and task scheduling.Based on the above background and requirements,the topic of this paper comes from the actual project of the enterprise.It has designed and implemented a Web application system based on multi-dimensional data,capable of data extraction,processing and visual analysis.The visual analysis of multi-dimensional data can be realized by simply dragging and dropping chart components.The main contents of this paper are as follows:(1)Learn about the background of multi-dimensional data visualization analysis tools and the current research status.According to the business requirement scenario,this paper studies the requirement participants and specific activities of the module use cases such as data sources,data sets,visualization reports,scheduling tasks and authorization management of the system.In addition,this paper studies non-functional requirements such as reliability,safety,and efficiency of the system.(2)In response to the complex process of the traditional report making,this paper provides a drag-and-drop visualization report production through the visualization technology.In order to implement the integration of heterogeneous data in the application scenario,the system uses Kettle technology to extract the data provided by the user into the My SQL which is an intermediate database,process the connection between different data tables and implement the data cleaning and conversion.This paper use Sqoop to extract the processed data in My SQL to the Hive data warehouse.In order to speed up the query response speed,it select the Apache Kylin multi-dimensional analysis engine to pre-calculate the data in Hive to build a data cube.When users are drilling the data,the query interface of Kylin is called to query.Finally,this paper describes the detailed design of the system database and each functional module,it also describe the realization of the front and back ends of the system.(3)This paper writes test cases for each functional module in the multidimensional data visualization analysis tool to verify and ensure that each functional module is performed well.The system reliability,safety and performance are tested,and the test results meet the requirements.In summary,the various modules of the multi-dimensional data visualization analysis tool designed and implemented in this article have functions which are performing stably and reliably.And,the system has good query performance.