Research And Implementation Of Multi-dimensional Data Analysis System For Education Big Data

With the advent of the era of big data, the data of the large scale and complex structure has brought some directly challenges to the traditional multi-dimensional data analysis system. Especially in the field of education management, growing business data make traditional school situation analysis system under a lot of pressure, such as inadequate performance of high speed data extraction and write, low query efficiency under the large scale of data, and cannot provide support for unstructured data, etc. This makes a lot of difficulties in the analysis and evaluation process of the complex school situation.The traditional technology of data collection, management and analysis is no longer adapt to the requirements for efficiency and scalability of education big data analysis.To solve the above problems, this article was designed and implemented a multi-dimensional data analysis system for education big data based on big data technology, the system has the characteristics of high performance, high scalability and friendly interface.This thesis first to have an in-depth study and analysis on the relevant technology. According to the characteristics of column based storage mode in HBase, designed a multidimensional data model based on HBase, so the fact table is stored in HBase database by dimensions is achieved.This brings to the creation and maintenance of dimensions of greater flexibility, and improves the scalability of the multidimensional data model, and enable the system to provide support for unstructured data.At the same time, we made integration of the HBase and Hive, using HiveQL which is service from Hive and based on MapReduce framework to improve the efficiency of data extraction, transformation and load.Then we designed a multi-dimensional data analysis system for education big data in the view of the big data processing procedure. The system is composed of cloud data module, business intelligent analysis module, information display module and system management module, using Hive to realize the function of statistical analysis and correlation analysis, using Mahout to achieve data mining functions such as clustering and classification; The system also has the evaluation function of the first-class indicator, second-class indicator and third-class indicator.Finally the building process of system environment and the implementation principles of key functions is given. The comparison experiments proved that the implementation system can meet the actual needs of high efficiency and good scalability in the big data environment.