A Social Security Data Warehouse Analysis System Based On Pre-compucation

In recent year,with the gradual improvement of the social security institution and the expanded coverage of the social security system,the information construction of social security has encountered new challenges.The social security service database management system has accumulated lots of business data,which is not standardized and not unified.To solve these problems,this paper proposes the construction of social security data warehouse and designs the data warehouse analysis system based on pre-computation.This paper describes the design and implementation of a social security data warehouse analysis system based on pre-computation.The system uses Sqoop and Flume to collect the social security service database file and the server log file,the collected data is stored in Hive and Kafka,and the data warehouse is built on Hive.Then,the business data is analyzed by OLAP analysis tools.The architecture of the system supports the ROLAP engine,Impala,and the MOLAP engine,Kylin.The pre-compucation implementation of the system is based on Kylin subsystem.The decision maker chooses the OLAP engine according to the requirement of the analysis,and makes the online analysis processing interactively by the way of accessing the web page.The result of the query is directly returned to the decision maker in the visual way of form and graph.After that the paper also puts forward two improvements on the precomputation technology of the system,improving the Trie algorithm to be segmented and redesigning the implementation of bitmap operations.In addition,the system adds the metadata automatic configuration function and optimizes the OLAP query cache.Finally,the paper makes a comprehensive functional test and performance test.The test results show that the response of the social security data warehouse analysis system based on the pre-computation delays to second and even millisecond,which is far less than the traditional solution based ROLAP.In addition,the improvement of the system is compared with the former,and the test results show that the improvement can improve the performance of the system and the efficiency of the system.