Design And Implementation Of Industrial Data Analysis Platform Based On Big Data

The Industrial data analysis platform based on Big Data is for a certain ministry.It needs to summarize the economic operation data of certain industrial enterprises,and provide decision-makers with a clear and accurate data service platform to facilitate the relevant government management departments to formulate economic policies or management decisions.The data covers enterprise,legal person,financial and other information,and provides a visual data service by summarizing and analyzing the above information.The focus of this thesis is to analyze the development status and main problems of China's industry data analysis,and to summarize,collate and collate the data that has been manually reported or distributed in various systems into standard normative data.Through the basic information of the enterprise,the enterprises are classified,the visual analysis and mining of enterprise data are realized,and the key data urgently needed by the decision makers is extracted.Aiming at the needs analysis of decision makers,establish corresponding strategic measures to improve the quality of data services.The completion of the system provides a useful complement and improvement to the existing industry data analysis system.The system is mainly divided into six modules,namely system management,data collection,data review,data summary,progress monitoring and data prediction module.The system management includes functions such as adding,modifying,deleting and exporting the system.Data collection includes task query,report task,and related functions used in the process of filing.The data review function includes querying,reviewing,and reviewing data for economic units that have reported data.The data summary contains the query for the custom summary table,add the summary form,the design summary form,and the summary statistics calculation data.The progress monitoring module includes information such as viewing existing tasks,displaying the distribution of provincial and municipal completions,the top ten completions of provinces and cities nationwide,and the completion of the last ten provinces and cities.The data prediction module can be used to predict the future value according to the selected indicator,and the data prediction uses the ARIMA model to obtain better prediction data.The implementation of this system adopts B/S architecture,the development framework adopts Spring Boot,the development language selects JAVA,the JDK version is 1.8.0 181,the development tool is eclipse,the middleware adopts Tomcat,the data warehouse adopts Hive(2.3.4)and the database is HBase(0.98.13).In the process of development project,the author is mainly responsible for the demand research of the industrial data analysis platform,establishing the index system according to business needs,installing Galaxy Kirin operating system,building Hadoop cluster,installing ZooKeeper component,HBase database,DM7 database,mainly responsible for data collection.,data summary,data prediction module design and writing,participate in the system management,data review and progress monitoring module design.Finally,write the corresponding test cases to perform functional and non-functional testing of the system,and write the corresponding installation and deployment manual.At present,the industrial data analysis platform has been put into use,and the statistical industry data is accurate,which can greatly improve the work efficiency of data collection,and the user feedback is excellent.