Design And Implementation Of Production Big Data Aggregation And Management System Based On Hive And Apache Kylin

The “Made in China 2025” plan puts forward the goal of building a manufacturing powerhouse.As an important part of the manufacturing industry,the injection molding industry is also continuously retrofitting and upgrading.At present,the information technology construction of the injection molding industry is still popularization.Some injection molding production enterprises have begun to pay attention to the importance of industrial software systems for production control and operation management of enterprises.They have gradually built up a manufacture production information processing chain from the automated system of mechanical equipment to the industrial software system.However,due to the long-term informatization construction,one-sidedness of understanding,inconsistency of system implementation standards,closure of department management and other reasons,the problem of “information islands” has emerged.This has caused a large amount of production data to fail to be fully integrated,shared and applied.It seriously restricting the development of data-driven smart production models.This paper aims at the problem of “information islands” in the process of informatization construction of injection molding industry mentioned above.It designs and implements a production big data aggregation and management system to manage the data in the injection molding production process,and feeds converged data back into isolated industrial software systems by using OLAP on Hadoop technology and visualization technology.The system enables interoperability of industrial software systems and provides data-driven decision support for injection molding companies.This article has the following four main tasks:1.Concerning the inconsistency of data storage in production data,the use of dimensional modeling techniques has gradually built a Hive data warehouse system for injection molding production big data to achieve unified data storage.2.For the data fusion problem of heterogeneous systems in industrial software systems,using ETL technology through data extraction,data cleaning,data conversion,data loading and other steps,the source data is stored in the Hive data warehouse in a consistent manner to achieve interconnection of systems.3.In response to Hive's high time-consuming query problem,the Apache Kylin engine is used to pre-aggregate the production data in the data warehouse to increase the query speed of the data.In addition,I extend the function on the basis of the engine to realize the requirements of building and managing data cubes automatically.4.For the problem that the data in the industrial software system is difficult to share and apply,a data cube-based data visualization platform is constructed through a server-side logic architecture,and a Web service support for the integration of industrial software systems is provided to achieve system interoperability.This paper aims at the integration,sharing and application of production data for the "information islands" problem in the injection molding industry,which will help promote the construction of the industrial Internet.