Implementation Of Industrial Big Data Monitoring And Analysis Platform Technology Based On Hadoop

In view of the problems faced by industrial enterprises under the traditional industrial architecture,such as data island,unclear data processing technology and low efficiency of data utilization,the relevant technical architecture of industrial timing big data cloud platform based on Hadoop has been deeply studied in this paper.A small private industrial big data cloud platform based on this architecture is built to realize cloud configuration,remote monitoring,prediction and analysis of factory equipment.The actual application of this cloud platform shows that it is of great significance for the application of big data technology in industrial enterprises and the promotion of intelligent decision-making by managers.In this paper,the above industrial big data cloud platform is divided into cloud data acquisition subsystem,cloud data distributed storage subsystem and cloud data distributed analysis subsystem,and each subsystem is designed and implemented.The main work of this paper is as follows:Firstly,by developing the data acquisition client based on the OPC UA protocol and integrating Flume,Sqoop and other related components,the cloud data acquisition subsystem of the cloud platform was built.The final experimental test shows that the cloud data acquisition subsystem supports the data acquisition of multiple data sources,as well as the high availability and scalability of this subsystem.Secondly,through the deployment and integration of HDFS,HBase and other relevant functional components in the Hadoop ecosystem,real-time database Redis and traditional relational database MySQL and other components,the cloud data distributed storage subsystem based on the Hadoop ecosystem has realized high reliability and diversified storage and utilization of industrial big data.Then,through the research and application of the GRU on Spark method,the high-efficiency construction of industrial timing big data model based on the method and the prediction analysis with high accuracy are realized.Experimental results show that the GRU on Spark method used in this paper,on the basis of retaining the simple structure of GRU threshold recursive neural network,makes foll use of the feature of Spark parallel computing framework relying on memory computing,which not only ensures the accuracy of model prediction,but also improves its training speed.This further indicates the feasibility of applying this method to industrial time series big data prediction and analysis scenarios.Finally,through the further optimization and development of the visualization framework of Abp Zero Core,the visualization management of the above subsystems of the industrial big data cloud platform in this paper is realized.Experimental results show that the overall architecture of the cloud platform is feasible and highly reliable.