The Design And Implementation Of Smart Manufacturing Big Data Based On Hadoop Handle Platform

In the era of Industry 4.0,with the rapid development of sensor technology and computer technology,the scale of data in the industrial field has grown rapidly,and the value contained in the data has also increased.This is also the pain point to be solved by the traditional manufacturing enterprise information system—— Integrated storage and analysis of massive industrial data.There is a phenomenon of "data island" among the business information systems of traditional manufacturing enterprises.The poor data circulation of each business information system hinders the development of data-driven intelligent manufacturing.Therefore,how to solve the problem of integrated storage and analysis of intelligent manufacturing data,and how to mine deep value from massive data,has important research significance and use value.This paper originates from the national key research and development plan "Research and Development of Network Collaborative Manufacturing Integration Technology and Platform Based on Industrial Interconnection",aiming at the problems existing in group manufacturing enterprises,such as difficulty in gathering and utilizing data resources of the whole process and whole industry chain of product manufacturing,and low efficiency of operation and service.A Hadoop-based intelligent manufacturing big data platform has been developed to realize integrated storage and analysis of multi-source massive data of group manufacturing enterprises.Firstly,the characteristics,classification and data sources of industrial data are analyzed,and then the data acquisition scheme and data storage scheme are designed according to the different data types and characteristics,and an ETL data acquisition system for multi-source historical data is designed and implemented.And Hive-based data warehouse,provides a solution for the integrated storage of multi-source massive data.Secondly,the common algorithms in the field of big data processing are optimized and implemented.Aiming at the shortcomings of the traditional K-means clustering algorithm K value and the initial cluster center that are difficult to determine,the Canopy algorithm and the maximum and minimum distance algorithm are optimized,and the parallelization of the K-means clustering optimization algorithm is completed by using the Flink distributed computing framework,which improves the computational efficiency and accuracy of the clustering process.Aiming at the problem that the computing efficiency of DBSCAN clustering algorithm is slow,the parallelization of DBSCAN algorithm is realized by using Spark distributed computing framework,and a good acceleration ratio is obtained without obvious loss of clustering accuracy.Finally,based on the VMware Workstation virtual machine software,the big data basic platform Hadoop cluster and related components are built,and the platform is initially implemented according to the B/S architecture and the front-end and back-end separation architecture,which reduces the coupling between the front-end and the back-end,and uses Spring Boot in the back-end.The framework completes the implementation of the business code,adopts HTML,CSS and Java Script components in the front end,and verifies the relevant functions of the platform,which verifies the feasibility of the platform in this paper.