Design And Research Of Industrial Data Platform Based On Hadoop Ecosphere

At present,“getting the data is equivalent to getting the world” has become a global consensus.Data is no longer a “by-product” of social production.It has become a raw material for production,can be processed many times,and excavated more value from it.It can be predicted that the foundation of the company's core competitiveness in the future is the accumulation,processing and liquidation capabilities of the company's big data.However,as the “eyes of pearl” manufacturing of the real economy,it is becoming th e main battlefield for big data integration.With the continuous upgrading of industrial stru cture and the cross-border integration of technology among software,network and equipm ent,all of them are industrial Data provides a certain technical foundation for the develop ment of manufacturing.Thesis proposes an industrial data platform design based on Hadoop ecosystem,and conducts analysis and research;discusses the HDFS distributed file system,Spark computi ng framework,column database,visualization application of information technology indus trial data platform Multiple key technical points throughout the process.Analyzed the deta iled operation mechanism of the Hadoop ecosystem components used,and supplemented t he relational database optimization strategy about column design and massive parallel pro cessing for the shortcomings of Hadoop's actual data storage scenarios;Thesis summarizes the platform requirements from the actual project,sorts out the ov erall customer requirements of the platform,and then describes the technical function mod ules required by the platform from the technical point of view,and then from the perspecti ve of operation and maintenance personnel and customers,and from stability and interpers onal interaction.The other factors analyzed the non-functional requirements of the platfor m,and finally determined the final scale and expected effect of the platform.Based on the abundant theroreticalfoundation,this paper studies the platform architec-ture design and verifies the feasibility of the overall technology through the construction and testing of various application modules of marketing,human resources,manufacturing,and database backup.At the same time,the reliability and scalability of the platform are g uaranteed.