Evolution Of Application Software Based On Hadoop Platform Method And Technology Research

Due to the upgrade of software requirements and versions,the reconstruction and evolution of software code has become the main means and key technical requirements for software maintenance.Hadoop distributed software With the continuous development of distributed big data processing technology,the version is continuously upgraded,and the application software based on Hadoop platform also needs to be upgraded.This upgrade process presents new challenges for software evolution technology.This paper proposes an evolution method and technology based on Hadoop platform application software,and discusses the technical process of evolution method implementation with concrete examples to verify the effectiveness of evolution method and technology.This paper mainly discusses the function migration and calculation speed improvement in the evolution of Hadoop platform application software.In terms of function portability,the mapping relationship between Hadoop1.x and the upgraded version 2.x and 3.x base class libraries is studied,including the evolution process of HDFS and MapReduce,and the Hadoop version evolution code complexity and platform upgrade process are analyzed.In terms of speed improvement,the evolution of MapReduce tasks and the monitoring and management of MapReduce tasks are evolved,and the MapReduce disk read and write operations to memory operation improvement algorithms and the algorithms for how MapReduce jobs are converted into Spark jobs are given.A case study of the InnerJoin application in the enterprise ETL(Extract Transform Load)processing is carried out to analyze the application difference of the InnerJoin in the MapReduce and Spark programming modes.This paper studies the transaction collection event in the Internet of Things application layer event ALE(Application Level Event),and gives the evolution of the ALE layer transaction collection event processing function,transforming the original function and improving the calculation speed.Finally,the paper analyzes and tests the evolutionary algorithm and evolution software,and gives the test cases of the evolution of InnerJoin application and EPCGlobal IoT application software in enterprise ETL processing.Experiments and tests show that the proposed evolutionary algorithm and evolutionary software system are feasible.