Research And Application Of Data Mining Technology Based On Spark In ERP System

In this society of information explosion and constantly improved and popularized,applied machine learning to analyze massive data efficiently has been an inevitable trend for data mining.For those machine parts manufacturing enterprises,there are lots of data in their ERP system,if there is no relevant data mining technology to support,the information which is helpful for business decisions will be lost and leads to the delay and misalignment of business decision plans,at the same time,with the long-time interaction of the system,it will make the traditional analysis method difficult to face such huge data and lead to the insufficient mined information and lack of real-time information.Based on the above background,this paper mainly use the data mining technology under big data,it constructed a big data mining platform based on Spark to assist decision-making.The research states the scenarios and key technologies of different data mining technologies and discusses the overall architecture of ERP system,analyzes the business logic and business requirements based on machine parts manufacturing enterprises.Using Hadoop to build the Spark big data analysis platform and constructing the test dataset through the existing part order data in the ERP system,the Stacking fusion framework is realized.The comparative analysis shows that the Spark big data platform can improve the prediction speed under the massive data while the Stacking fusion algorithm can Greatly improve the prediction accuracy.An optimized genetic algorithm is proposed to screen the optimal features for different feature selection problems.The comparison of predictions on multiple data sets shows that the algorithm can screen out key features to improve prediction accuracy.Finally,the data in ERP system is introduced.The type and storage method,built an ERP data mining framework from data processing,data storage to task prediction,and introduced how to use the optimized genetic algorithm and Spark big data platform for enterprise ERP data mining tasks.Through big data tool Hadoop and Spark data mining platform,this paper makes data analysis and research on manufacturing enterprises' ERP system,solves the information flow phenomenon during the operation of enterprise,decreases the phenomenon of information isolated island,transforms the data generated by the enterprise into valuable information and make more correct decision for the enterprise personnel.