Mass Sales Data Processing Platform Design And Implementation

In recent years,with the rapid development of Cloud Computing and Big Data technology,data analysis and data mining have been taking more and more important position for enterprise development.Based on data processing results,it can help optimize current business workflow,even provide data support for determining the direction of business development in the future.In various industries,data is considered to be the treasures of enterprise,because data makes enterprises more competitive through acquiring potential values from data with new data processing technology.During business developing,there is no doubt that enterprises have accumulated massive sales data including customer information,product information,contract information etc.however,these data has been not developed completely against its value,only for performance assessment and information query.before Cloud Computing and Big Data technology have emerged,traditional data processing model can not process so much data within appropriate time users expect.This paper is going to design and implement a high effective and useful data processing platform with Cloud Computing and Big Data technology,executing data analysis and data mining on sales data.With this data processing platform help,it is possible to simplify and optimize business workflow,be easier to process data analysis and acquire necessary data for making business determinations.This paper,first of all,introduces study background,main tasks and technologies used in the data processing platform implementation.Based on business characteristics and business requirements,also introduces the platform design and implementation approachs.Then picks up two important functions,big data analysis function and data aggregation fucntion,to specifically explain their design and implementation.At last,system test results prove that on the platform data processing function can be executed with high efficiency.Using Cloud Computing and Big Data technologies not only improve utilization of system resources but also the spent time for processing massive data is acceptable to users.In this paper,data analysis and data miming are implemented by the MapReduce of the Hadoop,divide the whole job into several tasks and execute in parallel.The data will be processed,exists in different system,so moving the data into the data warehouse of the platform has to be done before data pcossing and data mining.the data warehouse is implemented by the HDFS of the Hadoop,has to synchronize with other system to make sure that the data in the data warehouse is valid.This data processing platform is a web system providing users web interfaces.Users submit data processing requests on web interfaces,also can check the data processing results from web interfaces.web interfaces are designed to be easy to use,data processing results are shown users with chart,table etc.downloading data processing results is also supported.Currently,the system design and main function implementation have already finished,some supporting fuctions are still ongoing.So far,the feedback from users who use this data processing platform,is very good.According to the test results,the high efficiency of data processing on this platform has been proven as well.