Design And Implementation Of Data Warehouse For Electronic Business Platform Based On Big Data

Under the background of "Internet Plus",W Company,as an e-commerce platform,is undergoing a rapid development.With the geometric growth of the business,the support of company's original data application for the company's business is in straitened circumstances.Relying on big data technology is a way of reform.In the construction of enterprise big data platform,the company hopes that big data system can transfer to real-time data,so that it can better and faster provide solid technical foundation and backing for company's operation and development.At present,the company's main KPI involves sales volume,the number of customers,flow and orders.Through these indicator,operators and corporate executives can intuitively understanding the status of the company.Because the big data at this stage mainly provide the offline data,the business staffs and decision-makers are lagging behind in understanding the company's business information,so that unable to make timely business adjustments.Therefore,the business staffs of each department and company's executives put forward higher requirements for real-time data.So it is necessary to build real-time data application to provide more efficient business data for the company.Because big data team's exploration and application of real-time data is still in the initial stage,it needs to gradually promote the collection,processing and storage of real-time data.In order to maintain normal operation without affecting existing business,in the design,the application layer should provide smooth transition from offline data to real-time data,so as to normally and orderly promote the whole project.The paper put forward to a data board solution which is suitable for the e-commerce platform,proposed the logic structure with front-end web,intermediate layer API data interface and back-end offline calculation and real-time computing cluster.Hadoop distributed technology structure is designed in offline terminal and the technology structure of real-time terminal is designed based on kafka,storm,redis and hbase.The real-time data board is designed and accomplished in the system based on the framework design,which connects the links of real-time data acquisition,processing and achieves such functions as operational monitoring panel,category panel,schedule panel,preheating activity panel,pre-payment panel and data configuration panel.It can be used to display the real-time on the e-commercial platform so as to provide operators and decision makers of e-commerce with the accurate data in multi-directions by the real time means so that they can make management and decision in more effective way.The system has the following characters: data visualization,real time data display and the combination of offline data display and real time display.Operators can obtain the data visually through graphs to obtain the latest operation data and indexes by real time means,and the real-time data panel also provides the historical data that is used to compare with the real-time data for analysis.Finally,it tested and applied of e-commerce platform data board.The test result shows that the system performs well in both function and performance testing.In the continuous improvement,the system periodically meets the needs of business personnel and executives,providing powerful data support for company's business and operation.