Research And Implementation Of Agricultural Products Circulation Multidimensional Analysis System Based On Hadoop

In recent years,catalyzed by the Internet,the circulation mode of agricultural products in China has undergone great changes.The traditional offline circulation mode has been transformed into the "Internet +" agricultural products e-commerce mode and the peasant household origin live supply mode.At the same time,the circulation data of agricultural products also presents a massive outbreak trend.How to dig deeply and effectively integrate the circulation data of agricultural products scattered everywhere,conduct scientific multi-dimensional analysis and research,and break the problem of "information island" of agricultural products? At present,it has become the focus of the government and enterprises.In view of the above problems,this thesis puts forward the research topic of multi-dimensional analysis system of agricultural products circulation based on Hadoop distributed computing platform,and uses big data technology to solve the problems of fusion,storage and multi-dimensional analysis of massive agricultural products circulation data.The main research work of this thesis is as follows:(1)Through technical research,based on Hadoop ecosystem technology,the environmental deployment of relevant technologies is completed,and the agricultural product circulation data sets from heterogeneous data sources are extracted,converted and loaded to Hadoop distributed File System(HDFS)for storage.(2)A five-tier data warehouse architecture model based on Hive was proposed and designed,which realized the storage management of data granularity between different levels in the data warehouse and accelerated the speed of data query.(3)The optimization scheme based on Apache Kylin multi-dimensional analysis is studied.By integrating Apache Kylin,Hive and HBase technologies,the dimensional data model is taken as the access source respectively,and the Cube construction algorithm is used to optimize the data Cube prediction and realize the data subsecond query analysis.(4)According to the results of multi-dimensional analysis,data visualization is realized by using the program of Echarts 4.0.Finally,the usability of the system is verified by the functional test,which has a certain practical value.