Agricultural Product Price Analysis And Forecast System Design Based On Hadoop+Spark Platform

In the agricultural product market,the price of agricultural products is affected by factors such as production costs and market supply.Moreover,in different seasons,agricultural products in different regions are affected differently.This makes the same kind of agricultural products,market prices in different regions are quite different,at this time the market information of agricultural products is particularly important.For ordinary farmers,traders and other people affected by price fluctuations of agricultural products,access to timely and effective agricultural market information can be concluded as soon as the price fluctuations,early prevention,and make appropriate decisions to avoid Major losses.Of course,if the access methods and methods are relatively closed,then when the price fluctuates,we will be caught off guard and lose a lot.Big data technology can process large amounts of unstructured data through distributed architectures such as Hadoop,MapReduce,and Spark,seeking deep data values and data relationships of big data,and making full use of the results of data mining and analysis to help decision makers.A rational,scientific decision.There is a huge amount of data in the agricultural product market.Through big data technology,we can obtain the insights and new values of agricultural products,early detection of market rules and market conditions.Based on the HP 360PGEN81 U server hardware platform and the Hadoop,Hive,and Spark distributed big data software platform deployed on it,this paper analyzes and forecasts agricultural product price data.The specific work is as follows:(1)Build Hadoop,Hive,and Spark big data frameworks;(2)Prepare Python3 program to crawl data on the provinces,cities,names,prices,price units,year,month,day,real-time,and types of agricultural products on the Agricultural Products website such as Nonghui.com and store them in MYSQL.(3)Export the information about the agricultural products that have climbed off from MYSQL into a text file and upload it to HDFS;(4)Map the data on the HDFS to the spark-sql database on the Spark-SQL program through the HQL program,and clean out the number of agricultural products in each province,the proportion of each province in the total number of the country,each The average,maximum,and minimum price of each day in the province,the average,maximum,and minimum prices for each province's monthly price;(5)Using the Scala language to carry out the Holt Winters(three-exponential smoothing)model to make it more able to handle a province column,and use themodified Holt Winters to forecast the price of agricultural products in the future;(6)Data from spark-sql cleansing and data predicted by Holt Winters is imported into the MYSQL database,and data visualization is performed using techniques such as SpringMVC,Ajax,and Echarts;...