Research And Implementation Of Big Data Analysis System For Entertainment Consumption

In today's society,people's standard of living is gradually rising,and at the same time,consumer spending will leave corresponding footprints,and will produce corresponding records.Because people consume a large amount every day,they will produce a large number of consumer records.If you want to quickly analyze the consumer concerns and related behaviors from the vast amount of data,it is difficult to meet the demand with traditional methods of analysis.Traditional methods are time-consuming and difficult to achieve the desired results.The consumer record contains many consumer information,if through the analysis of specific tools to analyze these data.it will analyze many useful information from the data,Including consumer habits,sales of goods and so on.Today's popular computing frameworks are the Hadoop computing framework and the Spark computing framework.The main purpose is to analyze and deal with the mass data.This article uses the HDFS in Hadoop and Spark computing framework to analyze the entertainment consumption records of the majority of consumers,and draw the actual situation of entertainment consumption.Firstly,this dissertation summarizes the research status of data processing and analysis system,and introduces the related theory and technology;Secondly,the system requirements are analyzed in detail,and the overall design framework of the system is derived.The main functions of the system include aggregation statistics,random sampling,Top 10 hot categories and Top 10 active session;Then,the design idea and main functions of the data acquisition algorithm are designed in detail,and the database is designed in detail;Finally,the system environment is built,the data acquisition algorithm and the main functions of the system are implemented.After the implementation,the functional test and non functional test are carried out.The running results show that the system can operate normally and achieve the desired goal.