Design And Implementation Of Campus Card Consumption Analysis System Based On Hadoop

With the rapid development of large data technology,with the integration of the field is more and more,commercial online shopping user data analysis,bioinformatics gene comparison,log analysis of the system operation and maintenance,etc.,large data technology Application value in many aspects have been reflected.The campus card management system,in the university information management,made a very important contribution,but for the campus card system stored in a large number of data and the data contained in a large number of business information and management information,our analysis It is relatively slow.The arrival of large data technology has undoubtedly led us to study the campus card in the data brought about a new technical support,with large data technology to analyze the data,and in which to identify potential links and laws for the campus construction and management work,has an invaluable decision-making reference.Therefore,the establishment of a large data platform based on the campus card consumption analysis system has practical significance."This thesis analyzes the transaction data of campus card system as the object of analysis and uses the large data related technology of Hadoop ecosystem to analyze and study the consumption analysis system which is assisted in the campus card management system.First,the core applications of Hadoop platform HDFS distributed storage system and MapReduce distributed parallel computing processing mechanism,Hive data warehouse tools and Sqoop data transfer tools.And then the school decision-making level and the staff to explore the actual needs of consumption analysis to understand and decompose,get the consumption analysis system needs analysis.And use the commonly used Hadoop business processing framework to design and implementation of demand analysis,the use of Sqoop tools to the card transaction data from the relational database into the HDFS storage system,the use of MapReduce technology for data preprocessing,and then use the Hive tool for data statistics Analysis and re-import the MySQL database for web display.I also use the MapReduce technology alone on the transaction data for another analysis of technology to try and explore.Finally,constructive advice on consumer information that can be provided to school decision-makers and staff is initially available.The work of this thesis shows that the design and implementation of Hadoop-based campus card consumption analysis system is very meaningful.At the same time,the present work also laid the foundation for the comprehensive information analysis and data mining of the school in tomorrow.Platform and provide more far-reaching decision-making recommendations for effective exploration.