Research And Implementation Of Campus Card Data Mining Based On Hadoop

With the increase of the various affairs system in Colleges and universities,the accumulated data of teachers and students is also growing rapidly,which has formed a typical big data environment.As a part of the digital campus,campus card stores the records of a variety of school activities,such as canteen food consumption records,boiling water consumption records,supermarket shopping records,library access records,electricity payment records,book lending records,sports records,etc..there are a lot of valuable information hidden in these records.it is difficult to find it by intuitive feelings.we must dig it out through the data mining methods.through the mining of these data,we try to discover useful knowledge,then the school managers can be more rational and have a more clear understanding of teachers' and students' consumption pattern and learning details.This would provide a valuable references for the rational allocation of resources in Colleges and universities,the plan and construction of campus and the management of teachers and students.In this paper,we use the Hadoop which is a popular data processing framework to clean,analyze and mine large scale data accumulated over the years in the campus card.First of all,this paper analyzes the importance of mining campus card data and the research status of related technology.Then it introduces the related hadoop technology(HDFS file system,Hive data warehouse,MapReduce distributed computing framework),FP-Growth algorithm and decision tree algorithm used in data analysis,Finally,using sqoop,Hive and other technologies,we build a data warehouse based on campus card data.On the basis of this,we counted dinning information,predicted the poor students in the school by C4.5 decision tree algorithm and excavated the students' consumption habits by FP-Growth algorithm.In the analysis of the campus card data,we first counted number in the school cafeteria meal.It allows us to discover the cyclical changes in the number of people eating in the cafeteria,but also have more intuitive understanding on the peak meal time.Then we used the C4.5 decision tree algorithm to predict the degree of poverty of the students,after pruning method,the accurate rate is nearly 85.4%,which has some reference value for the evaluation of poor students.Finally,we used FP-Growth algorithm to mine a large number of frequent patterns,And get a large number of association rules between students and businesses,merchants and Merchants.It makes schools and businesses to have a clearer understanding of the students' spendinghabits.At present,most of the information platforms in Colleges and universities are only concerned with the establishment of transaction management system,the use of data mining is rare.I believe that with the continuous development of big data,machine learning and other technologies,the campus data mining will play an increasingly important role in school management.