A Basic Data Platform And An Analysis System Of User Behavior On Campus Network

Recently,with the development of the technologies like big data,Internet of things and cloud computing,the networks of universities also come to a new stage,namely the era of big data.Big data has brought unprecedented changes to our lives,and as a new field for big data application,the technology will also bring great help to the students and the staff in the universities.The data source on campus network is rich and the amount of data is huge,we can deeply understand the behavior and habits of users on campus network and improve the level of school teaching and management by mining users' online data.In this thesis,the mutiple online data sources of users on campus network are deeply studied,as well as the overall design and the realization to the system have been carried on.On the one hand,the data warehouse is built through data preprocessing,and the basic data platform is realized.On the other hand,some new improved algorithms and formulas are proposed on this basis to further explore users' online behavior,and the user behavior analysis system is implemented.The main work and results of this thesis are as follows:The system is overall designed through studying and analyzing the multiple collected online data sources.Among them,in order to realize the integration and unified management of the online data of users on campus network,the data platform is designed,so as to build a data warehouse and realize the basic data platform.In order to mine the online data of campus network users and obtain the better clustering results,data mining method based on user filtering is adopted to analyze user behavior.After studying the characteristics of campus network users and the similarity algorithm of mobile trajectory,a new formula for users' online activity and a trajectory similarity algorithm based on improved LCSS(Longest Common Subsequence)are proposed.The rationality of the formula and the validity of the improved algorithm for clustering analysis are proved by experiments.In order to analyze users' online behavior comprehensively and deeply,we conduct general statistical analysis and cluster anlaysis on campus network users from the perspectives of users' online content,mobile trajectory and online activity and define the user behavior labels,then construct the label system.In order to select the effective features of user behavior,the feature selection algorithm is studied and designed.This algorithm combines the advantages of several feature selection algorithms and improves the accuracy of feature selection.In order to improve the accuracy of user portrait model,a two-level fusion frame algorithm is designed.By combining the advantages of the two integrated algorithms about Stacking and Bagging,this algorithm achieves a strong learner through two-level fusion to improve the accuracy of tag prediction.The experimental comparison proves that the user portrait model based on feature learning algorithm and fusion framework can effectively improve the accuracy of user attribute label prediction.In order to improve the performance of massive data mining on campus network,Hadoop technology is adopted to build a big data platform,and a basic data platform and analysis system of user behavior on campus network are designed and implemented.The basic data platform carries out data mining preprocessing for multiple online data sources,builds data warehouse,solves the problem of "information island",and realizes the integration and unified management of online data of campus network users.The analysis system of user behavior analyzes the real online data of users in our school,adopts different visualization methods to intuitively display the general statistics and data mining results of the campus network's online data,and designs the user label weight to highlight the user's online characteristics,accurately draw the user portrait,and make it convenient for university administrators to accurately locate and effectively manage users.In this thesis,the data mining technology is used to conduct in-depth mining of the online data of campus network users,and the results are analyzed visually and the user portrait is drawn,providing decision-making support for colleges and universities in resource allocation,talent training,student management and other aspects.In the future,more data resources and business systems will be integrated to provide more accurate services for analyzing user behavior characteristics.