Research And Implementation Of Large Data Analysis Platform For Campus Wireless Network Log

In the rapid development of information technology today,through the Internet generated the amount of network log data also will be rapid growth.Today,web logs have become a typical representation of massive amounts of data.In the face of massive online log,the traditional single node analysis method has become weak or even unable to deal with.How to deal with these massive amounts of data,and to be used,is that we face a problem.This thesis studies the background and significance of Hadoop.In this thesis,the current popular large data technology in-depth study and research.The combination of large data technology and campus wireless network log analysis.This thesis first uses Flume technology to collect the log file data.Through the distributed file system HDFS on the collection of log data files for distributed storage.And then use MapReduce technology to collect the data to clean and deal with.And then use the Hive tool to analyze the data,and through the relevant technology to analyze the results into MySQL and other relational database.Finally,through the web interface to analyze the results of the show.In this thesis,through the above technology,the establishment of a campus based on Hadoop log analysis platform.The article intends to use the network log data analysis platform to collect and analyze the data in the campus network log and use the analysis results,hoping to bring some help to the management of the school.