The Design Of Web Log Analysis System Based On NoSQL

Nowadays the Internet develops rapid, and Inetrnet companies which are accessed by tens of thousands or even millions of users also face great challenges. In order to provide high quality of service and understand the users' access features as well as users' needs, it's urgent to analyse the users' access behavior. Improving the site according to the regular pattern of study the user behavior will attract more users.In this case, customers will get a better experience and companies will obtain more opportunities. Therefore, Web log analysis is generated. The so-called Web log analysis is to collect all log information when users visit the web site, then to conve,clean and mine the log data.As the growth of Internet companies and the expansion of the size of the application system, the magnitude of the corresponding log information is also synchronized growth. Traditional stand-alone log analysis program can not meet the needs of today's Internet companies log analysis. Large-scale data processing platform becomes ideal platform for log analysis.The Hadoop, distributed computing framework, and MongoDB, distributed database, are studied in this paper. Efficient web log analysis program is designed based on Hadoop and MongoDB. During algorithm design, the mapping table document optimization is proposed in this paper to improve the efficiency of log analysis. Partial merger of key algorithm has been designed to reduce the amount of data to improve the efficiency of writing to files and transmission.This paper describes the design and building of client application framework based on SliverLight in order to develop the client's specific application modules efficiently.