Research Of Log Anomaly Detection And User Behavior Analysis Based On Web Application

The rapid development of mobile Internet and the popularity of mobile devices have led to an explosive growth in the number of mobile applications and web pages,driving the vigorous development of servers and back-end Web applications.Web applications have become an important entry and support for people to access Internet information services.As the Internet gradually infiltrates people's lives,people's requirements for service quality,user experience,and security status of web applications also increase.While application developers and website operators are rushing to bring products to market,they also pay great attention to product optimization to improve user stickiness.During product operation,how to collect log data,analyze product performance,detect abnormal status,and tap user characteristics in order to further improve operational performance,improve service quality,and provide personalized service,is a very important research direction.Based on the current situation and technical basis of performance and security monitoring as well as user behavior analysis research of back-end web applications,this thesis designs and implements the web application-based log collection,anomaly detection and user access behavior analysis platform,which mainly includes the following four parts:(1)Work on the solution of the collection and preprocessing of Web access logs,analyze the basic running status of the web application,and display the analysis results in the forms of visual charts.(2)Apply support vector machine to detect the log anomaly.In order to achieve a good detection effect,various research work is carried out,including researching the feature extraction method for the request URL of the Web log.After comparing the characteristics of normal request statements,SQL injection statements and XSS injection statements with statistical analysis methods,based on the differences between the three,feature selection and extraction are performed and a three-layer SVM detection model is proposed;Combining grid search and K-fold cross validation to find the optimal parameters;Normalizing the feature attributes of the data set.Finally,it is proved by experiments that these steps effectively improve the classification effect,and the three-layer detection model has certain practical significance.(3)An analysis process combining user clustering algorithm and association rule mining is proposed to analyze user access behavior.Firstly,the user access interest characteristics are extracted from the log,and the user access preference model is established.Then,an initial cluster center optimized K-Medoids algorithm is proposed and applied to divide users into groups according to their similarity of access interest.Further,use the improved FP-Growth algorithm with front-and-back component constraint to mine the relevance of web pages in the same group of users,and explore the rationality of the link relationship between the pages.(4)Design the architecture and operation process of the platform,and combine the popular micro-services to provide platform construction solutions based on Dubbo and Docker.At last an example of case is provided to vertify the feasibility and effectiveness of the platform.