Design And Implementation Of HTTP Traffic Detection Platform Based On Spark And N-gram

In recent years,the popularity of Internet technology has brought great convenience to people's work and life.However,along with the de-velopment of the Internet,cyber security incidents have also exploded,jeopardizing the privacy and property security of Internet users.Using the machine learning model to perform security detection on HTTP network traffic can effectively identify attacks and protect user privacy and property security.However,the traffic detection method based on the machine learn-ing model also faces difficulties to extract features of traffic,making it dif-ficult to detect unknown attacks and difficult to detect large-scale network traffic data.This thesis studies the problems existing in the traffic detection method based on machine learning model,and proposes a feature extrac-tion method based on the improved combination of N-gram and Word2vec.The method can effectively extract the feature data of the traffic data,and solves the problem that the traffic feature extraction is difficult.A similar attack clustering algorithm based on vector space distance is designed.In this thesis,a similar attack clustering algorithm based on vector space dis-tance is designed and applied to the actual traffic clustering.The effective-ness of the algorithm is verified through the manual analysis of the cluster-ing results,which solves the problem that it is difficult to detect the un-known attack mode in the traffic detection method based on machine learn-ing model.Combined with the above feature extraction method and clustering algorithm.this thesis designs and implements the HTTP traffic detection platform.Through the actual environment of the platform,platform de-ployment and platform function testing,the accuracy and effectiveness of the platform are verified.