Malicious URL Detection Based On DPI Data

With the development of Internet infrastructure,the scale of Internet users is growing rapidly.At the same time,there are also a huge number of malicious URL in the Internet,such as Trojan horse website,phishing site,malicious website and so on.Therefore,detecting and identifying malicious URLs in DPI data has become a great challenge.Network operators use DPI(Deep Packet Inspection)system to analyze network traffic,record the behavior of users,and produce a large amount of DPI data.If we can detect malicious URLs from DPI data,the harm of malicious URLs can be effectively curbed from the source and the network environment can be purified.Single computer can not handle the DPI data because of the huge data load,so in order to detect malicious URLs from the massive DPI data,we need to build a distributed computing system first.In this paper I ed and decomposes of detection process reasonably and designed a high-performance detection system using distributed technology.I developed the cluster monitoring system which make the cluster easy to expand and maintain.Secondly,in this paper I combined three filters based on manual rules,XgBoost model and natural network model,then I developed a high-performance malicious URL detecting system.The detecting system estimate the safety of the URLs in DPI data through relevant features of URLs crawling on Internet.Finally,I analyzed system performance by conducting pressure test and on the system.I write scripts for automated deployment and monitoring.The system has become a robust system which can handle heavy load and easy to maintenance.