Research On Covert Communication Detection Techniques Of Trojan Based On Deep Learning

With the popularization of the Internet in various fields of human society,as a medium for transmitting various important information,the information leakage on the Internet has occurred from time to time.Attackers usually use Trojan horses to steal sensitive information,and while various intrusion detection technologies have advanced,Trojan's anti-detection technology is also constantly evolving.Especially the application of encryption technology and tunnel technology makes Trojan horse communication highly covert and harder to detect.The detection technology for Trojan horse covert communication has become a research hotspot,but traditional payload signature-based methods cannot detect unknown Trojan horses.The method based on shallow machine learning has insufficient ability to characterize Trojan horse features.At present,there are no researches combining communication features and deep learning methods in the field of Trojan horse covert communication detection.Therefore,this article starts with the network communication and studies the Trojan horse encryption communication behavior and the tunnel communication mechanism.It is found that even if the communication is encrypted,the Trojan still has uniqueness in the communication behavior,but the distinguishability of various features is different.Therefore,a feature selection method based on the values of neural network weights is proposed,and a multilayer neural network model is established to detect encrypted Trojan communications.For the Trojan tunnel communication,it is found that the traffic payload has distinguishable characteristics,and the convolutional neural network is used to learn and identify the characteristics of the traffic payload.Through experimental analysis,the effectiveness of the two methods is verified.Based on this,the Trojan detection system is designed and implemented,and it has the ability to detect a variety of Trojan covert communications.The work done by this article is as follows:First of all,the relevant research of Trojan detection is analyzed.For traditional methods based on payload signature and shallow machine learning,the detection range for Trojan covert communication is limited.Focused on the analysis of the Trojan's communication process,as well as the communication mechanism of the HTTP tunnel Trojan horse,DNS tunnel Trojans and traffic encryption Trojans.This paper summarizes the characteristics of Trojan horse communication behavior and finds that there are distinguishable features in the transmission layer payload of the tunnel Trojan traffic.Secondly,in view of the problem that the previous feature selection method is not good enough,a method using neural network weight to filter the features is proposed,which can accurately select the feature.On this basis,a multi-layer neural network model for Trojan horseencryption communication detection is proposed.This model can improve the ability to learn the Trojan horse encrypted communication behavior pattern well,and can effectively detect the encrypted communication Trojan.Then,a detection method for Trojan horse tunnel communication based on convolutional neural network is proposed.Through research,it has been found that the existing detection methods for Trojan horse communication content,such as deep packet inspection,cannot detect newly emerged Trojan horses and multiple protocol tunnel Trojans.However,because the transport layer payload usually contains rich application layer protocol information and content information,the payload bytes can be converted into picture pixels.The convolutional neural network,which has achieved remarkable results in the field of image recognition,can be used to accurately identify multiple kind of protocol tunnel Trojan communication traffic.Finally,the Trojan horse covert communication detection system was designed and implemented,and the system performance was tested.The system is applied to the traffic flow data set collected in the laboratory network.The test results show that the system can effectively detect the Trojan communication traffic.Compared with the existing detection methods,the false positive rate of this system is reduced.And it has good detection capabilities for HTTP tunnel Trojans,DNS tunnel Trojans,and traffic encryption Trojans.