Research On Detection Of Vulnerability Attack Inside Network Flows

With the development and popularization of computer networks, vulnerability attack has become one of the main problems of network security due to its prevalence, widespread impact, serious consequences and other characteristics. At present, the relevant security researchers have proposed many defensive strategies and detection mechanisms against vulnerability attack, but the network security situation is still grim. Therefore, how to effectively detect vulnerability attack inside network flow is still one of the core issues of information security.The detection of vulnerability attack can be carried out from external and internal of programs. For internal way, the defender systems determine whether the program is under attack through checking if its control flow and other sensitive data have been tampered at run-time. For external way, the defender can find and prevent the attack in time based on shellcode detection inside the input stream. In contrast to the internal detection, shellcode detection is a more effective means to defense against vulnerability attack.Currently, shellcode detection methods can be generally divided into static and dynamic methods. The static method has the advantage in speed, and the dynamic method obtains better detection accuracy. However, many existing detection mechanism is purely static or dynamic methods, sacrificing efficiency or accuracy of the detection, because all of them have not combine with the two detection methods'advantages. In addition, the use of multi-state, deformation and ROP and other emerging technologies brought new challenges to shellcode detection, which means that simple static or dynamic methods unable to meet the current needs of detection against vulnerability attack.In this paper, we first describe the basics of vulnerability attack from the attackers' perspective, then analyze and make a comparison on technical characteristics between common attack approaches, after that we propose three new heuristics for shellcode detection through a deep study on the realization principle and detection technology of the shellcode. In the last, we propose a hybrid method of shellcode detection, combining with both the static and dynamic approaches. In summary, this article performs in-depth research on shellcode principles and the relative detection technology, detecting vulnerability attack by external way, and we achieve two major contributions:1. Propose three new heuristics for shellcode detection, including LEH, FDR and TIAT through which we can effectively enhancing the existing methods'effect of detection against staged shellcode.2. Propose a new hybrid shellcode detection method, combing with both the static and dynamic approach. Unlike simple static or dynamic methods, the new method find an effective way to balance accuracy and efficiency, which means the new method can detect shellcode accurately whit a relative less time-cost.Experimental results show that the proposed heuristics can efficiently identify corresponding staged shellcode, while the new method has good detection efficiency.