Malicious Code Analysis Based On Operation Virtualization And Temporal Logic

Malicious code has been becoming a great security problem. Running on the objective machine, malicious code could infect document and file, damage normal system and networks, steal personal confidentianls or even economy and military secrets, lead to inevaluable loss of money and security. Therefore, the malicious code is always the security concern. In order to identify and prevent malicious code, malicious code analysis based on behavior is an important way, because it could provide timely information of damage assessment and recovering system. This analysis technology has been the research focus in the anti-malware domain. And how to improve the efficiency and accuracy of malicious code analysis is the focuse of the research.Usually, the analysis method of malicious code based on behavior executes malicious code in a simulation system or a virtual machine. This method analyzes the malicious behavior of malicious code through monitoring their system API calls, and it can make system state rollback through the system snapshot. However, the execution environment provided by the simulation system or virtual machine can be detected by some malicious code easily. And the way of rollback based on system snapshot is more time-consuming, which can affect the efficiency of analysis work. The existing analysis methods only describe the single calling relationship between the malicious code operations. However, there are several relationships between the malicious code in practice, such as temporal relationship, the relationship between subject and object. The method will certainly reduce the accuracy of malicious code analysis if it can not fully describe the relationships between these operations.To deal with these deficiencies, the paper proposes an operation virtualization method. This method provides a real system environment for malicious code to run, and ensures the execution environment is not detected, so that malicious code can run normally and the analysis tool can get a complete native API calling sequence. This method can achieve rapid system state rollback through deleting malicious execution traces directly. Experiment shows that the method can improve the efficiency of malicious code analysis. Meanwhile, the paper proposes a method of behavior analysis based on temporal logic. This method describes several relationships between malicious code operations through formal logic, and could verify whether a target file has malicious behavious through the design of appropriate behavior deciding algorithms. Experiment shows that the method can describe malicious behavior accurately, and decide malicious code and its variants effectively. It has high accuracy and low false alarm rate. Compared with the existing main products of malicious code analysis, the method in this paper can extract and determine malicious operations more detailed and accurately.Based on the above methods, the paper designs and implements an automated malicious code behavior analysis system named OV_MAS. First, the task dispatcher in this system opens the target file in sequence. And the system obtains the important Native API calling information of the running target file through the code operation monitor in kernel. Next, the system analyzes the malicious behavior of the target file through behavior detection engine and malicious behavior database. Finally the analysis report is given, which contains the malicious behavior, the target process and the API calling fragments satisfying the malicious behavior.