Research On Anti-analysis Technology Of Malicious Code And Corresponding Countermeasure

In recent years, the malicious code is to show explosive trend. Furthermore, theanti-analysis technologies such as code obfuscation and entry point obscuring make the detectionmore and more difficult. A mass of research indicate that static analysis has the defect of “whatyou see is not what you execute” and dynamic analysis faces multipath exploring problem. Theanti-analysis technologies mainly exploit the weakness of analysis means and then make themfailure or analysis time too long to give up.Based on the dissecting of anti-analysis technologies, this thesis proposes a method ofmalicious code similarity analysis based on API dependence to identify malicious code variants.First of all, to identify the implicit jump and eliminate corresponding obfuscation in disassembly,we improve the process recognition algorithm and then increase the accuracy of disassembly.Next, EPO (Entry Point Obscuring) is a technology which makes the multipath exploring moredifficult. Aiming at it, we design and implement an EPO scanner to determine whether EPO-typeinfection exists in an executable file or not. By getting the actual entry point, the relative path ofmalicious code is mainly concerned to enhance the purpose of dynamic analysis. Finally,traditional SCDG (System Call Dependence Graph) can not eliminate API feature obfuscationcaused by API noise and API rearrangement. In order to solve this problem, this thesis presents aprogram behavior description method based on SCDG composed of the control dependence andfour types of data dependence between APIs. A malicious code similarity analysis framework isdesigned based on API dependence. In this framework, API noise and API rearrangement areeliminated through data dependence analysis and control dependence normalization, whichimprove the accuracy of the similarity analysis of malicious code.The above method has been applied to a reverse analysis tool developed by our project team.Experiment results show that: compared to similar analysis means, the improved disassemblyalgorithm has a better effect on identifying implicit jump and EPO, and the malicious codesimilarity analysis framework can identify malicious code variants more accurately.