Research And Implement On Static Malware Detection System Based On Program Semantics

Malwares are used not only to show off hackers'skills but also to gain illegal incomes by criminals. Signatures of signature-based approach ignoring program behaviors are not general enough to match unseen malwares, each signature can detecte only one variant. Observation of the behaviors of an executable will always be limited to a certain time span or a specific execution path is the Achilles'heel of dynamic detection. So the above methods are inadequate in the face of emerging of obfuscated malware variants motivated by huge profits.Program semantics provides a formal model for program behaviors. So if took semantics into account, all possible execution paths of a suspicious executable could be checked and similarities among variants, hidden deliberately in the syntax level, would appear clearly. But the concrete, complete semantics of a program is not computable and all non trivial questions on it are undecidable theoretically. In this paper, model checking malicious behavior and OOA(Objective-Oriented Association) mining-based malware detection are unified into the same semantics-based framework by abstracting complete semantics into two different approximative semantics levels and observeing program behaviors through system calls.Essence of model checking malicious behavior is seeking a specific path (template of malicious behavior) along which a malicious act is achieved in all execution paths. In the referenced method, specifications of malicious behaviors are too complex because of the absence of data flow analysis. In this paper, semantics model is constructd by disassembling the suspicious executable firstly, and carrying out control flow and data flow analysis on CFG(Control Flow Graph). Malicious acts are abstractd into uniform finite state automatas, which are finally transformed into the corresponding CTL(Computation Tree Logic) formulas and validation process is completed by labeling algorithm. Our final experiments show that specifications of malicious behaviors can be simplified greatly as the result of the bringing in the data flow and control flow analysis.The principle of OOA mining-based malware detection is checking the existence of particular subsets(rules) in the whole set of APIs called by the suspicious program. In this paper, API set of an executable file is extracted by the IDA Pro plugin, rules satisfying specific goals are mined by our quick OOApriori algorithm, and used to classify samples. A significant reduction in the execution time of OOApriori, improvement of rules'quality, as well as an effective solution to the problem of sample sensitivity of the referenced method are accomplished through changing the rule-mining strategy, strengthening the rule-selection criteria, introducing accumulation mechanism of multi-rules and arbitration mechanism of multi-classifier.Semantics-based Malware Detection(SMD) system introduces heuristic detection module as a powerful remedy for the limitations of above two methods to packing and API hiding technologies. Finally SMD system can not only detect unknown malwares and those variants without signatures database, but also can offer the information about the sub-categories and malicious behavior possession of the malware in some accuracy.