PyReunpacker: An Automatic Unpack System Base On Dynamic Analysis

At present,a large number of malicious software and benign software use packer to protect themselves.Data shows that 58% of the malwares downloaded from the Internet are packed,and 35% of these 58% packed malwares use custom packer.Therefore,it is becoming more and more necessary to automate the unpacking of the packed software.This article designs and implements PyREUnpacker,an automated unpacking system based on dynamic analysis.PyREUnpackeruse dynamic analyzing the the sample,put it into a virtual machine,monitoring the unpacking state of the sample and saving it.By using the WX-then-CALL(Written-then-Executed-then-CALL)model and the maximum ESP(Extended Stack Pointer)search algorithm,PyREUnpackercan automatic unpack packed sample when the packing algorithm used for the sample is unknown.PyREUnpackeris mainly composed of 5 modules: packer detection module,unpacking analysis module,memory dump module,image process module and PE fix module.The packer detection module calculates the special entropy of the sample to be tested,and calculates the corresponding packed probability according to the obtained entropy,and judges whether the sample is packeded by the packed probability; the unpacking analysis module runs the packed sample in a virtual environment.By monitoring the memory write,instruction execution,instruction jump and API call behavior in the process of running,the unpacking analysis module can judge the unpacking progress of the sample,and save the memory data through the memory dump module.After the dynamic analysis is finished,the unpacking analysis module will also use the WX-then-CALL model and the maximum ESP search algorithm to find the OEP; the memory dump module will analyze the process memory space and save the available memory area and information for the processing of the image process module; the image process module first filters the memory images generated at different points in time,selects the appropriate basic PE image file,and then merge the data in the heap according to the location of the OEP,and to obtain a PE image file; finally,the PE fix module will fix EP,IAT and section protect of the PE image file.The test results show that PyREUnpacker's packer recognition accuracy rate reaches85.0%,which can detect most packers.And the unpacking success rate reaches 82.5%,which is higher than the automated unpacking tool for comparison tests,which is 53%.The result shows that PyREUnpackercan automatically unpack most of the packers and improving the efficiency of malicious code analysis.