A Study On Packed Detection And Exuviate Based On Weighted Euclidean Distance

Software protection has entered a new era because of the appearance of shell. Packing technology is already widely used in software protection. In software reverse analysis, software cracking and malicious software detection, unpacking seems to have become one of the most important parts. Packed software detection is the premise of unpacking, which laids a solid foundation for the work of subsequent unpacking. Nowadays, relatively mature and widely-used shell detection algorithms are mostly based on the shell characteristics, by establishing shell feature library and querying feature string to determine whether files are packed, however, these algorithms can not detect feature string outside shell feature library, and are very easy to be deceived. Our research focuses on studying the correlative technologies of software unpacking algorithms.Firstly, the paper introduces the status quo of packed PE file detection, unpacking technology and some relevant software at home and abroad, and then analyzes them deeply; Describes the image optional head, section, import table, resources and other relevant structures in detail, and highlight the PE file format.Secondly, the current PHAD use shell detection method based on feature vectors to increase the accuracy rate. However, its feature vectors can not exactly reflect the feature of shelled file and each vector element in the have the same weight. As a result, PHAD has a high false negatives rate and false positive rate. By introducing strategies such as increasing the elements of feature vectors, excellent sequence correlation method to determine the weight values and weighted Euclidean distance to calculate distance, proposed a shell detection algorithm based on weighted Euclidean distance. Shell detection algorithm is designed to address these deficiencies in PHAD; the algorithm is consisted of two phases: determining threshold and shell detection. The two phases both includes corresponding preparation to determine whether the file satisfies the PE file format and calculate value of NCV. In the threshold determining phase calculate the distance according NCV of the sample, then choose the minimum as threshold and provides it to the shell testing phase. In the shell detection phase compares the distance value with the threshold, then determines whether the file is packed or not. The test result shows the false negatives rate, false positive rate of new algorithm account for 1/16 and 1/8 of PHAD and the accuracy rate of shell detection increase 12%.Finally, analyzes and makes research on the process of shell loading and implementing, proposes a memory-monitoring-data-collecting algorithm which can efficiently extract the hidden information in packed program. The algorithm traces memory during the executive process of packed program, collects data that packed programs need to write when memory write operation occurs. After determining test space based on the data collection algorithm, combining the shell detection algorithm and ESP law method of manual unpacking technology, designed an original- program-entry-point-determined algorithm. The algorithm obtains entry point of packed program, performs dump operation and then completes the unpacking.This paper has a certain theoretical reference value and practical value to further research on unpacking PE files.