Research Of Binary Program‘s Buffer Overflow Based On Dynamic Taint Analysis

Software vulnerability exploiting has become an important research aspect in computer security, because of the inevitability of software security vulnerabilities and the enormous harm it brings. The basic thought is detect whether the program being tested has vulnerabilities by analyze the program’s source code, binary code and the corresponding information in its host system when the program executed with different kinds of techniques. Technique based on the dynamic taint analysis as a main software vulnerability detective method has a well development recent years, however its detection efficiency is low in consequence of its detective of only execute a single path, low coverage and hardly get more information from a binary program’s internal structure. GA(Genetic algorithm) has been caused attention widely for automatic generation to reduce the number of grade and improve the test efficiency. Currently it’s inefficient and hardly applied to practice that most of the GAs only solve single-path or consider path coverage ratio as criteria. Consequently the research based on multi-path GA as the path selection algorithm has a great and practical significance to improve the efficiency and ensure the coverage ratio.In this paper, we approved a detective method of binary program’s buffer overflow vulnerability based on dynamic taint analysis after analysis the principle of buffer overflow deeply and extensive research on domestic and foreign technique on software vulnerability, and cover its shortage by taking an improved heuristics search algorithm as the test case generation. The algorithm improve the program’s path coverage and branch coverage, and therefore improve the efficiency. The main research is as follows:1. Design and implement a binary dynamic taint analysis model. We designed a dynamic taint analysis model by establishing explicit tainted objects、designing the method of taint transmission and the binary code software overflow vulnerability monitoring mode, which is to monitor the software vulnerability behaviors caused by the dangerous function calls. We implement it with Pin, a dynamic binary instrumentation tool.2. Design and implement an intelligent path selection model. We take the heuristic search algorithm multi-path GA as the path selection algorithm. We compared different execute path’s similarity with path coverage and branch coverage, redesigned the fitness function and the operating operator to improve the ability of the GA’s coverage and research, proposed an improved GA to raise the ratio of path coverage in this paper. And finally implement it with Java.3. Propose a prototype system IFANG based the above models. IFANG can detect and analysis the binary application’s vulnerabilities by the interaction of the different modules in the system. Experimental results show that IFANG can cover the shortage of the dynamic taint analysis, and has a good efficiency and practice automated test data generation algorithm than others in solving the problem of multi-path data generation.