Research On Dynamic Detection For Buffer Overflow Vulnerabilities Based On Simulator

Buffer Overflow is common software security vulnerability. Malicious users can use the overflow vulnerability of the procedure to attack the target computer systems, to access or corrupt sensitive information, or even worse, to control the host system to attack others. Among the hundreds of thousands of network attacks each year, about half of them are related to buffer overflow vulnerabilities, which cause more than 10 billion dollars'economic loss. Buffer overflow vulnerability has already become one of the most dangerous and common software security vulnerabilities.Nowadays, the methods of buffer overflow vulnerabilities detection are classified into two categories: static detection for buffer overflow (SDBO) and dynamic detection for buffer overflow (DDBO). SDBO methods for direct detection of source code generally suffer from higher rate of false positives. DDBO method which monitors program action in run-time, relatively speaking, has a lower false alarm rate and higher detection accuracy. However, there are obvious problems and shortcomings for DDBO: (1) It can't entirely get rid of the demand of Source code. (2) Because of the limitation of the operating environment, it can only test the application program, but can't do anything about the operating systems. (3) Difficulties in overflow location. (4) High dependent on the source code, it is difficult to ensure the coverage of the code.Focusing on the problems and shortcomings of the existing buffer overflow vulnerability detection, this paper presents an effective manner which is special for executable code, and runs target programs in the system-wide simulator under the dynamic-path-tracking monitoring based on the systematic study, comprehensive analysis and comparison of the DDBO approach. The main work and innovation includes five aspects.1) To present system-wide environmental simulator and to study the construction of the simulator control technique focusing on the problem of the weakness in controlling of the current dynamic environment and accessing to information. This technique enables the goal program to run in the environment completely controlled, and can obtain all kinds of state-level information at any time, including hardware state. It provides a good foundation platform.2) To put forward a dynamic instruction interpreter focusing on the problem that executable code is difficult to read and track. The interpreter will transform executable code into meta-instruction which is form-unified, structured, and easy-to-track. It not only improves the target instruction-set of the back-end path-tracking, but also facilitates the front-end extending to multi-platform environment.3) Aiming to improve code coverage and solve the problem on vulnerabilities location, to present Dynamic Path-Tracking (DPT) technique based on taint-propagation. DPT puts forward the rules of judging buffer overflow, the ways to locate the vulnerabilities, the condition of overrunning the buffer, and the algorithm for multi-path search which aims to improve code coverage.4) To design and implement a prototype system called SimDDBO, for the use of Simics which is a full-system simulator, which is based on the above-mentioned research results.5) To test SimDDBO system with the benchmark and three Win32 applications to verify the validity of SimDDBO,as a result of finding two new vulnerabilities tentatively in Baidu Hi which is given verification by BugTraq.To sum up, this paper is of important theoretical and application value in improving the degree of automation and the accuracy of detecting overflow vulnerabilities.