The Research On Buffer Overflow Attack Defense Based On Trusted Execution Environment

With the promotion of modern IT technology, smart mobile devices based on embedded technology are widely used in many fields of our lives, such as finance, military, industrial control and network communication, etc. In these fields, the security problem of embedded system is becoming more and more important. As one of the most common software vulnerabilities, buffer overflow poses a grave threat to embedded system security, thus improving the ability to defense buffer overflow attack of the embedded system has become an important topic.This topic is a part of the project called "TEE-the trusted execution environment" of Beijing Watchdata Co. Ltd., which focused on the study of buffer overflow attack defense in embedded systems.This paper analyzed the principles, methods and the damages of buffer overflow attack, conducted researches according to several kinds of buffer overflow vulnerabilities of the trusted execution environment, and designed defense methods to BSS buffer overflow, heap overflow attacks and stack overflow attack. These defense methods were verified and realized. The main research work is as follows:(1) This paper analyzed the security problems of current mobile Internet, studied the problem of buffer overflow attacks in TEE, the basic principle of buffer overflow attacks and the impacts of attacks;(2) In view of the principle of BSS overflow attacks, stack overflow attack and heap overflow attacks, several prevention technologies are studied:method based on data protection technology, method based on the attack code protection technology and defense method based on boundary detection technology. According to their advantages and disadvantages, an overall security architecture system is designed;(3) According to the analysis of the BSS overflow attacks, the MMU-based defense method to buffer overflow attack is designed and implemented, which resolved the BSS overflow attack problem effectively. Through further study on the buffer boundary detection, the defense methods of BSS are improved, which the empty map idea is applied in. This defense method not only reduces consumption of system memory, but also guarantees the security of the system.(4) Through the study of heap memory management mode and the features of the heap memory buffer overflow attack, this paper proposes a new way of heap memory allocation, which has been realized to prevent heap memory overflow based on this design.(5) By analyzing the features of the stack overflow attack, the stack frame switching method was designed to protect the important data in the stack frame effectively and to ensure that the overflow attacks in the stack frame will not spread to other stack frames.